Job Title

Dev Ops Engineer

Job Description Summary

Job Description

Job Description and Purpose
Weâ™re seeking a hands-on DevOps Engineer to build and standardize Azure DevOps automation for data/analytics deliveryâ”primarily focused on Databricks, while also supporting deployments across other Azure components. Youâ™ll create repeatable CI/CD patterns, improve deployment reliability, and enable teams to ship changes safely across environments.
KEY ACCOUNTABILITIES
· Design, build, and maintain Azure DevOps CI/CD pipelines (YAML-first) for Databricks and Azure platform components.
· Implement automated deployments for Databricks assets, including notebooks, repos, jobs/workflows, libraries, and configuration across dev/test/prod.
· Develop reusable pipeline templates, shared task groups, and standardized âœgolden path❠patterns for engineering teams.
· Implement Infrastructure as Code (IaC) for Azure and Databricks resources (e.g., Terraform and/or Bicep/ARM), including environment promotion strategies.
· Establish release practices: branching strategy, artifact/versioning, approvals, change control, and rollback/restore patterns.
· Integrate secure authentication and secret management with Azure Key Vault, service connections, managed identities/service principals, and least-privilege RBAC.
· Automate deployments/integration for related Azure services as needed (commonly including):
· ADLS Gen2, Key Vault, Azure Monitor/Log Analytics
· Azure Data Factory (if in use), Event Hubs/Service Bus
· Azure Functions/Logic Apps (where applicable)
· Improve reliability through quality gates: automated tests, linting, policy checks, and deployment validations.
· Implement monitoring/alerting for pipeline health and Databricks job outcomes using Azure Monitor, Log Analytics, and pipeline telemetry.
· Produce clear documentation: runbooks, pipeline guides, onboarding material, and operational procedures.
JOB REQUIREMENTS & QUALIFICATIONS
⢠3â“7+ years of experience in DevOps / Platform Engineering / SRE with strong delivery ownership.
⢠Demonstrated expertise building Azure DevOps Pipelines (YAML), including multi-stage pipelines, variable groups, approvals, environments, and artifacts.
⢠Experience automating deployments for Databricks using one or more of:
o Databricks CLI and/or REST APIs
o Terraform Databricks provider
o Packaging/release patterns for notebooks/jobs/libraries
⢠Strong knowledge of Azure fundamentals: networking, RBAC, Key Vault, storage, and monitoring.
⢠Proficiency with scripting/automation (Python, PowerShell, Bash) and Git workflows.
⢠Hands-on experience with IaC (Terraform preferred; Bicep/ARM acceptable). INCO: âœCushman & Wakefieldâ