Data Reliability Engineer

Key Responsibilities



*


*


*


*


*


*


*


*


*


*


*


*

Essential Skills - What You'll Bring

• Proven experience in data engineering, platform engineering, site reliability engineering, DataOps, or a closely related role focused on data platform reliability and operations.

• Strong hands-on experience with Azure-based data platforms, particularly Azure Databricks and core Azure data services such as Data Lake Storage, Data Factory/Synapse, and analytical stores, with familiarity of equivalent services in AWS.

• Strong understanding of modern data platform architectures, including data lakes, warehouses or lakehouses, orchestration frameworks, transformation pipelines, streaming services, and analytical consumption layers.

• Experience designing and implementing monitoring, observability, logging, alerting, and incident management approaches for Databricks workloads and Azure data services, using tools such as Azure Monitor, Log Analytics, or similar.

• Strong understanding of data quality, reconciliation, validation, and lineage concepts, and practical experience implementing control frameworks that protect critical data flows and products.

• Hands-on experience with Azure Purview or comparable data cataloge and lineage tooling, including configuration of collections, classification, ownership, and lineage for key datasets.

• Good understanding of reliability engineering principles such as availability targets, resilience patterns, recoverability, service health indicators, and operational readiness assessments.

• Experience using scripting and automation (for example, Python, PowerShell, or similar) to remove operational toil, improve repeatability, and strengthen recovery and deployment processes.

• Ability to diagnose and resolve complex issues that span data pipelines, integrations, cloud infrastructure, configuration, and source or downstream systems, and to drive pragmatic remediation.

• Strong collaboration and communication skills, with the ability to work effectively across data engineering, architecture, cloud platform, security, delivery, and operational teams, and to explain issues and trade-offs clearly to technical and non-technical stakeholders.

• Experience with Azure-native observability tooling and patterns for Databricks and the wider Azure data stack, and exposure to equivalent AWS monitoring approaches.

Desirable Skills - What Makes You Stand Out

• Experience tuning and optimising Databricks jobs, clusters, and workflows for performance, cost efficiency, and reliability in production environments.

• Familiarity with CI/CD practices, infrastructure as code, and automated testing for data platforms, supporting safe, repeatable, and low-risk changes to data services.

• Experience working in regulated or contractually governed environments such as public sector, health, employment services, or similarly controlled industries, where se...