Lead Platform Engineer

Description

Lead Platform Engineer

This position reports into Senior Manager, Data & Analytics Platforms and is based at Bangalore, India.

What you will do

The Lead Platform Engineer is responsible for leading the design and optimization of our Databricks Lakehouse platform, ensuring it meets the organization's performance, scalability, and reliability needs.

You'll collaborate with cross-functional teams to deliver high-impact projects, mentor team members, and drive the adoption of best practices.

Key Responsibilities

• Serve as the technical lead for Databricks-related projects, ensuring best practices and adherence to architecture standards.

• Provide advanced troubleshooting and performance optimization for Databricks workloads.

• Lead the development of scalable and reliable ETL pipelines, data models, and workflows using Databricks and related technologies.

• Monitor and improve the security, performance, and cost-efficiency of the Databricks platform.

• Stay updated on emerging Databricks features and implement them to enhance platform capabilities.

• Engage with cross-functional teams to understand business requirements and translate them into technical solutions.

• Mentor junior engineers and provide guidance on complex technical challenges.

• Establish and document coding standards, deployment processes, and development best practices.

• Conduct code reviews and technical training sessions for the team.

• Take ownership of critical initiatives and ensure timely delivery of high-quality solutions.

• Identify risks, propose mitigation strategies, and resolve bottlenecks in ongoing projects.

What we are looking for

Required Qualifications

• Proven expertise with Databricks, Spark, and Delta Lake, including performance tuning and optimization.

• 6 years of experience in data engineering or data platform roles, with at least 2 years in a lead role.

• Proficient in Python, SQL, and other programming languages used in Databricks development.

• Experience with cloud platforms (e.g., AWS, Azure, GCP) and their integration with Databricks.

• Strong understanding of data governance, security, and compliance frameworks.

• Excellent communication skills and the ability to convey technical concepts to non-technical stakeholders.

Preferred Qualifications

• Experience with data platform migration projects or integrating Databricks with other tools in the modern data stack.

• Familiarity with CI/CD practices for data engineering pipelines.

• Knowledge of data visualization tools (e.g., Power BI, Tableau) and their integration with Databricks.

Lead Platform Engineer

This position reports into Senior Manager, Data & Analytics Platforms and is based at Bangalore, India.

What you will do

The Lead Platform Engineer is responsible for leading the design and optimization of our Databricks Lakehouse platform, ensuring it meets the organization's performance, scalability, and reliability...