Lead Software Engineer - DataBricks, Spark, Terraform

We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.

As a Lead Software Engineer at JPMorganChase within the Corporate - Employee Platforms, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way.

As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm's business objectives.

We use AI-assisted development as part of our day-to-day workflow, including GitHub Copilot for coding and native Databricks tools such as Databricks SQL Assistant and Genie to accelerate development, troubleshooting, and self-service analytics-while maintaining strong engineering controls and review practices.

Job responsibilities



* Platform leadership & architecture - Define and drive the technical roadmap for our Databricks lakehouse/database platform (ingestion, storage, modeling, serving) with clear standards and reference patterns.


* Data modeling & database engineering - Design curated datasets (e.g., medallion architecture) using Delta Lake, dimensional/semantic modeling where appropriate, and enforce consistent naming, partitioning, and performance practices.


* Reliability & operations - Build for availability and predictable performance; establish SLOs, runbooks, alerting, incident response, and operational hygiene for pipelines and SQL workloads.


* Security, governance & access controls - Implement and maintain strong governance (e.g., Unity Catalog), least-privilege access, auditing, data classification, and lifecycle management.


* Performance & cost management - Tune Spark/SQL workloads, optimize clusters/warehouses, manage caching and storage patterns, and implement cost observability/chargeback as needed.


* Engineering excellence - Set standards for code quality, testing, CI/CD, branching strategy, documentation, and review.

Establish reusable libraries/templates and enforce consistency across teams.


* Mentorship & collaboration - Coach engineers, lead design reviews, and partner with stakeholders to translate business needs into scalable data platform capabilities.


* Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.


* Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automati...