Lead Software Engineer - Python, Databricks and AWS

We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.

As a Lead Software Engineer at JPMorgan Chase within the Corporate Technology, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way.

As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm's business objectives.

Job responsibilities


* Architect the lake house: design bronze/silver/gold (or equivalent) layers, domain data products


* Deliver ingestion at scale: implement resilient ingestion from AWS sources into Databricks (batch + streaming), including CDC where needed.


* Build maintainable pipelines: use Delta Live Tables (DLT) and/or standard Jobs with clear modular structure, testing, and documentation.


* Operational excellence: productionize workloads via Databricks Workflows/Jobs, robust retries, checkpointing, idempotency, and safe re-runs.


* Governance by design: enforce least privilege, data classification (PII), auditing, lineage/metadata, and controlled sharing/consumption.


* Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.


* Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.


* Performance & cost management: tune Spark/Delta workloads, right-size clusters, optimize storage layout, and manage job/warehouse spend.


* Lead and mentor: set engineering standards, run design reviews, drive code quality, and upskill engineers in Spark/Databricks best practices, cross-functional delivery: translate stakeholder needs into technical plans, communicate tradeoffs, and align with security/platform teams.


* CI/CD and IaC: Terraform (preferred) for Databricks + AWS resources; promotion across environments.


* Testing: unit/integration tests for transformations, data quality checks, contract testing, and replay/backfill procedures, version control & code review discipline; clear documentation and runbooks.

Required qualifications, capabilities, and skills


* Formal training or certification on software engineering concepts and 5+ years hands on Software Development Life Cycle experience


* Strong data engineering experience, including proven leading delivery/architecture for multi-team da...