Lead Software Engineer - Data Engg. - Databricks / Snowflake

We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.

As a Lead Software Engineer at JPMorgan Chase within the (IAM) Identity and Access Management Data team, you will play a crucial role in designing, developing, and maintaining scalable data processing solutions using Databricks, Python, and AWS.

You will collaborate with cross-functional teams to deliver high-quality data solutions that support our business objectives.

Job responsibilities


* Execute creative, data-driven software solutions end-to-end (design, development, troubleshooting), thinking beyond routine approaches to solve complex technical problems.


* Design and build a control plane for enterprise data pipelines, standardizing pipeline definition, scheduling, deployment, governance, and run-time management (Databricks today; extensible for future engines).


* Develop self-service APIs/SDKs, templates, and configuration-driven onboarding with consistent guardrails (standards, validation, environment promotion, approvals) and centralized pipeline metadata (ownership, SLAs/SLOs, dependencies, schema/parameter/version tracking).


* Design, develop, and maintain scalable data pipelines and processing workflows using Python, PySpark, SQL, Databricks on AWS; develop fact/dimension models for analytics and reporting.


* Ensure data quality, security, lineage, and operational transparency via standardized observability (logs/metrics/traces), dashboards, alerting, runbooks, and automated remediation patterns (retries/backfills, common-failure automation).


* Lead and participate in the full SDLC (requirements, design, build, test, deploy, maintain), acting as SRE/production support for pipeline and platform services to improve stability and reliability.


* Collaborate with stakeholders to shape data management strategy and translate requirements into scalable, compliant solutions; document data flows, logic, and transformation rules for knowledge sharing.


* Mentor engineers and lead communities of practice to drive adoption of modern engineering practices and tools, fostering an inclusive, high-performing culture; utilize firm-approved AI-assisted development tools to accelerate delivery and testing


* Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.


* Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.

Required qualifications, capabil...