Lead Software Engineer - Databricks / PySpark / AWS

Job Summary:

We have an exciting and rewarding opportunity for you to take your software engineering career to the next level.

As a Lead Software Engineer at JPMorgan Chase within the Corporate Technology, you play a crucial role in an agile team dedicated to enhancing, building, and delivering trusted, market-leading technology products in a secure, stable, and scalable manner.

As a key technical contributor, you are tasked with implementing critical technology solutions across multiple technical domains, supporting various business functions to achieve the firm's business objectives.

Job Responsibilities:


* Design solutions at the appropriate level of detail and drive consensus among peers as needed.


* Champion software engineering best practices within the team.


* Collaborate with software engineers and cross-functional teams to design and implement deployment strategies using AWS Cloud and Databricks pipelines.


* Lead the design, development, testing, and implementation of application solutions.


* Partner with technical experts, stakeholders, and team members to resolve complex technical challenges.


* Proactively address issues to support leadership objectives and prevent customer impact.


* Design, develop, and maintain robust data pipelines for ingesting, processing, and storing large volumes of data from diverse sources.


* Implement ETL (Extract, Transform, Load) processes to ensure data quality and integrity using tools such as Apache Spark and PySpark.


* Monitor and optimize the performance of data systems and pipelines.


* Apply best practices for data storage, retrieval, and processing.


* Maintain comprehensive documentation for data systems, processes, and workflows.

Required Qualifications, Capabilities, and Skills:


* Formal training or certification on software engineering concepts and 5+ years applied experience.


* Proficiency in programming languages such as Python and PySpark.


* Experience designing and implementing data pipelines in cloud environments.


* Strong background in design, architecture, and development using AWS Services, Databricks, Spark, Snowflake, and related technologies.


* Experience with CI/CD tools such as Jenkins, GitLab, or Terraform.


* Familiarity with containerization and orchestration technologies including ECS, Kubernetes, and Docker.


* Ability to troubleshoot issues related to Big Data and Cloud technologies.

Preferred Qualifications, Capabilities, and Skills:


* 5+ years of experience leading and developing data solutions in AWS Cloud.


* 10+ years of experience building, implementing, and managing data pipelines using Databricks on Spark or similar cloud technologies

JPMorganChase, one of the oldest financial institutions, offers innovative financial solutions to millions of consumers, small businesses and many of the world's most prominent corporate, institutional and government clients under the J...