Databricks Cloud Big Data Engineer III

Be part of a dynamic team where your distinctive skills will contribute to a winning culture and team.

As a Databricks Cloud Big Data Engineer III at JPMorgan Chase within the Enterprise technology in Employee Experience Employee Core Data Platform (Palmos) Group, you serve as a seasoned member of an agile team to design and deliver trusted data collection, storage, access, and analytics solutions in a secure, stable, and scalable way.

You are responsible for developing, testing, and maintaining critical data pipelines and architectures across multiple technical areas within various business functions in support of the firm's business objectives.

Job responsibilities


* Supports review of controls to ensure sufficient protection of enterprise data


* Being responsible for advising and making custom configuration changes in one to two tools to generate a product at the business or customer request


* Updating logical or physical data models based on new use cases


* Frequently uses SQL and understands NoSQL databases and their niche in the marketplace


* Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems.


* Creates secure and high-quality production code and maintains algorithms that run synchronously with appropriate systems.


* Produces architecture and design artifacts for complex applications while being accountable for ensuring design constraints are met by software code development.


* Gathers, analyzes, synthesizes, and develops visualizations and reporting from large, diverse data sets in service of continuous improvement of software applications and systems.


* Identifies hidden problems and patterns in data proactively and uses these insights to drive improvements to coding hygiene and system architecture.


* Contributes to software engineering communities of practice and events that explore new and emerging technologies.


* Adds to team culture of diversity, equity, inclusion, and respect.

Required qualifications, capabilities, and skills


* Formal training or certification on software engineering concepts and 3+ years applied experience.


* Experience with AWS EMR, Apache Spark - Scala, Python, Kafka and Infrastructure as Code (IaC) tools like Terraform.


* Knowledge on AWS Lake Formation, AWS Glue Data Catalog and fine-grained access control.


* Hands-on practical experience in system design, application development, testing, and operational stability.


* Experience across the data lifecycle


* Significant experience with statistical data analysis and ability to determine appropriate tools and data patterns to perform analysis


* Proficient in coding in one or more languages (Python, Pyspark.).


* Experience in developing, debugging, and maintaining code in a large corporate environment with one or more modern ...