Lead Data Engineer

At Elanco (NYSE: ELAN) – it all starts with animals!

As a global leader in animal health, we are dedicated to innovation and delivering products and services to prevent and treat disease in farm animals and pets.

At Elanco, we are driven by our vision of Food and Companionship Enriching Life and our purpose – all to Go Beyond for Animals, Customers, Society and Our People.

At Elanco, we pride ourselves on fostering a diverse and inclusive work environment.

We believe that diversity is the driving force behind innovation, creativity, and overall business success.

Here, you’ll be part of a company that values and champions new ways of thinking, work with dynamic individuals, and acquire new skills and experiences that will propel your career to new heights.

Making animals’ lives better makes life better – join our team today!

Your Role: Lead Data Engineer

 

Data Engineering at Elanco delivers products and thought leadership that transform how the organization leverages data.

The ERP Enterprise Data Product (EEDP) team is a critical pillar of this strategy, responsible for modernizing our complex ERP data landscape.

EEDP provides a robust, near real-time replication of SAP S/4HANA data into Google BigQuery, which is then integrated into our Enterprise Lakehouse (Databricks) to power advanced AI/ML, conversational analytics, and global reporting.

 

Your Responsibilities : 


* Complex Pipeline Ownership: Independently design and implement complex end-to-end data pipelines spanning multiple sources and large SAP datasets (Configuration, Master Data, Transactional).

Anticipate scaling needs, data skew, and growth using patterns like partitioning, clustering, and incremental backfills.


* Technical Mentorship: Act as the primary gatekeeper for technical standards; ensure junior engineers are proficient in foundational tooling (e.g., Git branching, environment setup) and consistently apply testing patterns.


* Production Ownership: Own routine production incidents in familiar pipelines (schema drift, late files, logic errors) and deliver permanent fixes by updating code, tests, and documentation.


* Stakeholder Collaboration: Work directly with analysts and data scientists to refine requirements into concrete technical stories, proactively surfacing edge cases such as late-arriving data or schema changes.


* Standards Advocacy: Ensure the adoption of Elanco’s data standards and foundational tools (GitHub, VS Code, SonarQube, Copilot) across the team, using code reviews as a teaching tool.


* Efficiency & Optimization: Monitor key pipelines using logs and dashboards; identify and fix performance or memory issues using query plans and Spark UI without assistance.


* Quality Assurance: Design and implement meaningful data quality checks (referential integrity, distribution checks) and ensure tests run in CI/CD schedules.


* Continuous Improvement: Proactively identify and fix knowledge silos by...