Principal Data Engineer

The Enterprise Corporate Data Team is looking for a Principal Data Engineer, a senior technical leader responsible for architecting the core data infrastructure and platforms that power enterprise-scale AI applications.

Reporting to the VP of Engineering, this role will focus on building systems to surface content, audience, products etc through semantic search capabilities to support personalization, audience discovery, and intelligent content discovery.

The Principal Data Engineer will lead the end-to-end design and implementation of scalable pipelines,platforms and systems that support semantic search across massive volumes of structured/semi-structured data using GeN AI technology.

This individual will also co-ordinate with a team of off-shore engineers, ensuring consistent delivery, code quality, and alignment with business and technical goals.

The ideal candidate will possess an entrepreneurial ethos, an ability to operate in a dynamic environment, and a working knowledge of the current digital media landscape.

The candidate should be an expert/knowledgeable with Search systems including but not limited to Similarity, hybrid and semantic search.

This role is based in New York City.

Key Responsibilities:

● Lead the design and implementation of high-performance OLAP and OLTP systems to support similarity and semantic search.
● Architect scalable data platforms that integrate structured and unstructured data,including behavioral signals, content metadata, and user engagement data for Gen AI use cases.
● Build systems that enable semantic enrichment of content through entity recognition, disambiguation, normalization and deduplication techniques.
● Lead the design and build of high throughput, low latency and highly relevant Enterprise search systems using Vectors, Graph and other search strategies.
● Familiar with relevance measurement techniques like DCG, NDCG etc.
● Partner closely with other Data engineers, ML engineers and data scientists to deploy and operationalize models for content and audience intelligence.
● Oversee and co-ordinate with an offshore engineering team, providing technical guidance, code reviews, and project oversight to ensure timely, high-quality
deliverables.
● Ensure best practices in data governance, quality, observability, and documentation across all engineering workflows.
● Collaborate with stakeholders across product, marketing, and data science to translate business needs into scalable AI data systems.
● Well versed in architecting, designing and developing large scale OLTP and OLAP systems.
● Experience building and operating streaming systems using messaging systems like Kafka, Pub/sub, SQS etc.
● Experience building an RAG/Graph RAG system with Google, OpenAI or another Gen AI platform.
● Experience building a knowledge graph using Neo4j, Spanner, Neptune or another tool is a plus

Qualifications:

● 10+ years of experience in data engineering, with significant experience bui...