Senior Data Engineer - Technology

Position Summary

We are seeking a Senior Data Engineer to own and modernize our data pipeline architecture.

The immediate mission: migrate ETL workloads out of Redshift stored procedures and legacy SSIS packages into scalable, maintainable pipelines using AWS Glue and S3.

The longer-term vision: help us evolve from batch-oriented processing toward near real-time analytics using stream processing technologies like Apache Flink and ClickHouse, among others.

You will be embedded in a cross-functional engineering team and have real influence over the architecture as we make this transition.

Objectives


* Design and build modern ETL/ELT pipelines using AWS Glue, S3, and related services to replace legacy stored procedures and SSIS jobs.


* Architect data transformation workflows that are testable, version-controlled, and observable.


* Optimize and maintain our Redshift data warehouse, including materialized views, query performance, and cost efficiency.


* Drive the evolution from batch ETL to near real-time stream processing, evaluating and implementing technologies such as Apache Flink, ClickHouse, Kafka, Kinesis, or equivalent platforms.


* Design pipelines that support both near real-time and batch workloads as the platform transitions.


* Partner with product and analytics teams to ensure data models support reporting, AI/ML, and customer-facing features.


* Establish patterns and best practices for pipeline development that the broader team can adopt.


* Participate in production support and incident response for data infrastructure.

Requirements

Education/Experience:


* Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.

Minimum of 5+ years in data engineering roles.

Skills:


* Deep experience with AWS Glue (PySpark/Python), S3, and Redshift.


* Proven track record migrating ETL workloads from legacy tools (SSIS, stored procedures, or similar) to modern cloud-native pipelines.


* Strong SQL skills with best practices and SQL linting, particularly in Redshift or other columnar/MPP databases.


* Experience with or strong interest in stream processing frameworks (Flink, Spark Streaming, Kafka Streams, or similar).


* Familiarity with data pipeline orchestration, monitoring, and error handling patterns.


* Experience with infrastructure-as-code and CI/CD for data pipelines.

Desired Skills:


* Experience with Aurora MySQL or other relational databases, and NoSQL such as DynamoDB.


* Hands-on experience with near real-time OLAP engines (ClickHouse, Apache Druid, or similar).


* Exposure to streaming data infrastructure (Kinesis, Kafka, MQTT).


* Familiarity with IoT or utility/metering data.


* Experience with dbt, Airflow, or Step Functions.


* Strong communication skills, with the ability to explain complex data architecture concepts to technical and non-technical stakeholders.


* A deep understanding of software devel...