Data Engineer

Join our dynamic team as an Internal Database Engineering Professional and help architect the future of our data infrastructure! We're seeking a talented individual who can transform complex data challenges into elegant solutions while driving innovation in our data engineering practices.

What will you do:



* Design, develop and engineer data lake by data source identification, data extraction, data modelling, data store design and governance


* Develop data flows to ingest and transform both structured and unstructured data using warehousing and big data methodologies and tools to build refined and aggregated data assets for business intelligence and AI enablement.


* Implement efficient patterns for data storage, partitioning, semantic layer and metadata structures to enable consumption of data as a product.


* Implement automation capabilities to enable governance features for data privacy and protection, data inventory and cataloguing, authorization and data security.


* Implement enhanced observability, cost management and reporting strategies for data engineering operations.


* Build relevant integration testing and quality assurance capabilities while ensuring compliance with architectural standards for development.


* Perform assessments (Proof of Concepts) of the latest tools and technology to solution and assist in evolving the technology landscape.


* Lead, mentor, and support the Data Delivery teams with methodologies, tools and technical knowledge.



* 6+ years minimum experience in technical roles with focus in data engineer.


* 3+ years of Hands-on experience in AWS services including but not limited to: S3, KMS, Lambda, Glue (or Spark), SQS, EventBridge, AWS Step functions etc.,


* 3+ years of hand on experience in writing clean, modular, testable, and maintainable code using either functional or OOP principles.


* Good understanding of AWS networking concepts: VPC, subnets, routing, NAT gateways, security groups.


* Proficiency in IAM concepts: roles, policies, assume-role, and cross-account access.


* Good experience in at least one programming language such as Python, Scala, Java etc


* Good understanding of Spark architecture, execution plans, and performance tuning.


* Strong SQL skills (complex joins, window functions, aggregations etc) and experience with Redshift and/or Athena, or any other distributed compute engines.


* Experience in writing and maintaining Terraform scripts.


* Familiarity with Git, GitHub workflows, and CI/CD tools like Jenkins.


* Hands on experience with data governance, cataloguing and discovery tools such as AWS Glue and Lake formation.


* Exposure to data lake architectures and data governance practices.


* Familiarity with the implementation of monitoring and logging.


* Advanced cloud and on-premises knowledge of NoSQL database structures including AWS DynamoDB, MongoDB, etc.


* Knowledge of Data Man...