Expert, Data Platforms Engineer

Do you dare to lead the digital transformation to create a more sustainable world?

At Schneider Electric, we are the leading digital partner in sustainability and efficiency for businesses, communities, and individuals.

Our goal is for all people to be able to make the most of our energy and resources through the integration of processes, software, big data, and energy technologies that enable integrated management of buildings, industries, and cities.

We are present in over 110 countries with a diverse and disruptive team of over 150,000 people.

If you are seeking an opportunity to propel your career and challenge your creativity, we want to meet you!

Schneider Digital is the digital department of Schneider Electric, leading the digital transformation in the company by giving support globally to our internal teams and our clients.

Schneider Digital consists of 6 Digital Hubs worldwide which are strategically located to ensure a 24/7 support across the company (France, China, India, USA, Mexico and Spain).

Our Digital Hub in Barcelona is formed by +450 employees working in strategic projects and different roles such as Data, Cybersecurity, ERP, Cloud, Infrastructures, IT Project Management or Digital Marketing.

At Data & Performance team of Schneider Electric, we are building Intel Data Store (IntelDS), a global Data Lake for enterprise data.

It is a Big Data platform fully hosted on AWS and connected today to more than 40 data sources.

The job purpose is to support the big data engineering team building and improving IntelDS by:



* Connecting new sources to enrich the data scope of the platform.



* Design and develop new features based on consumer application requests to ingest data in the different layers of IntelDS.



* Automate the integration and delivery of data objects and data pipelines.

What will you do?

Preparing data and make it available in an efficient and optimized format for our different data consumers, ranging from BI and analytics to data science applications.

It requires to work with current technologies used by IntelDS and in particular Apache Spark, Lambda & Step Functions, Glue Data Catalog, and RedShift on AWS environment.

This includes:



* Design and develop new data ingestion patterns into IntelDS Raw and/or Unified data layers based on the requirements and needs for connecting new data sources or for building new data objects.

Working in ingestion patterns allow to automate the data pipelines.



* Participate to and apply DevSecOps practices by automating the integration and delivery of data pipelines in a cloud environment.

This can include the design and implementation of end-to-end data integration tests and/or CICD pipelines.



* Analyse existing data models, identify and implement performance optimizations for data ingestion and data consumption.

The objective is to accelerate data availability within the platform and to consumer applications.



* Support client applications i...