Data Engineering Lead

As a Data Engineer, you will be responsible for transforming raw data from our applications into structured datasets for large-scale analysis and machine learning model training.

You will work closely with our development, data science, and business intelligence teams to ensure data integrity, quality, and accessibility.



* Data Pipeline Development: Design, develop, and maintain scalable data pipelines to process raw data from various sources.


* Data Transformation: Clean, transform, and enrich data to create high-quality datasets suitable for analysis and machine learning.


* Collaboration: Work closely with product teams, software developers, data scientists, and analysts to understand data needs and deliver innovative solutions.


* Data Management: Ensure data accuracy, consistency, and reliability across all datasets.


* Optimization: Optimize data processes for performance and scalability.


* Documentation: Maintain comprehensive documentation of data pipelines, processes, and schemas.

Working Conditions:


* 40 hours per week, with occasional, but rare, overtime


* Remote / Hybrid / Flexible Work Options Available


* Frequent interaction with developers, test automation engineers, QA, management, and members of other Verisk subsidiaries


* Some days we just leave the office and have fun team building activities!


* Regular team lunches!


* State of the art facility with basketball, volleyball, and gym


* Ping pong, foosball, fruit bowls, snacks


* Fun and energetic teams


* Time for innovation, Hack-A-Thons, and learning



* Educational Background: bachelor's degree in computer science, Data Engineering, or a related field.


* Experience: 3+ years of experience as a Data Engineer or in a similar role.


* Technical Proficiency:
+ Programming Languages: Proficiency in Python, SQL, and familiarity with languages such as C# or Java.
+ Data Processing: Experience with ETL tools and frameworks (e.g., Apache Airflow, Luigi, DBT).
+ Big Data Technologies: Hands-on experience with big data technologies such as Hadoop, Spark, and Kafka.
+ Database Management: Strong knowledge of relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra).
+ Cloud Platforms: Experience with cloud services (e.g., AWS, Google Cloud, Azure) and their data processing tools (e.g., AWS Glue, Google BigQuery).
+ AI: Familiarity and enthusiasm for bleeding-edge analytical enablement using tools such as Large Language Models and Prompt Engineering.


* Data Warehousing: Knowledge of data warehousing concepts and solutions (e.g., Redshift, Snowflake).


* Version Control: Proficient with version control systems (e.g., Git).

Machine Learning: Understanding of machine learning concepts and experience working with data for ML model training.

About Us
For over 50 years, Verisk has been the leading ...