Data Engineer – Azure & Databricks

Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure and Databricks.
• Integrate data from diverse internal and external sources to ensure consistency and quality across the platform.
• Manage and optimize Azure Data Lake and distributed data processing environments.
• Develop sophisticated data models and datasets to support business analytics and long-term grid planning.
• Implement monitoring and automation for high-performance data workflows.
• Collaborate with stakeholders to translate business requirements into technical and AI-enabled solutions.

Requirements

• You have 5+ years of expert knowledge in SQL and data access frameworks.
• You bring 3+ years of data engineering experience with Azure, Databricks, and PySpark.
• You possess 3+ years of experience with Python, specifically utilizing Pandas and Polars.
• You have 3+ years of experience building dashboards and reporting solutions with Power BI or Streamlit.
• You possess 3+ years of experience using advanced Excel features and VBA.
• You bring experience with Git and collaborative software development practices.
• You have a strong focus on code quality, maintainability, and knowledge sharing.
• You are fluent in English with knowledge of Dutch and/or French.

Nice to Haves

• Knowledge of R, particularly data.table.
• Familiarity with AI/ML concepts and AI-enabled data solutions.
• Experience documenting solutions using Markdown and Confluence.
• Prior experience in the energy sector, power systems, or grid planning.