Data Engineer – Kafka & Elasticsearch

Responsibilities

• Design, implement, and manage event-driven architectures using Apache Kafka as a core buffer and processing layer.
• Build and maintain Elasticsearch stacks, including indexing strategies and lifecycle management for structured and unstructured data.
• Develop a data service layer using REST APIs and microservices to make data available for analysis and monitoring.
• Deploy and orchestrate data components within Docker, Kubernetes, and OpenShift environments.
• Optimize data pipelines and ensure system observability through GitLab CI and Python scripting.
• Ensure data consistency across the entire ecosystem while implementing robust error handling and performance tuning.
• Document technical configurations, schemas, and architectural choices to maintain high standards of knowledge transfer.

Requirements

• 3+ years of experience with Apache Kafka, including Kafka Streams, Connect, Schema Registry, Avro, and ksqlDB.
• 3+ years of experience with Elasticsearch, covering Kibana, ingest pipelines, and index lifecycle management.
• You possess strong knowledge of container orchestration using Docker, Kubernetes, and OpenShift.
• You have experience developing data service layers through REST APIs and microservices.
• You bring proficiency in Python scripting and experience with PostgreSQL databases.
• You're familiar with CI/CD tools, specifically GitLab CI, for automating data pipelines.
• You have expertise in observability, error handling, and the optimization of complex data systems.
• You are methodical and precise, with a strong ability to document technical architectures and configurations.
• You are proactive and solution-oriented, thriving in multidisciplinary team environments.
• You have active knowledge of Dutch, French, and English.

Nice to Haves

• Willingness to adapt to flexible schedules within a complex technical environment.