About the Role
We are seeking a highly skilled Data Engineer with expertise in managing, designing, and optimizing data pipelines utilizing Apache Airflow, Snowflake, and Apache Kafka.
This individual will play a pivotal role in architecting robust, scalable, and efficient data solutions, ensuring the integrity, reliability, and accessibility of our data infrastructure
✅ Responsibilities:
Develop and implement data models to support business requirements, optimizing for performance and scalability;
Design, build, and maintain scalable data pipelines using Apache Airflow;
Implement and maintain Kafka-based streaming data pipelines for real-time data processing and integration with various systems;
Integration to third party databases and APIs;
Establish monitoring, alerting, and maintenance procedures to ensure the health and reliability of data pipelines;
Collaborate with cross-functional teams including data scientists, analysts, and stakeholders to understand data requirements.
✅ Requirements:
Proficiency in Python, SQL, and experience with data manipulation and transformation;
Data warehouse and data modelling techniques;
Experience in designing, building, and maintaining complex data pipelines using Airflow;
Proven track record in data engineering roles, with a focus on designing and implementing scalable data solutions using Snowflake or Redshift;
In-depth understanding and practical experience in implementing Kafka-based streaming architectures for real-time data processing.
Tech Stack
Apache AirflowSnowflakeApache KafkaPythonSQLData modeling