Lead Data Engineer (With Streaming)

We are looking for a Data Engineer with strong hands-on experience in batch data processing and practical exposure to streaming solutions. The ideal candidate understands event-driven architecture and can design scalable, real-time data pipelines on Azure. The role includes designing reliable near real-time data platforms supporting analytical and operational use cases.      Tasks:   Design and implement scalable batch data pipelines on Azure using modern ETL/ELT approaches Develop and optimize data transformations using Apache Spark and SQL, ensuring performance and maintainability. Design efficient data models for analytical and reporting use cases (data lakes, data warehouses). Ensure data reliability, quality, monitoring, and performance in distributed environments. Build and maintain near real-time data pipelines using Apache Spark (Structured Streaming). Implement Change Data Capture (CDC) pipelines from relational databases when required. Contribute to event-driven solutions using message brokers where streaming use cases are present. Engage with stakeholders to refine data requirements and architecture choices. Perform code reviews, promote best practices, and mentor junior engineers. Work independently while maintaining effective communication with stakeholders.     What We're Looking For:   4–6 years of experience as Data Engineer Strong SQL and Python skills (production-grade code) Experience with Apache Spark (including Structured Streaming) Hands-on experience with Azure (Synapse / ADF / ADLS / Databricks) Understanding of distributed systems fundamentals Experience designing ETL/ELT pipelines Experience working in Agile environment English C1.   What Will Set You Apart:   Experience with Apache Kafka (event streaming) and/or Apache Flink (stream processing engine). Experience with Change Data Capture (CDC) tools (e.g. Debezium, SQL Server CDC, Azure CDC, Kafka Connect). Experience with schema management (e.g. Avro, Schema Registry). Experience with Delta Lake / Delta Live Tables. Knowledge of event-driven architecture patterns. Experience with CI/CD for data platforms.