About the Role
We are actively seeking an experienced Senior Data Integration Engineer to join our team. In this role, you will play a crucial part in designing, developing, testing, deploying, maintaining, and enhancing data integration pipelines for our cutting-edge projects. If you possess a wealth of experience in data integration and pipeline development and thrive in a dynamic, collaborative environment, we welcome you to be a key player in shaping the future of our data-driven initiatives. Responsibilities Design, develop, test, deploy, maintain, and improve data integration pipelines Utilize Python and common libraries for efficient data integration and pipeline development Employ advanced SQL for analytical purposes and to optimize complex queries Collaborate with cross-functional teams to ensure the success of data integration projects Implement and maintain data integration and pipeline developments in an AWS environment Optimize and debug user-defined functions, views, and indexes for efficient data integration Utilize GitLab for source control and Jenkins for build and continuous integration Contribute to the enhancement and maintenance of data integration pipelines Provide insights and recommendations for improving data integration processes Collaborate with the wider engineering team to drive standards of excellence across all data integration projects Requirements At least 3+ years of hands-on experience in data integration and pipeline development Strong expertise in Amazon Web Services (AWS) Cloud for data integration with Apache Spark, Glue, Kinesis, and S3 Proficiency in Python for data integration, with real-life experience in Python development Solid understanding of advanced SQL for analytical purposes and complex query optimization Experience with Databricks in an AWS environment for effective data integration Knowledge of source control systems such as GitLab and continuous integration tools like Jenkins Analytical experience with databases, including writing complex queries, query optimization, and debugging Strong communication and collaboration skills, essential for effective teamwork Fluent English language skills at an Upper-Intermediate level Nice to have Familiarity with other data integration tools and technologies