We are a team that values each other the most. If you join us, you can freely pinpoint areas for improvement, communicate your vision for both personal and organizational growth, and count on our full support. We openly share suggestions, observations, and knowledge. Every piece of feedback is an opportunity for us to learn. On a daily basis, we lead by example, share our experiences, and help each other out. Check out the job description and don't hesitate to get in touch with us. 😉
Develocraft is looking for a Senior Data Engineer for our client operating in the financial services sector.
Salary to be agreed. We offer a flexible budget based on experience and expertise.
Start ASAP.
Technical Knowledge:
- Hands-on experience in designing, building, and maintaining data platforms, particularly Microsoft Fabric or similar solutions.
- Experience in Data Engineering, including data pipeline design, data transformation, and orchestration.
- Strong understanding of Lakehouse architecture, Data Warehousing, and data modeling.
- Experience implementing AI, Machine Learning, and Generative AI solutions.
- Knowledge of Large Language Models (LLMs), vector databases, embeddings, prompt engineering, and Retrieval-Augmented Generation (RAG) architecture.
- Experience with Microsoft AI technologies, including Azure AI Services, Azure OpenAI, Microsoft Copilot, and AI capabilities within Microsoft Fabric.
- Ability to design and build data solutions that support AI and Machine Learning workloads.
- Proficiency in Python, Spark, SQL, and data analytics/AI libraries.
- Knowledge of MLOps and AIOps, including model deployment, monitoring, and lifecycle management.
- Experience integrating systems through APIs, file-based interfaces, and event-driven architectures.
Engineering Competencies:
- Hands-on experience with Azure DevOps, including repositories, work item management, and deployment pipelines.
- Ability to manage services throughout their entire lifecycle, from design and implementation to maintenance and continuous improvement.
- Strong troubleshooting skills related to data platforms, integrations, and AI services.
- Ability to produce clear and well-structured technical documentation.
Security and Collaboration:
- Understanding of security best practices, access control, and data governance.
- Knowledge of Responsible AI principles, AI model governance, and ethical AI practices.
- Ability to identify AI-related risks and implement appropriate safeguards.
- Self-driven with the ability to collaborate effectively with both business and technical teams.
- Strong communication skills, with the ability to explain technical concepts to non-technical stakeholders.
- Results-oriented mindset with a focus on delivering high-quality, secure, and maintainable solutions.
Experience:
- Experience delivering enterprise data platform, analytics, and AI projects.
- Practical experience in Data Engineering, Platform Engineering, and AI Engineering.
- Hands-on experience with Microsoft Fabric, Azure Data Services, and AI technologies.
- Ability to translate business requirements into scalable data and AI solutions.
- Experience supporting and maintaining production AI services.
- Experience working in regulated industries, such as the financial sector, is considered a strong advantage.
Data Platform Engineering (Microsoft Fabric):
- Design, build, maintain, and enhance the Microsoft Fabric data platform, ensuring its availability, performance, scalability, and reliability.
- Develop and maintain data pipelines for data ingestion, transformation, and delivery.
- Design and manage Lakehouse and Data Warehouse solutions.
- Monitor platform health, troubleshoot incidents, and ensure operational continuity.
Data Engineering and Data Modeling:
- Design, develop, and maintain data models for reporting, analytics, and AI workloads.
- Build optimized datasets and semantic models.
- Ensure data quality through validation, error handling, and adherence to data modeling best practices.
AI Engineering:
- Design, implement, and maintain AI solutions using Microsoft AI and Azure AI services.
- Develop Generative AI solutions, Retrieval-Augmented Generation (RAG) applications, intelligent search capabilities, and AI agents.
- Prepare and manage data for AI models, including prompt engineering, model lifecycle management, and deployment.
- Collaborate with business stakeholders to identify AI use cases and continuously monitor the quality and effectiveness of AI solutions.
Platform Integration and Data Access:
- Integrate source systems with the Microsoft Fabric platform.
- Design APIs and data access services for reporting, business applications, and AI solutions.
- Integrate AI services with enterprise applications while ensuring compliance with security and architectural standards.
Engineering Practices:
- Leverage Azure DevOps for source code management, work item tracking, and deployment automation.
- Implement and maintain CI/CD and MLOps practices.
- Create and maintain clear, comprehensive technical documentation.
Security and Governance:
- Manage access control, auditing, and data security across the platform.
- Ensure compliance with regulatory requirements and Responsible AI principles, including AI model governance, data lineage, monitoring, and risk management.
Collaboration and Project Delivery:
- Translate business requirements into scalable data and AI solutions using Microsoft Fabric, Azure, and AI technologies.
- Support the onboarding of new data sources, platform automation, and continuous platform enhancement.
- Manage and prioritize the engineering backlog using Azure DevOps.
- Act as a technical expert, providing guidance and resolving complex issues related to data platforms and AI solutions.