Senior Data Engineer (Azure Fabric, PySpark with AI Skills)

Our client is a global technology and digital transformation organization specializing in software engineering and IT consulting, serving international clients across multiple industries. The company operates in a fast-paced, collaborative environment, with a strong focus on innovation, engineering excellence, and continuous professional development.

Role objective

• Design, develop and maintain scalable data pipelines within a modern cloud-based data platform

• Contribute to the development of a Lakehouse architecture leveraging Azure technologies and

Databricks for advanced analytics and AI-driven use cases

Skills

• 5+ years of experience in Data Engineering or related engineering roles

• Strong experience with Azure-based data platforms, ideally Azure Fabric

• Strong hands-on experience with Databricks

• Good understanding of Databricks features such as Unity Catalog, Delta Lake, clusters, notebooks

and jobs

• Advanced Python programming skills

• Hands-on experience with PySpark and SparkSQL

• Experience implementing batch data processing pipelines

• Experience with Spark Streaming

• Knowledge of OneLake / Delta Lake concepts

• Experience with Data Factory Gen2 and M-code

• Experience implementing CI/CD pipelines (Azure DevOps or similar tools)

• Understanding of Azure services and cloud data architecture

• Experience integrating data platforms with Power BI

Qualifications

• University degree in Computer Science, Engineering, Information Systems or related field

• Strong understanding of modern data platforms and big data ecosystems

• Experience working in Agile development environments

• Ability to work independently and collaboratively in distributed teams

Additional requirements

• Experience with Cosmos DB APIs (Mongo, Cassandra or Table) is considered beneficial

• Exposure to AI or Data Science workflows is a plus

• Experience working with large-scale data processing environments

• Professional conduct and confidentiality

• Design and implement scalable data pipelines using Azure Fabric

• Develop data processing and transformation logic using Python, PySpark and SparkSQL

• Work with OneLake and Delta Lake concepts to support modern Lakehouse data architectures

• Develop and support solutions using Cosmos DB (NoSQL API)

• Contribute to Azure Fabric workloads including Data Engineering, Data Factory Gen2 and Lakehouse

• Build and optimize Spark workloads using Databricks

• Implement CI/CD pipelines and follow DevOps best practices

• Integrate data solutions with Power BI for reporting and analytics

• Collaborate with AI, data science and product teams to enable data-driven and AI-powered solutions

• Ensure data quality, performance, reliability and security across data platforms

• Participate in Agile ceremonies and contribute to sprint deliveries

• Support production environments and contribute to continuous improvements