Senior Azure Data Engineer (Databricks)

Responsibilities

We are representing a confidential consulting firm seeking a Senior Azure Data Engineer with deep Databricks expertise. In this hands-on role, you will design, build, and deploy scalable and secure data engineering pipelines using the Databricks platform on Azure. You'll work directly with clients to understand their data needs and develop innovative solutions that drive business transformation.

Your day-to-day responsibilities will include:

• Designing and developing robust data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services
• Building real-time, event-driven solutions using technologies such as Kafka and Azure Event Hubs
• Developing and maintaining CI/CD pipelines using Azure DevOps, Jenkins, GitHub Actions, and modern DevOps practices to support reliable deployments
• Collaborating with clients and multidisciplinary teams to translate business requirements into scalable, production-ready data solutions
• Championing clean code, data lifecycle optimization, and engineering best practices while using approved AI tools and automation responsibly to improve collaboration and delivery

What We're Looking For

Required qualifications:

• Proven hands-on experience with the Databricks platform, including orchestration and enterprise-scale Azure data engineering solutions
• Strong technical skills in Python, PySpark, and SQL, with experience working with distributed data processing frameworks
• Expertise delivering end-to-end data pipelines across ingestion, transformation, and serving layers within modern Data Lakehouse architectures
• Experience with schema design, GDPR-compliant data solutions, DevOps tooling, and CI/CD processes
• A collaborative approach to engineering with experience using AI-enabled tools responsibly to improve productivity while maintaining appropriate human oversight and solution quality

Bonus qualifications:

• Development experience using Scala or Java
• Familiarity with Cloudera, Hadoop, Hive, Spark, and related distributed data ecosystems
• Experience working with sensitive data and applying data privacy regulations, including GDPR
• Curiosity to explore new technologies, including responsible AI and automation, to continuously improve engineering practices
• Passion for collaboration, knowledge sharing, and continuous learning within high-performing engineering teams

How We Work With You

Our talent team at Get A Job.ai will guide you through every step of the process. When you apply through our platform, one of our experienced recruiters will screen your application and conduct an initial conversation to understand your background and career goals. If there's a strong match, we will submit your profile to our client for consideration.

Please note: we ask that you do not contact the client directly. All communication and coordination will flow through Get A Job.ai to ensure a professional and streamlined experience for everyone involved.

Pay

Compensation details will be discussed during the screening process based on experience and qualifications.

Equal Employment Opportunity: Get A Job.ai is committed to creating an inclusive recruitment process. We welcome candidates from all backgrounds and will make reasonable adjustments to support accessibility needs at any stage. Please let us know how we can help.