Senior Data Engineer, Python, Spark

Responsibilities

We are representing a confidential streaming-technology client seeking a Senior Data Engineer to join their Manchester office on a hybrid basis. In this role, you will:

• Build highly scalable, fault-tolerant distributed data processing systems (batch and streaming) that handle tens of terabytes of data each day, supporting a petabyte-scale data warehouse
• Design and develop robust data solutions, streamlining complex datasets into simplified, self-service models
• Develop pipelines that ensure high data quality and resilience to imperfect source data
• Define and maintain data mappings, business logic, transformations and data quality standards
• Debug low-level systems, measure performance and optimise large production clusters
• Take part in architecture discussions, influence the product roadmap, and own new initiatives from concept to delivery
• Maintain and evolve existing platforms, introducing modern technologies and architectures where appropriate

You will play a pivotal role in designing data models and building scalable pipelines to capture business metrics across devices, web, and mobile clients—work that is essential to understanding user engagement and improving product experiences.

What We're Looking For

Our ideal candidate brings:

• Strong SQL skills
• Python proficiency (required)
• Proficiency in at least one object-oriented language
• Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto
• Solid background in data modelling, including the design, implementation and optimisation of conceptual, logical, and physical models for scalable architectures
• A degree in Computer Science (BS required; MS preferred)

Advantageous but not essential: Experience with AWS, GCP, or Looker

We value candidates who have built fluency with AI-assisted development tools—coding harnesses, agent frameworks, or similar technologies—and can describe projects where these tools helped ship real work. If you know how to drive an agent, verify its output, and ramp on an unfamiliar codebase with AI assistance, that's a plus.

How We Work With You

Get A Job.ai is a recruiting firm representing this confidential client. When you apply through our platform at getajob.ai, a member of our talent team will screen your application and coordinate next steps. If you're a strong match, we will submit your profile to the client for consideration. Please do not contact the client directly—all communication should flow through Get A Job.ai to ensure a smooth and professional process.

Pay

Compensation details will be discussed during the screening process with our recruiting team.

Equal Employment Opportunity: Get A Job.ai is committed to building diverse and inclusive teams. We welcome applicants of all backgrounds and provide equal employment opportunities regardless of race, color, religion, sex, national origin, age, disability, or any other protected characteristic.