Senior Data Engineer, Python, Spark

Responsibilities

We are representing a confidential streaming technology organization in their search for a Senior Data Engineer to join their data platform team in Cambridge. In this role, you will:

• Build highly scalable, fault-tolerant distributed data processing systems (batch and streaming) that handle tens of terabytes of data daily, supporting a petabyte-scale data warehouse
• Design and develop robust data solutions, streamlining complex datasets into simplified, self-service models
• Develop pipelines that ensure high data quality and resilience to imperfect source data
• Define and maintain data mappings, business logic, transformations, and data quality standards
• Debug low-level systems, measure performance, and optimize large production clusters
• Participate in architecture discussions, influence the product roadmap, and own new initiatives from concept to delivery
• Maintain and evolve existing platforms, introducing modern technologies and architectures where appropriate
• Design data models and build scalable pipelines to capture business metrics across multiple device types and platforms

What We're Looking For

Our client seeks a Senior Data Engineer with proven expertise in big data technologies and a strong foundation in software engineering. The ideal candidate will bring:

• Strong SQL skills
• Proficiency in Python (required)
• Proficiency in at least one object-oriented programming language
• Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto
• Solid background in data modeling, including the design, implementation, and optimization of conceptual, logical, and physical models for scalable architectures
• Experience with AWS, GCP, or Looker (advantageous but not essential)
• A degree in Computer Science (BS required; MS preferred)
• Experience or demonstrated fluency with AI-assisted development tools such as Claude Code, Cursor, or similar agentic engineering frameworks is valued—we're looking for engineers who can leverage AI to ship faster and scale their impact

This role is offered on a hybrid basis from the Cambridge office.

How We Work with You

Get A Job.ai represents top-tier technology companies seeking exceptional engineering talent. When you apply through our platform:

• You'll submit your application directly on getajob.ai
• Our talent team will conduct an initial screening to understand your background and career goals
• We'll present qualified candidates to our client for consideration
• Please do not contact the client directly—all communication flows through Get A Job.ai to ensure a professional, streamlined process

Pay

Compensation details will be discussed during the screening process and are competitive with senior data engineering roles in the Cambridge market.

Equal Employment Opportunity

Get A Job.ai is committed to fostering an inclusive recruitment process. We welcome applicants of all backgrounds and work with our clients to ensure fair and equitable hiring practices.