Senior Software Engineer, ML Infrastructure

Responsibilities

We are representing a confidential streaming-technology organization seeking a Senior Software Engineer, ML Infrastructure to join their conversational AI team in Cambridge. In this hands-on role, you will own production systems across ranking, model delivery, evaluation and LLM-agent behaviour, working closely with machine-learning, product, data and platform partners.

Your day-to-day responsibilities will include:

• Owning fulfilment-ranking pipelines from training orchestration and quality gates through deployment and online validation
• Building and operating an offline-evaluation platform, combining LLM-as-judge harnesses with deterministic checks for answer quality
• Developing LLM agent systems, including tool routing, retrieval, guardrails and answer caching
• Designing caching as an intentional latency and cost lever for high-volume services
• Building observability for ML and LLM systems, including latency attribution, quality metrics, tracing and per-request cost
• Improving the reliability and operability of distributed systems, and leading the diagnosis of difficult cross-service failures
• Driving accountable AI-assisted engineering practices across the team, building on agents already used in release workflows
• Working across machine-learning, product, data and platform boundaries to turn ambiguous problems into measurable delivery outcomes

What We're Looking For

Our talent team is seeking candidates with:

• Strong production software-engineering experience, including designing, testing, operating and debugging distributed services
• Meaningful ownership of ML infrastructure or production model delivery, with clear evidence of your contribution and its operational or adoption impact
• A practical understanding of the ML lifecycle: training, evaluation, versioning, deployment, online validation, monitoring and safe rollback
• Experience designing for latency, resilience, observability and cost in production systems
• Experience with LLMs, embeddings, semantic search, retrieval or agent systems, including how you evaluate quality and control failure modes
• Experience with caching systems and the trade-offs involved in using caching for low-latency services
• A commitment to automation, CI/CD, code quality and evidence-led engineering decisions
• A collaborative approach and the ability to explain technical trade-offs to partners across disciplines
• A degree in Computer Science, Electrical Engineering or a related field, or equivalent practical experience

Valued technical experience includes: AWS or GCP, Kubernetes, SQL warehouse and batch-processing technologies such as Trino, Presto or Spark, Kafka or other streaming systems, and Go or Python. Node.js experience is a plus.

AI tooling fluency: Our client values candidates who have built fluency across the agentic engineering toolchain—coding harnesses like Claude Code or Cursor, MCP servers, custom skills, or agent frameworks. You should be able to describe projects where you shipped real work with these tools, know how to drive an agent, verify its output, and ramp on an unfamiliar codebase with agent assistance.

How We Work With You

When you apply through Get A Job.ai, a member of our recruiting team will screen your application and qualifications. If there's a strong match, we will submit your profile to our client for consideration. Throughout the process, we act as your advocate and point of contact.

Important: Please do not contact the client directly. All communication regarding this opportunity should go through Get A Job.ai to ensure your application is properly represented.

The role is based in Cambridge with a hybrid working arrangement (generally in-office Monday through Thursday, with flexible remote work on Fridays).

Pay

Compensation details will be discussed during the screening process with our recruiting team.

Equal Opportunity: Get A Job.ai is committed to inclusive recruiting practices. We welcome applicants of all backgrounds and provide support throughout the application process.