AI Infrastructure Engineer, Serving Platform

Responsibilities

Our talent team is representing a confidential AI technology organization seeking an AI Infrastructure Engineer to join their ML Infrastructure team in London. In this role, you will design and build platforms for scalable, reliable, and efficient serving of large language models. The platform you develop will power both research and production systems across various environments.

Your day-to-day work will include:

• Building and maintaining fault-tolerant, high-performance systems for serving LLMs and other models at scale
• Developing internal platforms to empower LLM capability discovery
• Collaborating with researchers and engineers to integrate and optimize models for production and research use cases
• Conducting architecture and design reviews to uphold best practices in system design and scalability
• Developing monitoring and observability solutions to ensure system health and performance
• Leading projects end-to-end, from requirements gathering to implementation, in a cross-functional environment

What We're Looking For

Required qualifications:

• 4+ years of experience building large-scale, high-performance backend systems
• Strong programming skills in one or more languages such as Python, Go, Rust, or C++
• Experience with LLM serving and routing fundamentals including rate limiting, token streaming, load balancing, and budgets
• Experience with LLM capabilities and concepts such as reasoning, tool calling, and prompt templates
• Experience with containers and orchestration tools such as Docker and Kubernetes
• Familiarity with cloud infrastructure (AWS, GCP) and infrastructure as code tools like Terraform
• Proven ability to solve complex problems and work independently in fast-moving environments

Preferred experience:

• Experience with modern LLM serving frameworks such as vLLM, SGLang, TensorRT-LLM, or text-generation-inference

How We Work with You

When you apply through Get A Job.ai, our recruiting team will review your qualifications and conduct an initial screening. If there's a strong match, we will submit your profile to our client for consideration. Please do not contact the client directly—all communication and coordination will be managed through our team to ensure the best experience for you.

Pay

Compensation details will be discussed during the screening process based on your experience and qualifications.

Equal Employment Opportunity: Get A Job.ai is committed to inclusive hiring practices and equal opportunity employment. We welcome applicants from all backgrounds regardless of race, color, religion, sex, national origin, age, disability, veteran status, sexual orientation, gender identity, or any other protected characteristic.