Senior Software Developer: Models Team (Token Factory)

Responsibilities

Our talent team is representing a confidential AI infrastructure organization seeking a Senior Software Developer for their models team in London. In this role, you will:

• Onboard state-of-the-art open-source large language models into production infrastructure
• Design and implement advanced inference optimization techniques including cache-aware routing, NUMA-aware deployments, KV-cache offloading, and disaggregated serving architectures
• Maintain and extend forks of leading inference frameworks such as vLLM and TRT-LLM
• Build tooling and automation for performance testing, quality assurance, hyperparameter optimization, and automated rollout pipelines
• Support high-stakes customer proof-of-concepts requiring demanding production-scale model serving
• Collaborate with cross-functional teams including cloud infrastructure, observability, and platform engineering
• Develop backend systems using Go and Python to scale AI model serving at production scale
• Implement autoscaling solutions with high-speed model loading over high-performance networking

What We're Looking For

Required qualifications:

• Production experience serving large language models
• Strong programming skills in Python and/or Go
• Proven experience designing and operating highly scalable, highly available distributed services

Nice to have:

• Contributions to vLLM, SGLang, TRT-LLM, or NVIDIA ecosystem open-source projects
• Deep understanding of KV cache management, speculative decoding, and quantization techniques
• Experience with LLM evaluation frameworks and performance benchmarking
• Strong knowledge of Kubernetes and distributed serving architectures
• Familiarity with InfiniBand, RoCE, or high-performance networking technologies
• Understanding of autoscaling and fault-tolerant systems

How We Work With You

Candidates apply directly through Get A Job.ai. Our recruiting team will conduct an initial screening to understand your experience and career goals. We will then present qualified candidates to our client for consideration. Please note that you should not contact the client directly during this process—all communication will be coordinated through our team to ensure the best experience for everyone involved.

The role is based in London with remote collaboration across Europe. The team meets in person every one to two months for planning and collaborative work sessions.

Pay

Compensation details will be discussed during the screening process based on experience level and qualifications.

Equal Opportunity: Get A Job.ai is committed to inclusive hiring practices. We welcome applications from candidates of all backgrounds and work with clients who share our commitment to diversity and equal opportunity employment.