Python Inference Engineer

About This Role

We are representing a confidential cloud infrastructure organization that is building production inference systems for AI and machine learning workloads. We're seeking a Python Inference Engineer to join their platform engineering team and work on large-scale model deployment infrastructure.

Responsibilities

• Build and improve the inference layer of a production AI platform serving models at scale
• Integrate and operate inference frameworks including vLLM, SGLang, and TensorRT-based systems
• Bring new language and multimodal models into production environments
• Optimize inference latency, throughput, memory usage, GPU utilization, and cost efficiency
• Debug complex performance and reliability issues across model code, inference frameworks, GPU execution, networking, and Kubernetes
• Collaborate with platform, infrastructure, product, and customer-facing teams to deliver reliable features
• Contribute improvements back to open-source inference projects where appropriate

What We're Looking For

Required:

• 5+ years of experience writing reliable, well-tested production code
• Strong Python skills and experience designing production systems
• Hands-on experience with PyTorch and deploying machine learning models
• Experience with Linux, Docker, and Kubernetes in production environments
• Experience in at least one area: distributed systems, GPU computing, ML runtimes, model optimization, or cluster scheduling
• Ability to debug complex problems spanning software, infrastructure, and hardware layers
• Strong sense of developer experience—you think about how systems are used, not just how they work
• Genuine interest in inference engineering and motivation to learn new frameworks
• Good communication and collaboration skills

Nice to Have:

• Experience with vLLM, SGLang, NVIDIA-based frameworks, or similar inference engines
• Experience running GPU workloads in production
• Knowledge of inference techniques: quantization, continuous batching, speculative decoding, prefix caching, chunked prefill, or LoRA serving
• Experience with CUDA, Triton, TensorRT, or other GPU programming tools
• Experience profiling and improving model performance metrics
• Experience with distributed inference, multi-GPU systems, or autoscaling
• Contributions to open-source ML, inference, or infrastructure projects

Location and Work Arrangement

This role is based in Cyprus or Serbia with flexible hybrid or remote options available depending on the specific position requirements.

How We Work With You

As a specialized recruiting firm, our talent team partners with leading technology organizations to connect exceptional engineers with transformative opportunities. When you apply through Get A Job.ai, a recruiter from our team will review your background and conduct an initial screening conversation. If there's a strong match, we will submit your profile to our client and coordinate all interview stages. Please apply exclusively through our platform—do not contact the client organization directly, as all applications must come through our official recruitment process.

Pay

Compensation details will be discussed during the screening process and vary based on experience level and location.

Equal Opportunity

Get A Job.ai is committed to equal opportunity employment. We evaluate all candidates based on qualifications and merit, without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or other legally protected characteristics.