SDET – AI

Responsibilities

We are representing a confidential enterprise software organization seeking an SDET specializing in AI quality assurance. This is a pioneering role focused exclusively on testing GenAI and agentic systems in London.

In this position, you will:

• Develop and execute end-to-end automated test coverage across evaluations, APIs, UIs, and performance tests using Python
• Implement testing strategies for GenAI features, including conversational AI, agentic systems, and LLM-powered workflows
• Run and maintain automated test suites for prompt testing, including regression tests designed to detect unintended changes in model behavior
• Support and maintain evaluation frameworks to measure GenAI performance across accuracy, relevance, safety, consistency, and latency
• Contribute to curating high-quality test datasets and "golden examples" that represent diverse user scenarios and edge cases
• Assist in adversarial testing to identify potential model hallucinations or biases, and support integration of production alerting systems
• Utilize internal tools and testing frameworks to help developers validate their GenAI code and assist with user acceptance testing
• Take accountability for your workstream's automated coverage, documenting testing procedures and known issues in clear, accessible formats

What We're Looking For

Our client requires candidates with:

• Solid QA or test engineering experience, with exposure to or keen interest in AI/ML systems
• Hands-on Python coding experience
• Good understanding of GenAI technologies including LLMs, prompt engineering, and AI application patterns
• Solid experience with test automation frameworks and scripting (Python, Pytest, API, and UI testing)
• Practical knowledge of software testing methodologies (functional, integration, regression, performance, security testing)
• Ability to execute test cases and evaluation criteria for non-deterministic systems
• Experience with API testing tools (Postman, REST Assured) and backend testing
• Familiarity with CI/CD pipelines and automated testing integration

How We Work With You

When you apply through Get A Job.ai, our talent team will review your qualifications and conduct an initial screening. If there's a strong match, we will submit your profile to our client for consideration. The client will then conduct their interview process directly with qualified candidates.

Please note: do not contact the client directly. All applications and communications should go through Get A Job.ai to ensure proper consideration.

Pay

Compensation details will be discussed during the screening process with our recruitment team.

Equal Employment Opportunity

Get A Job.ai is committed to fostering an inclusive recruitment process. We welcome candidates of all backgrounds and provide reasonable accommodations as needed throughout our hiring process.