SDET – AI

About This Opportunity

We are representing a confidential enterprise SaaS organization seeking a Senior SDET with specialized expertise in AI quality assurance. This is a pioneering role focused exclusively on testing and evaluation of generative AI and agentic systems. You'll architect testing strategies, develop evaluation frameworks, and establish quality metrics specifically designed for LLM-powered applications.

This position is based in London and offers the opportunity to shape quality standards at the intersection of software testing and artificial intelligence.

Responsibilities

• Design and architect comprehensive test plans, evaluation frameworks, and quality metrics for LLM-powered applications and agentic workflows
• Develop robust, end-to-end automated test suites in Python covering evaluations, API checks, UI checks, and performance tests
• Create automated prompt testing systems and regression suites to detect unintended changes in model behavior
• Establish multi-dimensional evaluation criteria to measure GenAI quality across accuracy, relevance, safety, consistency, and latency
• Perform adversarial testing (red-teaming) to identify and mitigate model hallucinations, biases, and security vulnerabilities
• Develop internal tools and testing frameworks enabling engineering teams to validate their own GenAI implementations
• Deploy real-time monitoring and alerting systems to detect quality degradation of AI features in production
• Own the overall quality and testing strategy for your workstream, maintaining high-fidelity test datasets and defining QA best practices

What We're Looking For

• Deep understanding of GenAI technologies including LLMs, prompt engineering, and AI application patterns
• Strong hands-on Python coding experience
• Expert experience with test automation frameworks and scripting (Python, JavaScript, Selenium, Pytest)
• Strong background in software testing methodologies including functional, integration, regression, performance, and security testing
• Proven ability to design test cases and evaluation criteria for non-deterministic systems
• Strong analytical and problem-solving skills with attention to detail
• Experience with API testing tools (Postman, REST Assured) and backend testing
• Strong experience with CI/CD pipelines and automated testing integration

How We Work With You

Candidates apply directly through Get A Job.ai. Our talent team conducts an initial screening to understand your background and ensure alignment with the role requirements. We then submit qualified candidates to our client for their interview process. Please do not attempt to contact the client directly, as all communication is managed through our recruiting team to ensure a professional and streamlined experience.

Pay

Compensation details will be discussed during the screening process based on experience and qualifications.

Equal Opportunity

Get A Job.ai is committed to inclusive hiring practices. We welcome candidates from all backgrounds and provide equal opportunity regardless of race, color, religion, sex, national origin, age, disability, or any other protected characteristic.