Senior QA Automation Engineer – AI

About This Opportunity

We are representing a confidential enterprise software organization that is building cutting-edge AI-powered features into their core platform. They are seeking a Senior QA Automation Engineer to pioneer quality assurance practices specifically for their GenAI and agentic systems. This is a unique opportunity to shape testing strategy for LLM-powered applications in a mature, global technology company.

This position is based in London and offers the chance to work at the intersection of quality engineering and artificial intelligence, developing frameworks and methodologies that don't yet exist in traditional QA playbooks.

Responsibilities

• Design and architect comprehensive test plans, evaluation frameworks, and quality metrics designed specifically for LLM-powered applications and agentic workflows
• Develop robust, end-to-end automated test suites in Python across evaluations, API checks, UI checks, and performance tests
• Create automated prompt testing systems and regression suites to proactively detect unintended changes in model behaviour
• Establish multi-dimensional evaluation criteria to measure and track GenAI quality across accuracy, relevance, safety, consistency, and latency
• Perform adversarial testing (red-teaming) to identify and mitigate model hallucinations, biases, or security vulnerabilities
• Develop internal tools and testing frameworks that enable software engineering teams to seamlessly validate their own GenAI implementations
• Deploy real-time monitoring and alerting systems to quickly detect and flag quality degradation of AI features in production
• Own the overall quality and testing strategy for your workstream, maintaining high-fidelity test datasets and defining QA best practices

What We're Looking For

• Deep understanding of GenAI technologies including LLMs, prompt engineering, and AI application patterns
• Strong hands-on Python coding experience
• Expert experience with test automation frameworks and scripting (Python, JavaScript, Selenium, Pytest)
• Strong background in software testing methodologies including functional, integration, regression, performance, and security testing
• Proven ability to design test cases and evaluation criteria for non-deterministic systems
• Strong analytical and problem-solving skills with attention to detail
• Experience with API testing tools (Postman, REST Assured) and backend testing
• Strong experience with CI/CD pipelines and automated testing integration

How We Work With You

When you apply through Get A Job.ai, our talent team will review your qualifications and conduct an initial screening to understand your experience with AI testing and automation. If there's a strong match, we'll prepare and submit your profile to our client for consideration. You'll be guided through each stage of the process by our recruiters.

Important: Please do not attempt to contact the client directly. All communication and interview coordination will be managed through Get A Job.ai to ensure the best candidate experience.

Pay

Compensation details will be discussed during the screening process based on your experience level and the client's approved range for this position.

Equal Opportunity: Get A Job.ai is committed to inclusive hiring practices. We welcome applications from candidates of all backgrounds and will ensure that individuals with disabilities receive appropriate accommodation throughout the recruitment process.