Staff Quality Engineer

About This Opportunity

We are representing a confidential AdTech organization seeking a Staff Quality Engineer to join their AI and intelligent systems team. This is a remote-first role open to candidates across Canada and the United States.

As the most senior quality engineer on the team, you will set the technical direction for testing, evaluating, and shipping AI-powered systems. This is a hands-on, individual-contributor leadership role where you'll design quality strategies for non-deterministic AI systems, build testing frameworks, and raise the quality bar across fast-moving, cross-functional agile teams. Your work will directly impact product releases and help define what quality means for AI products at our client.

Responsibilities

• Own and drive end-to-end quality strategy for AI products, from test architecture through release readiness
• Design and build evaluation frameworks for non-deterministic systems, including LLM output quality, agent behavior, regression detection, and guardrail/safety testing
• Develop reliable, scalable automated test suites covering functional, system, API, integration, UI, and regression testing; integrate into CI/CD pipelines
• Establish quality metrics and benchmarks (accuracy, latency, cost, hallucination/failure rates) that enable confident releases and data-driven decisions
• Partner with Engineering Managers, Product Managers, Designers, ML/Data Science teams, and other engineering groups to align requirements and priorities
• Act as technical mentor across the QE organization, promoting best practices and leveling up AI system testing company-wide
• Participate in technical planning, code review, and architectural decision-making, advocating for testability and reliability from design stage
• Provide timely, actionable test feedback to engineering teams and stakeholders

What We're Looking For

• 7+ years of professional software quality and test automation experience with a track record of setting technical direction on complex systems
• Deep hands-on experience with API and UI test automation frameworks such as Playwright, Cypress, Jest, or JUnit
• Fluency in TypeScript, JavaScript, Python, or Java (TypeScript preferred)
• Experience testing AI/agentic systems (evaluations, prompt/regression testing, output validation) and using AI to scale automation (test generation, self-healing tests, failure triage) with technologies like OpenAI or Anthropic
• Experience with CI/CD tools such as Jenkins, GitHub Actions, or CircleCI
• Strong understanding of scalable system design for web-based APIs and backend processing systems
• In-depth knowledge of QA methodologies in fast-release environments
• Proven ability to define test case design standards and coverage strategies including edge/negative cases, compatibility, performance, and concurrency scenarios
• Experience owning test case management practices (e.g., TestRail) that others can build upon
• Knowledge of React, including adding data-testids
• Ability to explore codebases and use technical tools to understand implementation
• Experience coaching cross-functional teams on tooling, testing plans, and strategies

How We Work With You

Our talent team at Get A Job.ai will guide you through the entire process. Here's how it works:

• Apply directly through the Get A Job.ai platform
• A recruiter from our team will screen your application and conduct an initial conversation
• We will submit qualified candidates to our client for consideration
• Please do not contact the client organization directly

Pay

Pay via Get A Job.ai will be competitive and commensurate with experience. Our client offers a comprehensive benefits package including health benefits from day one, retirement savings programs, generous paid time off, mental health care programs, work-from-home reimbursements, professional development support, parental leave, and optional coworking memberships.

Get A Job.ai is an equal opportunity employer. We welcome candidates of all backgrounds, experiences, and identities. If you require accommodations during the interview process, please let our team know.