- Full Time
- Bengaluru, Karnataka
Kobie
About the team and what we will build together
We’re looking for an Lead AI QA Engineer with 6+ years of experience who thrives on designing test strategies and evaluation harnesses for production-grade, agentic AI systems in addition to experience in ETL. You have strong Python skills, hands-on experience testing LLM-powered features (prompt regression, tool/function-call validation, RAG correctness, and structured-output schema checks), and working knowledge of evaluation frameworks such as RAGAS, DeepEval, LangSmith or Langfuse. You are comfortable writing solid SQL, automating tests with PyTest, exercising APIs through Postman or REST clients, and shipping test pipelines using Git, Docker and CI tooling like Jenkins or GitHub Actions.
Kobie runs some of the largest loyalty programs in the world. We are building an internal agent platform on Dataiku that automates analyst workflows, surfaces insights from program data in Snowflake, and gives our teams an LLM-native way to work with complex loyalty logic. As an Lead AI QA Engineer on the India Tech Hub team, you will play a key role in protecting that platform — designing golden datasets, running LLM-as-judge and regression suites, and owning the quality bar for what goes to production. This is not a manual-only role: you will automate, build qa & automation strategies, roadmaps, instrument, monitor and partner closely with our U.S. AI & Innovation team and cross-functional partners across Engineering, Data, AI and Product.
To apply for this job please visit jobs.lever.co.
Explore Kobie online
Working in Bengaluru, India
Weather right now in Bengaluru, India: checking… · Local time: · Air quality: · Daylight: · UV index:
Bengaluru, also known as Bangalore, is the capital and largest city of the southern Indian state of Karnataka. As per the 2011 census, the city had a population of 8.4 million, making it the third most populous city in India and the most populous in South India. The Bengaluru metropolitan area had a population of around 8.5 million, making it the fifth most populous urban agglomeration in the country. It is located towards the southern end of the Deccan Plateau, at an altitude of 900 m (3,000 ft) above sea level. The city is known as India's "Garden City", due to its parks and greenery.
Karnataka is a state in the southwestern coast of India. It was formed as Mysore State on 1 November 1956, with the passage of the States Reorganisation Act, and renamed Karnataka in 1973. The state is bordered by the Lakshadweep Sea to the west, Goa to the northwest, Maharashtra to the north, Telangana to the northeast, Andhra Pradesh to the east, Tamil Nadu to the southeast, and Kerala to the so
🇮🇳 Relocation safety for India: High Risk — via Warnely, CC BY 4.0
National unemployment rate in India: 4.2% — via World Bank
- Elevation 921m (3,022 ft)
Source: Wikipedia (state)
Job details above are provided by the employer/source. The sections on this page are compiled from public data sources with AI assistance.
Accommodations: if you need a workplace accommodation to apply for or perform this job, see ADA.gov or EEOC.gov for guidance on your rights and how to request one.
Keep exploring on Get A Job.ai
Not quite the right fit? Your next opportunity is a click away.
- Browse all jobs
- More jobs by category
- Remote jobs you can do from anywhere
- Research typical pay for this role
- Set a job alert so new matches reach you first
- Upload your resume to apply faster
Hiring instead? Post a job and reach candidates searching right now.