AI Engineer, Product

About This Opportunity

We are representing a confidential AI organization seeking an AI Engineer, Product to join their team in Paris. You'll be embedded directly within a product team—working on domains like search, chat, documents, or audio—where you'll own AI quality end-to-end through rigorous evaluation, prompt engineering, and rapid experimentation.

This is a hands-on role for engineers who want to ship production AI features that matter. You'll define what "good" looks like, measure it, run experiments, and deploy what works while partnering closely with research teams to deliver measurable improvements in quality, latency, safety, and reliability.

Responsibilities

• Design and execute evaluations for your product area: reference tests, heuristics, and model-graded checks tailored to search relevance, chat quality, document understanding, or audio performance
• Define and track metrics that drive product success: task completion, helpfulness, hallucination detection, safety flags, latency, and cost
• Own prompt and orchestration design: write, test, and iterate on prompts and system prompts as core infrastructure work
• Run A/B tests on prompts, models, and configurations; analyze results and make data-driven rollout or rollback decisions
• Implement observability for LLM calls: structured logging, tracing, dashboards, and alerts
• Operate model releases: canary and shadow traffic, sign-offs, SLO-based rollback criteria, and regression detection
• Improve core behaviors in your product area—memory policies, intent classification, routing, tool-call reliability, or retrieval quality
• Create templates and documentation enabling other teams to author evaluations and ship safely
• Partner with research teams to diagnose regressions and lead post-mortems

What We're Looking For

• 3-4 years of professional experience as an ML engineer moving closer to product, or as a software engineer with production AI/ML experience
• Strong TypeScript or Python skills—both tracks are available depending on team fit
• Production LLM experience: prompts, tool/function calling, system prompts in real applications
• Hands-on expertise with evaluations and A/B testing; you can design metrics, not just execute them
• Comfortable implementing directly in product code, not only in notebooks
• Observability experience: logging, tracing, dashboards, alerting in production systems
• Product mindset: form hypotheses, run experiments, interpret results, and ship improvements
• Clear communication, autonomous work style, and focus on production impact over experimentation for its own sake

Ideal candidates also have:

• Safety systems experience: moderation, PII handling/redaction, guardrails
• Release operations: canary/shadowing, automated rollbacks, experiment platforms
• Prior work on search ranking, chat systems, document AI, or audio ML features

How We Work With You

Candidates apply directly through Get A Job.ai. Our talent team will conduct an initial screening, and if there's a strong match, we'll submit your profile to our client for consideration. Please do not contact the employer directly—all communication and coordination will flow through our recruiting team to ensure a smooth, professional process.

Pay

Compensation details will be discussed during the screening process and are competitive for the Paris market and this level of experience.

Equal Opportunity: Get A Job.ai is committed to inclusive hiring practices. We welcome applications from candidates of all backgrounds and work to ensure fair, equitable access to opportunities.