AI Benchmark Engineer | Native Language Specialist – French (France) – Remote

About This Opportunity

We are representing a confidential AI technology organization that is building a rigorous evaluation framework to test large language models on multilingual software engineering challenges. They need experienced native French-speaking software engineers to design and validate benchmark tasks that genuinely measure how well AI systems handle French-language terminal workflows, data processing, and locale-specific edge cases.

This is a remote, freelance opportunity offering flexible scheduling. You'll work as an independent contractor, contributing when your schedule allows.

Responsibilities

• Design and engineer evaluation tasks that test AI coding agents in authentic French-language environments
• Create realistic task assets using French datasets and files that remain in French throughout (no English translation shortcuts)
• Identify failure points where AI systems struggle with native French prompts and workflows
• Develop reference implementations and write deterministic verification scripts
• Analyze execution logs and calibrate task difficulty levels across different AI model tiers
• Participate in a multi-layer quality control process including creation, human review, calibration review, and audit phases
• Ensure benchmark integrity through rigorous testing of grammatical accuracy and fairness

What We're Looking For

• 5+ years of professional software engineering experience
• Background at recognized technology companies or graduation from top-tier engineering programs
• Native or near-native French fluency with deep understanding of grammar, register, and phrasing conventions
• High English proficiency for collaboration and documentation
• Strong Python skills and command-line/terminal workflow expertise
• Experience with coding agents and CLI-based development
• Technical understanding of multilingual text processing challenges including Unicode normalization, encoding/decoding, locale-dependent conventions (collation, casing, date formats), and safe string operations
• Ability to commit a minimum of 2 hours per day or 10 hours per week once tasks are accepted

How We Work With You

Candidates apply directly through Get A Job.ai. Our talent team will conduct an initial screening to understand your background and technical capabilities. You'll complete a GenAI skills assessment as part of the evaluation process. Qualified candidates will be submitted to our client for consideration. Please do not attempt to contact the client directly, as all communication is managed through our recruiting process.

This is a 1099 contractor position with fluctuating project availability, making it better suited as supplemental income rather than a primary role. No benefits such as health insurance, paid time off, or retirement contributions are provided. Work hours are not guaranteed and depend on project demand. You are responsible for your own tax obligations and we recommend consulting a tax professional.

Pay

Compensation details will be discussed during the screening process based on experience and project scope.

Equal Employment Opportunity: Get A Job.ai is committed to fair and inclusive hiring practices. We provide equal opportunity to all qualified candidates without regard to race, religion, color, national origin, sex, sexual orientation, gender identity, age, disability, veteran status, or any other protected classification.