Senior Software Engineer – AI Coding Benchmarks

Mindrift offers specialists the chance to contribute to advanced AI evaluation through project-based work in Poland. This role focuses on creating rigorous benchmarks that test and refine how artificial intelligence models perform on real-world software development challenges.

Role Overview

You'll be instrumental in building evaluation frameworks for AI coding agents—systems that attempt to solve authentic developer tasks. Your work determines what "success" means for these AI systems and ensures evaluations are fair, challenging, and reflective of actual software engineering scenarios. This is specialist work requiring deep technical judgment, not routine task creation.

What You'll Do

• Construct authentic developer environments complete with codebases, infrastructure, documentation, and realistic project history that form the context for evaluation tasks
• Author evaluation tasks set within these environments, specifying the problem statement, the criteria for a valid solution, and boundary conditions that distinguish correct from incorrect work
• Develop comprehensive test suites that validate AI-generated solutions fairly—accepting multiple valid approaches while reliably catching genuine errors
• Review agent performance against your evaluation frameworks, identify where and why systems fail, and refine tasks and acceptance criteria to improve the benchmark's utility

Requirements

• Minimum 5 years of professional software development experience
• Proficiency with key technologies: Python (FastAPI framework), JavaScript or TypeScript (React), containerization (Docker), relational databases (Postgres), message streaming (Kafka), and caching systems (Redis)
• Demonstrated ability to design and implement test strategies covering functional and integration testing
• English communication at B2 level or higher; CV submission in English required

Why Join Mindrift

Contribute directly to AI safety and capability evaluation. The work demands solving genuinely difficult problems—frontier models are already sophisticated at coding, making it non-trivial to design tasks that meaningfully test their limits. You'll work on a defined scope of approximately 20 hours per project task with full schedule flexibility, earning compensation up to $40 per hour equivalent based on your experience and output quality.

Applying for This Role

To apply, complete your application directly on this page, or you'll be redirected to the employer's application platform to finish submitting there.