Talent.com
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)Mindrift • Paris, Île-de-France, FR
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Mindrift • Paris, Île-de-France, FR
Il y a 24 jours
Type de contrat
  • Temps partiel
  • CDI
  • Télétravail
  • Quick Apply
Description de poste

Please submit your CV in English and indicate your level of English proficiency.


Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.What this opportunity involves:
We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks.


You'll create challenging tasks and evaluation criteria within realistic simulated environments:

  • Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
  • Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent
  • Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient
  • Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust

What this is NOT:

  • Not data labeling
  • Not prompt engineering
  • Not writing code from scratch - the agent writes most of the code; you guide and evaluate

What we look for:

  • 8+ years in software development
  • Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
  • Experience writing tests (functional, integration)
  • English proficiency - B2+

Why this is hard:
Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.

How it works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paidEffort estimate


Tasks for this project are estimated to take 30 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

Compensation:
Up to $200/hr equivalent, depending on level and pace. Tasks are estimated at ~30 hours each; you set your own schedule.

Créer une alerte emploi pour cette recherche

Senior Python Engineer - AI Coding Agent Evaluation (Freelance) • Paris, Île-de-France, FR

Offres similaires

Senior Python & AI Consultant - ML, Data & Architecture

Noveo GroupNanterre, Île-de-France, France
Temps plein

Une société de conseil en transformation digitale recherche un Consultant Développeur Senior Python & IA pour rejoindre son équipe à Nanterre.Le candidat idéal aura une forte expérience en développ... Voir plus

 • Offre sponsorisée

Senior AI Platform Deployment Engineer

Avensys Consultingparis, île- e france, France
Temps plein

A technology consulting firm in Paris is seeking a highly experienced Senior Forward Deployment Engineer to lead the technical implementation of AI platforms in large-scale environments.This role r... Voir plus

 • Offre sponsorisée

Senior Software Data Engineer

HG Insightsparis, île- e france, France
Temps plein

HG Insights is the global leader in technology intelligence, delivering actionable AI‑driven insights through advanced data science and scalable big data solutions.Our Big Data Insights Platform pr... Voir plus

 • Offre sponsorisée

Platform Engineer – Ai & Cloud

LenstraParis, France, FR
Temps plein

Lenstra was created by the passion of engineers specialised in Computer Science with a proven history in delivering top quality solutions to its customers.Bringing together work excellence and visi... Voir plus

 • Offre sponsorisée

Senior Fullstack Engineer — Ai-Pentesting Platform (Hybrid) - €80.000 - €105.000 A Year

Ai-Pentesting PlatformParis, France, FR
Temps plein

Senior Full-Stack Engineer needed in Paris to build scalable products, design schemas, and develop AI workflows.Requires 5+ years experience with TypeScript and PostgreSQL. Voir plus

 • Offre sponsorisée

Senior Software Engineer - Python / TS - Paris

Symbiotic Securityparis, île- e france, France
Temps plein

Senior Software Engineer - Python / TS - Paris /h3 p3 days ago Be among the first 25 applicants /p pDirect message the job poster from Symbiotic Security /p h3Talent Acquisition Partner at Symbioti... Voir plus

 • Offre sponsorisée

Senior Python Backend Engineer — Scale, Ai & Hybrid Work - €80.000 Par An

Une entreprise innovante en sécurité du codeParis, France, FR
Temps plein

Une entreprise innovante en sécurité du code recherche un développeur backend ou fullstack avec 5 à 10 ans d'expérience, maîtrisant Python et idéalement React.Vous travaillerez sur des probléma... Voir plus

 • Offre sponsorisée

Senior Software Engineer (Python) - Editeur Saas Leader Dans La Sécurité - Paris - €80.000 Par An

Leader mondial de la sécurité du codeParis, France, FR
Temps plein

Ingénieur logiciel expérimenté en Python, rejoignant une entreprise leader dans la sécurité du code pour travailler sur l'IA, la performance et la scalabilité de nouveaux produits SaaS. Voir plus

 • Offre sponsorisée

Senior Software Engineer (Python/Django) - (F/H/X) - CDI

Krakenparis, île- e france, France
Temps plein

Kraken Technologies is looking for a senior software engineer to work on the French energy market.The role focuses on domain modelling, building robust and fault‑tolerant software for smart meter d... Voir plus

 • Offre sponsorisée

Développeur Python Senior

Global Market SolutionsParis, France, FR
Temps plein

Nous recherchons un(e) Développeur Fullstack Python Angular Senior pour l'un de nos clients grands comptes, pour intégrer l'équipe chargée du support, de l'évolution et de la gestion de... Voir plus

 • Offre sponsorisée

Senior Applied Ai Researcher — Post-Training (Rl) & Evaluation

LumosParis, France, FR
Temps plein

About LumosLumos is a health and life-science AI evaluation company based in San Francisco.Our mission is to make health and life-science AI genuinely accurate and safe.We build the benchmarks, RL ... Voir plus

 • Offre sponsorisée

Senior Python Backend Engineer — Scale, Ai & Hybrid Work - €80.000 Par An

KatchMeParis, France, FR
Temps plein

Une entreprise innovante en sécurité du code recherche un développeur backend ou fullstack avec 5 à 10 ans d'expérience, maîtrisant Python et idéalement React.Vous travaillerez sur des probléma... Voir plus

 • Offre sponsorisée

AI Architecture Senior Lead - Cloud & LLMs

Orishaparis, île- e france, France
Temps plein

Une entreprise internationale cherche un Tech Lead pour concevoir et faire évoluer des architectures IA.Vous aurez pour mission de guider les équipes vers des standards solides tout en travaillant ... Voir plus

 • Offre sponsorisée

Ai Engineer Senior

AgorastoreMontreuil, France, FR
Temps plein

Leader des enchères BtoB et BtoG, nous propulsons notre stratégie IA à une nouvelle étape.Pour accompagner notre ambition produit au sein du Groupe Estela (Agorastore, Clictelia, VH Auctions), nous... Voir plus

 • Offre sponsorisée

Senior Fullstack Engineer | Ai Pentesting | Paris Office - €80.000 - €105.000 A Year

EscapeParis, France, FR
Temps plein

We're Escape, YC-backed, Series A, growing 15%+ month-over-month.Our AI-powered platform does automated penetration testing for companies likeZoom ,Air France‑KLM ,Vanta, andMiro.We find vulner... Voir plus

 • Offre sponsorisée

Quantitative Developer Python - Alpha Signals

Capital Fund Management (CFM)paris, île- e france, France
Temps plein

In collaboration with the Research teams, the Front Prediction team develops and maintains the prediction models (alpha signals) used to make decisions for our automated trading systems.Overview /h... Voir plus

 • Offre sponsorisée

Senior Python Engineer - Remote Ai Testing & Docker - Up To $50 An Hour - Remote

AI project consultancyParis, France, FR
Temps plein

Seeking a Senior Python Engineer for AI testing and Docker environment management.Requires 5+ years of experience, strong Python/Docker skills, and ability to work remotely. Voir plus

 • Offre sponsorisée

Senior Software Engineer: Quant Modeling & Cloud CI/CD

twentyAIparis, île- e france, France
Temps plein

A global leader in financial markets infrastructure is seeking a Software Developer to enhance a model development framework and collaborate with quantitative teams.Candidates should have 5+ years ... Voir plus