Talent.com
Mindrift
Freelance Agent Evaluation EngineerMindrift • Paris, Île-de-France, FR
Freelance Agent Evaluation Engineer

Freelance Agent Evaluation Engineer

Mindrift • Paris, Île-de-France, FR
Il y a 5 jours
Type de contrat
  • Temps partiel
  • CDI
  • Télétravail
  • Quick Apply
Description de poste

Please submit your CV in English and indicate your level of English proficiency.


Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.


We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks.

You'll create challenging tasks and evaluation criteria within realistic simulated environments:

  • Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
  • Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent
  • Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient
  • Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust

What this is NOT:

  • Not data labeling
  • Not prompt engineering
  • Not writing code from scratch - the agent writes most of the code; you guide and evaluate

What we look for:

  • 5+ years in software development
  • Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
  • Experience writing tests (functional, integration)
  • English proficiency - B2+

Why this is hard:
Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.

Requirements and benefits

Educational qualifications

  • A Master’s Degree in Computer Science, Software Engineering, Data Science / Data Analytics, Artificial Intelligence / Machine Learning, Computational Linguistics / Natural Language Processing (NLP), Information Systems or other related fields.
  • Bachelor’s degree is accepted if only candidate has 5 years of experience in the field.

Academic and/or Professional Experience

Candidates should have a minimum of 3 years of professional experience in related roles or domain - specifically for QA-automation/testing or cybersecurity roles

How it works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid

Compensation:
Paid per accepted task. Your rate depends on the qualification tier you reach and how efficiently you complete tasks — up to the equivalent of $50/hr. Because payment is per task, a faster pace raises your effective hourly rate.

Why this freelance opportunity might be a great fit for you?

  • Take part in a part-time, remote, freelance project that fits around your primary professional or academic commitments.
  • Work on advanced AI projects and gain valuable experience that enhances your portfolio. - Influence how future AI models understand and communicate in your field of expertise.
Créer une alerte emploi pour cette recherche

Freelance Agent Evaluation Engineer • Paris, Île-de-France, FR

Offres similaires

Qa/Qc Engineer – Suppliers | Inspection & Expediting (H/F) - €50.000 - €75.000 A Year

Vulcain Engineering GroupParis, France, FR
Temps plein

Location:France / International Projects - Contract Type:Permanent (CDI) - Sector:Energy / Industry / EPC ProjectsSTART:ASAPSALARY:€50,000 to €75,000 per year depending on profile and experienceVul... Voir plus

 • Offre sponsorisée

Site Reliability Engineer Freelance H/F - €150 - €960 Par Jour

KickloxParis, France, FR
Temps plein

Ingénieur Site Reliability recherché pour analyser les dysfonctionnements, développer le code Infra As Code, assurer le monitoring et collaborer avec les équipes. Voir plus

 • Offre sponsorisée

Project-Based Ai Evaluation Engineer - Up To $50 An Hour - Part Time - Remote

Tech Consulting FirmParis, France, FR
Temps partiel

A tech consulting firm based in France is looking for experienced developers to create challenging coding test cases for AI systems.This part-time, project-based opportunity requires strong skills ... Voir plus

 • Offre sponsorisée

Safe Release Train Engineer — Remote 2j, Cdi - €45.000 - €55.000 Par An

Consort GroupParis, France, FR
CDI

Une entreprise de services IT en Provence-Alpes-Côte d'Azur recherche un Release Train Engineer passionné par l’Agilité SAFe.Vous serez responsable de la coordination des équipes Agile et de l’anim... Voir plus

 • Offre sponsorisée

Customer Reliability Engineer - €60.000 - €80.000 Par An

HirestoneParis, France, FR
Temps plein

Ingénieur fiabilité client pour une scale-up en cybersécurité, responsable de la création d'équipe, du support client et de la documentation technique. Voir plus

 • Offre sponsorisée

Forward Deployed Engineer, Applied Ai - €205.000 - €220.000 A Year

AnthropicParis, France, FR
Temps plein

About AnthropicAnthropic’s mission is to create reliable, interpretable, and steerable AI systems.We want AI to be safe and beneficial for our users and for society as a whole.Our team is a quickly... Voir plus

 • Offre sponsorisée

Remote Ai Agent Engineer: Build Production-Grade Llm Automation - $30,000 - $60,000 A Year - Remote

JobgetherParis, France, FR
Temps plein

AI Agent Engineer to design and deploy AI-powered agents for marketing, operations, and patient engagement.Requires 5+ years of software development, Python/Node. Voir plus

 • Offre sponsorisée

Application Engineer - €50.000 - €65.000 A Year

IC’AlpsParis, France, FR
Temps plein

Application Engineer sought to develop product validation plans, assess application needs, and support product specification development. Voir plus

 • Offre sponsorisée

Pharma Equipment Solutions Sales Engineer – France (Remote) - €60.000 - €75.000 A Year

Uhlmann Pac-SystemeParis, France, FR
Temps plein

Sales Engineer sought to manage client relationships and drive commercial development in the French market. Voir plus

 • Offre sponsorisée

Research Engineer In Digital Solutions For Product Quality Management - €28.000 - €40.000 A Year

ArcelorMittalParis, France, FR
Temps plein

The Research Engineer will lead projects, develop predictive models, and create new digital services for product quality management within ArcelorMittal. Voir plus

 • Offre sponsorisée

Customer Solutions Engineer - €40.000 - €50.000 Par An

AskaraParis, France, FR
Temps plein

À propos d’AskaraAskara est une start-up SaaS en forte croissance qui développe un logiciel basé sur l’intelligence artificielle et piloté à la voix.Notre mission :simplifier le quotidien des denti... Voir plus

 • Offre sponsorisée

Bioimage Analysis Research Engineer - Tools & Training - €2.400 - €3.300 A Month

France - BioImagingParis, France, FR
Temps plein

Research Engineer needed in Paris to support bioimage analysis research.Development of image analysis tools and collaboration with biologists are required. Voir plus

 • Offre sponsorisée

Ai Research Engineer: Real-Time Alerts & Digital Twin - €2.290 - €2.486 A Month

UniversityParis, France, FR
Temps plein

A leading university in France is seeking a Research Engineer to develop an innovative AI-based tool for real-time data analysis.The role involves requirements analysis, functional specification, a... Voir plus

 • Offre sponsorisée

Strategic Supplier Development Engineer - €45.000 - €50.000 Par An

Société de solutions logistiquesParis, France, FR
Temps plein

Une société de solutions logistiques recherche un Responsable Approvisionnement à Toulouse.Ce poste implique la gestion d'un portefeuille de fournisseurs complexes, le suivi de leurs performances e... Voir plus

 • Offre sponsorisée

Ingénieur Cloudops Aws- Confirmé- Freelance H/F - €450 - €500 Par Jour

ArtemysParis, France, FR
Temps plein

Qui sommes-nous?On est uneESNspécialisée dans le domaine des infrastructures à taille humaine, avec plus de 30 ans d’histoire.Mais ce qui fait notre vraie différence ? On ne se contente pas de dire... Voir plus

 • Offre sponsorisée

The Nuclear Energy Agency - Ai Engineer - €7.782,4 A Month

Organisation for Economic Co-operation and DevelopmentGentilly, France, FR
Temps plein

Seeking an AI Engineer to design, develop, and deploy AI solutions using Azure AI Hub and Foundry for the Nuclear Energy Agency.Responsibilities include RAG pipelines, intelligent search, and workf... Voir plus

 • Offre sponsorisée

Founding Deployment Engineer - €50.000 - €100.000 A Year

LyskParis, France, FR
Temps plein

The Deployment Engineer will work with the founders to deploy prototypes with early customers, focusing on customer experience and technical enablement. Voir plus

 • Offre sponsorisée

Senior Ai Agent Engineer - Production, Paris/Remote - €90.000 Par An

Startup InnovanteParis, France, FR
Temps plein

Une startup parisienne recherche un expert en IA pour concevoir et orchestrer des architectures d'agents complexes.Expérience Python requise, télétravail partiel possible. Voir plus

 • Offre sponsorisée

The Nuclear Energy Agency - Ai Engineer - €7.782,4 A Month

Nuclear Energy AgencyGentilly, France, FR
Temps plein

Seeking an AI Engineer to design, develop, and deploy AI solutions using Azure AI Hub and Foundry, focusing on RAG, intelligent search, and workflow automation within NEA business processes. Voir plus

 • Offre sponsorisée

Ai Engineer - Fintech Up To 90k Hybride Paris - Up To €90.000 A Year

AlliaMontrouge, France, FR
Temps plein

Seeking an AI Engineer with strong software engineering skills to develop and ship AI-powered features for the accounting industry.This role involves designing, implementing, and iterating on AI so... Voir plus