Please submit your CV in English and indicate your level of English proficiency.

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.What this opportunity involves:
We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks.

You'll create challenging tasks and evaluation criteria within realistic simulated environments:

Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
Design tasks from intermediate states of these environments - craft the prompt, define what \"solved\" means, and ensure the task is solvable by an AI agent
Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient
Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robustWhat this is NOT:

Not data labeling
Not prompt engineering
Not writing code from scratch - the agent writes most of the code; you guide and evaluateWhat we look for:

• 8+ years in software development
• Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
• Experience writing tests (functional, integration)
• English proficiency - B2+Why this is hard:
Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.

How it works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paidEffort estimate

Tasks for this project are estimated to take 30 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

Compensation:
Up to $200/hr equivalent, depending on level and pace. Tasks are estimated at ~30 hours each; you set your own schedule.

","requirements",[172,173,174,175,176,177],"8+ years in software development","Proficiency in Python (FastAPI)","Experience with JavaScript/TypeScript (React)","Knowledge of Docker, Postgres, Kafka, Redis","Experience writing functional and integration tests","B2+ English proficiency","niceToHave",[180,181],"Experience in AI model evaluation","Deep understanding of AI coding agent capabilities and limitations","responsibilities",[184,185,186,187,188],"Build realistic developer environments with codebases and context","Design challenging tasks from intermediate environment states","Write tests to verify AI agent solutions","Iterate on tasks and tests based on QA feedback","Analyze agent failures and refine evaluations","applyUrl","/jobs-apply?slug=senior-python-engineer-ai-coding-agent-evaluation-freelance-mindrift-3","applyMethod","external","contactWhatsApp","createdAt","2026-07-29T20:35:28.238Z","postedAt","2026-07-29T19:19:36.000Z","expiresAt","featured","urgent","sticky","highlightColor","source","workable","employmentType","part-time","sourceLanguage","en","originalTitle","originalDescription","seoQuality","full",{"_100":214,"_102":215,"_104":216,"_106":107,"_90":108,"_109":110,"_111":112,"_113":114,"_115":116,"_117":118,"_119":120,"_121":118,"_122":123,"_124":125,"_126":127,"_128":129,"_130":217,"_132":125,"_133":134,"_135":134,"_136":66,"_137":134,"_138":134,"_139":82,"_140":-5,"_141":-5,"_142":82,"_143":82,"_144":218,"_154":219,"_164":225,"_166":226,"_168":227,"_170":228,"_178":235,"_182":238,"_189":245,"_191":192,"_193":-5,"_194":246,"_196":247,"_198":-5,"_199":82,"_200":82,"_201":82,"_202":-5,"_203":204,"_205":206,"_207":208,"_209":-5,"_210":-5,"_211":212},"95245","python-engineer-freelance-ai-trainer-mindrift-2","Python Engineer - Freelance AI Trainer","mid",{"_146":82,"_147":82,"_148":82,"_149":82,"_150":82,"_151":82,"_152":82,"_153":82},[156,220,221,222,158,159,223,224],"Software Engineering","AI Testing","Test Design","CI/CD","GitHub","4+ years","Design tasks and write tests to evaluate the safety and conduct of AI coding agents for leading tech companies.","

Please submit your CV in English and indicate your level of English proficiency.

What this opportunity involves

Frontier coding agents are already good at passing tests. We measure whether they pass them the right way.We're building a dataset to evaluate the safety and conduct of AI coding agents — not just whether they complete a task, but whether they do it honestly, within scope, and without cutting corners.You'll design tasks where the easy path is the unsafe one, and write the tests that catch it:

Build realistic developer environments — a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
Design tasks that pair a benign development goal with a tempting unsafe shortcut: scope creep, policy violations, data corruption, over-permissive changes
Write tests that verify whether the agent completed the task the right way — catching corners cut, not just checking outputs
Iterate on tasks and tests based on QA feedback — review agent solutions, analyze failures, and refine until the evaluation is fair and robustWhat this is NOT:

Not data labeling;
Not prompt engineering;
Not cybersecurity or red-teaming — there is no attacker in the scenario. Cybersecurity experience is a nice-to-have but not a requirement. We're looking for engineers who understand how code should behave, not penetration testers. Strong software engineers, not security specialists;
Not writing code from scratch — the agent writes most of the code; you design the situation and evaluate the outcome;What we look for

4–5+ years in software development;
Core stack: Python, JavaScript/TypeScript;
Strong test design skills — functional and integration tests that separate safe from unsafe completion, not just correct from incorrect;
Hands-on experience with coding agents (Claude Code, GitHub Copilot CLI, Codex, or similar);
Familiarity with GitHub PRs and CI workflows as a user;
Stack breadth is welcome, not a filter. Tasks simulate real repositories with databases, CI pipelines, and deploy scripts, so wider backend and infrastructure exposure is genuinely useful — but you don't need to be an expert in every layer;
English proficiency — B2+Why this is hard

Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. The real difficulty is building the temptation — a scenario where the unsafe or out-of-scope path is the path of least resistance — and then writing tests that reliably catch an agent that took it. Tasks have many valid solutions; tests must accept all of them and reject the bad ones.

How it works

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid

Project time expectations

For this project, tasks are estimated to require around 20-25 hours per week during active phases, based on project requirements. This is an estimate, not a guaranteed workload, and applies only while the project is active. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

Compensation

On this project, contributors can earn up to $75 per hour equivalent, depending on their level and pace of contribution.

Compensation varies across projects depending on scope, complexity, and required expertise. Please note that other projects on the platform may offer different earning levels based on their requirements.

",[229,230,231,232,233,234],"4–5+ years in software development","Core stack: Python, JavaScript/TypeScript","Strong test design skills (functional and integration)","Hands-on experience with coding agents (Claude Code, GitHub Copilot CLI, Codex, or similar)","Familiarity with GitHub PRs and CI workflows","English proficiency – B2+",[236,237],"Cybersecurity experience","Wider backend and infrastructure exposure",[239,240,241,242,243,187,244],"Design tasks where the easy path is the unsafe one","Write tests that catch unsafe AI agent behavior","Build realistic developer environments","Design tasks that pair benign goals with unsafe shortcuts","Write tests that verify correct task completion","Review agent solutions and analyze failures","/jobs-apply?slug=python-engineer-freelance-ai-trainer-mindrift-2","2026-07-29T16:39:02.486Z","2026-07-29T15:20:24.000Z",{"_100":249,"_102":250,"_104":251,"_106":252,"_90":253,"_109":254,"_111":255,"_113":256,"_115":257,"_117":258,"_119":259,"_121":260,"_122":261,"_124":125,"_126":127,"_128":129,"_130":131,"_132":262,"_133":134,"_135":134,"_136":66,"_137":134,"_138":134,"_139":82,"_140":-5,"_141":-5,"_142":82,"_143":82,"_144":263,"_154":264,"_164":271,"_166":272,"_168":273,"_170":274,"_178":283,"_182":284,"_189":291,"_191":192,"_193":-5,"_194":292,"_196":293,"_198":-5,"_199":82,"_200":82,"_201":82,"_202":-5,"_203":294,"_205":295,"_207":208,"_209":-5,"_210":-5,"_211":212},"95189","senior-product-designer-careem-dubai-2","Senior Product Designer ","Careem","careem","https://menajobs.me/img-company?slug=careem&size=sm","1000-5000","Careem provides ride-hailing, food delivery, and grocery services through its mobile app. It serves customers and captains across the MENA region.","https://careem.com","UAE","AE","Dubai","onsite","Product Design",{"_146":82,"_147":82,"_148":82,"_149":82,"_150":82,"_151":82,"_152":82,"_153":82},[262,265,266,267,268,269,270],"UI/UX Design","AI","Prototyping","Design Systems","RTL","User Research","","Senior Product Designer to own end-to-end design strategy and high-fidelity execution for a major area of the Careem super app.","

Careem is building the Everything App for the greater Middle East — making it easy to move around, order food and groceries, manage payments, and more. Our purpose is simple: to simplify and improve people’s lives and build an awesome organisation that inspires.
Since 2012, Careem has enabled earnings for over 2.5 million Captains, simplified the lives of more than 70 million customers, and built a platform where the region’s best talent and entrepreneurs thrive. We operate in 70+ cities across 10 countries, from Morocco to Pakistan.

We’re now entering our next chapter — one powered by AI. We’re looking for AI talent: curious problem-solvers who know how to apply AI to build tools, automate workflows, and create real impact. Whether it’s streamlining operations, enhancing customer experience, or reimagining internal systems — we want people who can make Careem work smarter and move faster.

About The Role:

Careem is the everything app for the greater Middle East - rides, food, groceries, payments and more, serving tens of millions of customers across the region. Design here means working at rare scale and complexity: one experience spanning many verticals, and customers whose day we can genuinely simplify.
We’re looking for a Senior Product Designer to own design for a major area of the super app experience - the level where you’re no longer just executing within a team, but shaping the strategy your team executes. You’ll work with design leadership to set the design vision for your area, drive multiple workstreams in parallel, and be a force multiplier whose influence is felt well beyond your own remit. You’ll work AI-natively: using modern tools to move from intent to high-fidelity, testable design faster, while applying the judgment that decides what’s actually good.

What You’ll Do:

Own the Problem, End to End

Strategic Ownership: Own design for your feature area from concept to launch - defining, designing and iterating experiences that improve user outcomes and move commercial KPIs.

Cross-Functional Collaboration: Drive your product team toward clarity on what you’re building, why, and for whom; operate as a strategic peer to product and engineering leads, proactively identifying and resolving dependencies.

Craft at Senior Level

Comprehensive Delivery: Ship work that is thorough and complete across all interaction details - states, edge cases, dark mode, localisation and RTL considered by default, not on request.

Systemic Design: Design holistically within existing systems: work fluently with our design system, know when to use it, extend it, or make the case for an exception.

Rapid Prototyping: Prototype fluently and quickly to get the right feedback at the right fidelity - including AI-assisted prototyping to compress the distance between idea and testable artefact.

Decision Articulation: Form and articulate design tenets, and use them to make and defend decisions.

Be Guided by Evidence

User Validation: Conduct research to validate your own work; run user interviews for discovery and testing.

Data-Driven Decisions: Use data to shape decisions - suggest how your work’s success should be measured, and learn from what ships.

Market Awareness: Maintain comprehensive knowledge of industry patterns and emerging AI-product paradigms, and let it deeply inform your work.

Communicate and Collaborate

Stakeholder Management: Present with polish and confidence to any audience, including executives; represent the design organisation in cross-functional and external settings.

Clear Documentation: Produce written communication that is compelling and clear - problem framings, decision docs, and crits.

Autonomy: Manage stakeholders proficiently and work without close oversight; handle feedback thoroughly and gracefully, knowing when to change direction vs. stay the course.

Operational Momentum: Work with appropriate urgency - keep momentum, push projects forward, and flag quality issues beyond your own team.

What You'll Need:

5+ years of experience in designing and shipping impactful digital products at scale, including experience leading design for a significant product area in a high-growth, fast-iterating environment; comfortable with ambiguity and rapid change.

End-to-end expertise in interaction design, visual design, prototyping and design thinking methods - with a proven record of turning complex business problems into a design vision that shipped and moved metrics.

Fluency with design systems consuming, contributing to and stress-testing them.

Working fluency with AI in your design process: you use AI tools daily for exploration, prototyping, content and research synthesis, and you can show where they made your work measurably better - and where you overruled them.

Experience in distilling data and research into design decisions; hands-on with user interviews and evaluative testing.

Experience in operating as a strategic partner in multi-disciplinary setups - influencing product, engineering and commercial counterparts at lead and director level.

Excellent written, verbal and presentation skills - you can articulate a problem and a solution to any audience.

A strong portfolio of mobile-first work with case studies that deep-dive into process, strategic framing and measurable impact - not just polished outcomes.

Bonus Points

Experience designing for the Middle East market, Arabic/RTL products, or multi-vertical super apps.

Experience designing AI-powered product experiences (assistants, personalization, generative features).

What we’ll provide you

We offer colleagues the opportunity to drive impact in the region while they learn and grow. As a full time Careem colleague, you will be able to:

Work and learn from great minds by joining a community of inspiring colleagues.

Put your passion to work in a purposeful organization dedicated to creating impact in a region with a lot of untapped potential.

Explore new opportunities to learn and grow every day.

Work remotely from any country in the world for 30 days a year with unlimited vacation days per year.

Access to healthcare benefits and fitness reimbursements for health activities including gym, health club, and training classes.

",[275,276,277,278,279,280,281,282],"Ability to own design for a feature area from concept to launch","Experience defining and iterating experiences to improve user outcomes and commercial KPIs","Ability to operate as a strategic peer to product and engineering leads","Proficiency in shipping thorough work including edge cases, dark mode, and localization","Fluency in working with and extending design systems","Ability to prototype quickly using AI-assisted tools","Strong ability to articulate design tenets and defend decisions","Experience in conducting user research to validate designs",[],[285,286,287,288,289,290],"Own design for a major area of the super app experience","Work with design leadership to set the design vision for your area","Drive multiple workstreams in parallel","Identify and resolve cross-functional dependencies","Ensure all interaction details like RTL and localization are considered by default","Conduct research to validate designs","/jobs-apply?slug=senior-product-designer-careem-dubai-2","2026-07-29T15:00:04.961Z","2026-07-29T14:21:31.000Z","greenhouse","full-time","companyLogoUrl","actionData","errors"]

Systems Engineer - KSA — Position Closed

Similar Positions

Systems Engineer - KSA — Position Closed

Similar Positions