AI Engineer / AI Software Engineer-Task Creator (RL Environments) (Contract)

Role Overview

Careerflow Human Data Labs partners with AI companies to bring real-world professional expertise into their products. You will build hard, realistic tasks used to test how well AI agents perform real work on a computer. Each task is a small scenario running on a Linux desktop involving real files, applications, and documents, paired with a program that automatically evaluates the agent's output.

Responsibilities

• Design realistic multi-step scenarios across multiple applications.
• Assemble necessary files such as spreadsheets, documents, emails, and data.
• Write clear, complete instructions for AI agents without providing giveaways.
• Write Python scripts to set up environments and automate result scoring.
• Test, run, and provide short documentation for each task.

Requirements

• Proficiency in Python for writing and debugging scripts.
• Heavy user of AI coding tools and models.
• Background in AI/ML, including experience with AI projects, companies, or agents and evaluations.
• Comfort with Ubuntu, virtual machines, and terminal usage.
• Strong attention to detail and precision in instruction writing.
• Basic knowledge of Git.

Nice to Have

• Experience building RL environments where agents act and programs score outcomes.
• Experience building tasks or benchmarks for computer-use agents.
• Knowledge of OSWorld or similar computer-use benchmarks.

Skills

• Python
• AI/ML
• Ubuntu
• Git
• RL Environments