Role Overview
Contract remote opportunity for an open source contributor to create realistic technical tasks, build reproducible evaluation environments, review model-generated work, and diagnose model failures.
Responsibilities
- Create realistic technical tasks based on professional, scientific, or engineering workflows.
- Build reproducible task environments, reference solutions, tests, and objective evaluation criteria.
- Review model-generated work for correctness, completeness, and technical quality.
- Diagnose model failures and explain why an approach or result is incorrect.
Requirements
- Strong programming ability, including practical proficiency with Python.
- Meaningful contributions to open-source software or substantial original public projects.
- Experience in at least one relevant area, including software engineering, machine learning, or data science.
- Publicly verifiable evidence such as meaningful merged contributions to established repositories.