# Machine Learning Research Scientist, Evaluations at Scale AI

> Scale AI is hiring for Machine Learning Research Scientist, Evaluations in San Francisco, CA; Seattle, WA; New York, NY (Full-time, 2-4 years experience), posted on 14 October 2026.

Canonical: https://devsunite.com/jobs/scale-ai-machine-learning-research-scientist-evaluations-san-francisco-oct2026-4a3b7f
Last updated: 2026-10-14

- **Company**: Scale AI
- **Location**: San Francisco, CA; Seattle, WA; New York, NY
- **Employment type**: Full-time
- **Experience**: 2-4 years
- **Role**: Machine Learning Research Scientist, Evaluations
- **Salary**: Competitive
- **Posted**: 2026-10-14

## Required skills

- Machine Learning
- LLMs
- Generative AI
- Deep Learning
- RLHF

## Job description

### Role Overview

Scale partners with leading AI labs to advance generative AI research, seeking Research Scientists and Engineers expert in large language model post-training (SFT, RLHF, reward modeling) to build benchmarks and diagnose model failure modes in text and multimodal settings.

### Responsibilities

- Analyze model behavior to identify, characterize, and diagnose failure modes in frontier LLMs and Agents, focusing on root cause analysis (RCA) of capability gaps, reasoning errors, robustness, and alignment issues.
- Design and build benchmarks and evaluation methods that measure LLM capabilities in both text and multimodal modalities.
- Apply post-training expertise (SFT, RLHF, reward modeling) to connect observed failures to the data and training interventions that address them.
- Publish research findings in top-tier AI conferences.

### Requirements

- Ph.D. or Master&#x27;s degree in Computer Science, Machine Learning, AI, or a related field.
- Deep understanding of deep learning, reinforcement learning, and large-scale model fine-tuning.
- Experience with post-training techniques such as RLHF, preference modeling, or instruction tuning, and with LLM evaluation or benchmark development.
- Excellent written and verbal communication skills.
- Published research in areas of machine learning at major conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, etc.) and/or journals.
- Previous experience in a customer-facing role.

## How to apply

Sign in on the job page to apply: https://devsunite.com/jobs/scale-ai-machine-learning-research-scientist-evaluations-san-francisco-oct2026-4a3b7f

## Related

- [Scale AI jobs and internships](https://devsunite.com/companies/scale-ai)
- [Tech jobs in USA](https://devsunite.com/jobs-in-usa)
- [All jobs](https://devsunite.com/jobs)
