Role Overview
Scale partners with leading AI labs to advance generative AI research, seeking Research Scientists and Engineers expert in large language model post-training (SFT, RLHF, reward modeling) to build benchmarks and diagnose model failure modes in text and multimodal settings.
Responsibilities
- Analyze model behavior to identify, characterize, and diagnose failure modes in frontier LLMs and Agents, focusing on root cause analysis (RCA) of capability gaps, reasoning errors, robustness, and alignment issues.
- Design and build benchmarks and evaluation methods that measure LLM capabilities in both text and multimodal modalities.
- Apply post-training expertise (SFT, RLHF, reward modeling) to connect observed failures to the data and training interventions that address them.
- Publish research findings in top-tier AI conferences.
Requirements
- Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or a related field.
- Deep understanding of deep learning, reinforcement learning, and large-scale model fine-tuning.
- Experience with post-training techniques such as RLHF, preference modeling, or instruction tuning, and with LLM evaluation or benchmark development.
- Excellent written and verbal communication skills.
- Published research in areas of machine learning at major conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, etc.) and/or journals.
- Previous experience in a customer-facing role.