Role Overview
The Speculative Decoding team at Together AI develops draft models and algorithms to accelerate large language model generation. As a research intern, you'll explore speculative decoding research questions, train and fine-tune draft models on GPU clusters, and test ideas against realistic serving workloads including long-context and agentic traffic.
Responsibilities
- Research and prototype new speculative decoding algorithms and draft model architectures
- Train, fine-tune, and distill draft models; design experiments to validate hypotheses
- Implement optimized versions for serving engines to demonstrate real-world efficacy
- Communicate project plans, progress, and results to the team
- Share findings through technical write-ups and blog posts, transitioning successful ideas to production
Requirements
- Currently pursuing a Master's or Ph.D. in Computer Science, Electrical Engineering, or related field
- Strong knowledge of Machine Learning and Deep Learning fundamentals
- Hands-on experience training/fine-tuning large models with distributed frameworks (e.g., Megatron-LM, FSDP)
- Familiarity with modern architectures like Mixture-of-Experts and hybrid attention