Role Overview
Waymo's Semantics Onboard team within Perception is building onboard multi-task, multimodal perception models for the Waymo Driver. The role brings foundation model reasoning to autonomous driving, tackling complex "long-tail" scenarios through distillation and quantization for real-time, onboard intelligence under strict latency constraints.
Responsibilities
- Architect and train large-scale, onboard ML perception models for vehicle safety and regulatory compliance.
- Drive cross-functional collaboration to engineer robust, high-reliability training pipelines in a rapid-delivery environment.
- Leverage deep computer vision expertise to design novel, custom architectures from first principles for complex perception challenges.
- Contribute to a positive team culture valuing diverse skill sets and backgrounds.
Requirements
- BS or MS in Computer Vision, Machine Learning, Robotics, or a related field.
- Fluency in Python or C++, with deep hands-on expertise in PyTorch or Jax for matrix manipulation and module implementation.
- Deep understanding and proven practical experience with model distillation frameworks and quantization techniques for real-time compute constraints.
- Demonstrated hands-on experience building, training, or deploying Multimodal Foundation Models or Vision-Language Models (VLMs).
Preferred Skills
- PhD in Computer Vision, Machine Learning, Robotics, or a related field.
- 2+ years of applied industry experience in autonomous vehicles, robotics, or complex ML systems.
- Hands-on experience managing and optimizing large-scale teacher-student training loops.
- Proven track record deploying Vision-Language queries in highly constrained, real-time environments.
- Experience with large-scale distributed training, PEFT, or RLHF for Foundation Models and VLMs.
- Deep expertise in long-context temporal reasoning for sequential decision-making or complex video understanding.
- First-author publications in premier CV/ML conferences, such as CVPR, NeurIPS, ICCV, or ECCV.