Role Overview
You will help build the next generation of DoorDash’s ML serving platform, connecting request routing and online feature retrieval with model inference on CPU and GPU infrastructure. This role focuses on solving demanding inference infrastructure problems, improving systems that route prediction requests and execute models at scale, and enhancing the disaggregated serving architecture.
Responsibilities & Requirements
- Build self-serve infrastructure enabling modelers to deploy, configure, validate, and operate models independently across business verticals.
- Evaluate and integrate evolving inference frameworks, bringing open-source models into production with reliable, efficient services.
- Improve routing, feature retrieval, and model execution components to scale independently as workloads change.
- Enhance Kubernetes deployment and autoscaling to meet latency and availability requirements while optimizing CPU and GPU usage.
- Create integrations, validation, and migration tooling to simplify adoption and rollout of new serving capabilities.
- Maintain performance and reliability through benchmarking, profiling, metrics, and tracing to reduce bottlenecks and improve on-call automation.