Role Overview
ByteDance's Inference Infrastructure team maintains AIBrix, a Kubernetes-native control plane for large-scale LLM inference. Part of Core Compute Infrastructure, the team builds cloud-native, GPU-optimized orchestration systems for hyperscale AI workloads. This graduate role focuses on LLM inference infrastructure, scheduling, and GPU acceleration across multi-cloud and global datacenters.
Responsibilities
- Design and build large-scale, container-based cluster management and orchestration systems.
- Architect next-generation cloud-native GPU and AI accelerator infrastructure for ML platforms.
- Collaborate on inference solutions using vLLM, SGLang, TensorRT-LLM, and other LLM engines.
- Stay current with open source (Kubernetes, Ray), AI/ML, and LLM infrastructure research.
- Write high-quality, production-ready, maintainable, testable, and scalable code.
Requirements
- Completing or recently completed a PhD in Software Development, Computer Science, Computer Engineering, or related technical discipline.
- Strong understanding of large model inference, distributed and parallel systems, and/or high-performance networking systems.
- Hands-on experience building cloud or ML infrastructure (resource management, scheduling, request routing, monitoring, orchestration).
- Solid knowledge of container and orchestration technologies (Docker, Kubernetes).
- Proficiency in at least one major programming language (Go, Rust, Python, or C++).