Role Overview
Prior Labs builds tabular foundation models and joined SAP as a frontier AI lab. As ML Engineer, Backend, you will own designing, building, and scaling core systems that bring foundation models to production, working directly with AI researchers to translate cutting-edge models into scalable serving infrastructure.
Responsibilities
- Own the inference serving path end-to-end: latency, throughput, batching, memory, and cost per prediction across managed API, customer VPCs, and self-hosted open weights.
- Design robust, scalable, and secure backend systems and production-grade APIs for serving foundation models.
- Develop high-quality, maintainable code for core backend services.
- Design, deploy, and manage cloud infrastructure focused on reliability, monitoring, observability, and cost-efficiency.
- Implement secure, GDPR-compliant systems including data storage, access control, usage tracking, and quota management.
- Drive engineering best practices for testing, CI/CD, documentation, and security.
Requirements
- 3+ years of professional experience in ML and backend engineering with production infrastructure.
- Proven experience deploying and operating ML models in production under real traffic.
- Hands-on experience designing and building production-grade APIs and backend services.
- Significant experience building and operating services on cloud platforms, with GCP preferred.
- Hands-on experience with Infrastructure as Code using Terraform.
- Experience with containerization and orchestration technologies: Docker and Kubernetes.
- Proficiency in Python.
Nice to Have
- ML infrastructure experience: MLOps, model serving frameworks, feature stores, data pipelines.
- Inference optimization: quantization, compilation, dynamic batching, caching.
- Experience with PyTorch and FastAPI.
- Open-source contributions.