Sarvam logo for SDE2 - API Platform job on DevsUnite

SDE2 - API Platform

Sarvam

Actively HiringFull-time
Apply
Location

Bengaluru

Experience

3+ years

Role

SDE2 - API Platform

Salary

Competitive

Share this job

Know someone who'd be a great fit?

Job Description

Role Overview

Sarvam's API platform team seeks an SDE2 to build and evolve the LLM API platform, handling routing, streaming, reliability, and performance.

Responsibilities

  • Build and evolve the LLM API platform from edge request to response.
  • Lead the Python→Go rewrite while maintaining reliability.
  • Implement streaming infrastructure (SSE, long-lived connections, cancellation handling).
  • Design routing and failover with weighted provider chains, retries, latency-driven admission control.
  • Ensure OpenAI spec compatibility across inference servers (vLLM, SGLang, Dynamo).
  • Build metering, billing, and rate-limiting for the commercial layer.
  • Deliver performance and reliability for multi-tenant, long-context agentic workloads.
  • Implement observability (logging, metrics, tracing) for production debugging.
  • Develop integration and load testing harness for fast, safe releases.
  • Collaborate with Inference and MLOps teams on production readiness for new models.
  • Uphold design-first, documentation-first culture (RFCs, written decisions).

Requirements

  • 3+ years building production backend systems.
  • Strong Python in production: asyncio, FastAPI or similar, experience under real load.
  • Working knowledge of Go or strong willingness to learn; work in both languages during rewrite.
  • Experience building low-latency, multi-tenant backend services.
  • Hands-on experience with streaming systems: SSE or WebSockets, long-lived connections, backpressure.
  • Strong PostgreSQL and Redis fundamentals: schema design, query performance, caching, rate limiting.
  • Comfort running services on Kubernetes in production.
  • Ownership mindset: test before shipping, debug production with metrics/logs/traces, write RCAs.

Required Skills

LLMsPythonFastAPIPostgreSQLRedis

Ready to apply for SDE2 - API Platform?

See how you stack up at Sarvam before you apply, or go straight to the application.

Apply Anyway