Role Overview
Design and ship AI-powered product features including LLMs, RAG, agents, and ML APIs into existing services, working closely with backend, frontend, and data science teams.
Responsibilities
- Integrate off-the-shelf and inhouse models into robust microservices and user facing flows.
- Design and implement RAG and workflow/agent pipelines including retrieval, context assembly, tools integration, guardrails, and fallbacks.
- Own AI service reliability in production: latency, throughput, cost, observability, circuitbreakers, and rollback/versioning of models and prompts.
- Collaborate with Data Scientists to productionize model training/finetuning outputs as stable APIs/workflows.
- Implement logging, feedback capture, and lightweight online evaluation hooks.
- Ensure safety, security, and compliance: prompt injection defenses, PII handling, and hallucination controls.
- Contribute to internal AI tooling, SDKs, and reusable components.
Requirements
- Strong software engineering in Python (and one of Node.js, Java, or Go).
- Experience with REST/gRPC APIs, queues, and microservices on cloud infra.
- Hands-on experience shipping at least one AI powered product to production.
- Practical knowledge of LLM concepts: prompts, context engineering, embeddings, vector search, and evaluation metrics.
- Understanding of integration patterns with third party AI providers (OpenAI, Anthropic, etc.) and vector DB.
- Hands-on understanding of agentic frameworks like LangGraph.
Skills
- Python
- LangGraph
- LLMs
- RAG
- Vector Databases