Postman logo for AI Engineer, Intern job on DevsUnite

AI Engineer, Intern

Postman

Actively HiringInternship
Apply
Location

San Francisco, CA

Experience

0-2 years

Role

AI Engineer, Intern

Salary

Competitive

Share this job

Know someone who'd be a great fit?

Job Description

Role Overview

Postman is seeking an AI Engineer Intern to work with its AI team on large-scale AI and agentic systems, from data pipelines to production deployment. You will own discrete pieces of real systems under senior mentorship, not shadow work or isolated coursework-style projects.

Responsibilities

  • Contribute to APIFlow-Bench: design and review benchmark tasks and mock API environments, extend the Python evaluation harness and task-generation pipeline, and help maintain the public multi-model leaderboard with statistical confidence intervals.
  • Help build an action-level AI safety benchmark that scores what agents do in simulated enterprise API environments; work on scenario design, threat modeling (prompt injection, data exfiltration, permission overreach), and auditable evaluation.
  • Fine-tune open-weight models for tool calling and agentic tasks using PyTorch (SFT, distillation, RL) on managed platforms and self-managed cloud GPUs; run rigorous experiments, ablations, and error analysis, and report results and cost honestly.
  • Evaluate ultra-low-bit quantized models for on-device use; extend quantized vs. full-precision comparisons and analyze where and why they diverge.
  • Help build Postman’s in-product AI agent (Agent Mode) with a minimal architecture that calls LLM APIs directly (tool loops, multi-step execution, checkpointing), primarily in TypeScript; read open-source agent harnesses and turn learnings into design specs and prototypes.
  • Document experiments, design decisions, and runbooks; flag safety, fairness, or privacy concerns in model or agent behavior.

Requirements

  • Currently pursuing a BS, MS, or PhD in Computer Science, Data Science, or a related quantitative field.
  • Hands-on experience training or evaluating ML models through course projects, research, hackathons, or a prior internship.
  • Solid Python fundamentals: data structures, functions, basic testing; comfortable writing and reviewing code outside notebooks.
  • Working knowledge of at least one deep-learning framework (PyTorch preferred).
  • Clear written and verbal communication, with a habit of documenting what you build.

Nice to Have

  • Experience fine-tuning open-weight LLMs (SFT, LoRA, RL, or distillation) with benchmark-measured improvement.
  • Experience building LLM agents (tool calling, multi-step loops) or LLM evaluation harnesses/benchmarks with statistical rigor.
  • Track record shipping real software end-to-end: APIs and services, CLIs, Docker, CI/CD, cloud; public code on GitHub is a plus.
  • Interest or experience in AI safety and robustness: red-teaming, prompt injection, agent security, fairness, or interpretability.
  • Exposure to model-efficiency work: quantization, low-bit inference, or serving optimization.

Required Skills

PythonPyTorchTypeScriptLLM fine-tuningDocker

Ready to apply for AI Engineer, Intern?

See how you stack up at Postman before you apply, or go straight to the application.

Apply Anyway