ByteDance logo for Agent Evaluation & Evolution Machine Learning Engineer Intern (AML-Ark-US) - 2027 Summer job on DevsUnite

Agent Evaluation & Evolution Machine Learning Engineer Intern (AML-Ark-US) - 2027 Summer

ByteDance

Actively HiringInternship
Apply
Location

Seattle, WA

Experience

0-2 years

Role

Agent Evaluation & Evolution Machine Learning Engineer Intern (AML-Ark-US) - 2027 Summer

Salary

Competitive

Share this job

Know someone who'd be a great fit?

Job Description

Role Overview

The Applied Machine Learning Ark team builds LLM service platforms offering Model-as-a-Service (MaaS) solutions, spanning LLM algorithms, training/fine-tuning/inference, prompt engineering, alignment, and intelligent agent systems. The US team operates large-scale log analytics pipelines and data-driven feedback loops for model improvement. This 2027 Summer internship provides hands-on experience applying ML knowledge to real-world LLM agent challenges.

Responsibilities

  • Design evaluation systems for LLM-based agents covering task success, tool use, reasoning quality, and reliability.
  • Build benchmarks and automated judging pipelines using rule-based checks, model-based judging, and human review.
  • Analyze agent execution traces and user feedback to identify failure patterns and drive system improvements.
  • Support the closed loop from experience to capability, collaborating with research, platform, and product teams to bring methods into production.

Requirements

  • Currently pursuing a Bachelor's degree in Computer Science, AI, ML, Data Science, or related field.
  • Solid foundation in machine learning and deep learning.
  • Hands-on experience with LLM-based systems (agents, tool calling, retrieval, multi-agent systems) via research, internships, or projects.
  • Strong Python skills and experience with a mainstream ML or agent evaluation framework.
  • Demonstrated research or engineering ability through publications, substantial projects, internships, or open-source work.

Preferred Qualifications

  • Publications at top-tier ML/NLP venues (NeurIPS, ICML, ICLR, ACL), especially in agent learning or evaluation.
  • Experience with metric design, model-based judging, annotation, or statistical analysis.
  • Familiarity with LLM post-training, reasoning/planning, continual learning, or large-scale log and trace analysis.

Required Skills

PythonPyTorchLangChainHugging Face

Ready to apply for Agent Evaluation & Evolution Machine Learning Engineer Intern (AML-Ark-US) - 2027 Summer?

See how you stack up at ByteDance before you apply, or go straight to the application.

Apply Anyway