ByteDance logo for Agent Evaluation & Evolution Machine Learning Engineer Graduate (AML-Ark-US) - 2027 Start job on DevsUnite

Agent Evaluation & Evolution Machine Learning Engineer Graduate (AML-Ark-US) - 2027 Start

ByteDance

Actively HiringFull-time
Apply
Location

Seattle, WA

Experience

0-1 years

Role

Agent Evaluation & Evolution Machine Learning Engineer Graduate (AML-Ark-US) - 2027 Start

Salary

Competitive

Share this job

Know someone who'd be a great fit?

Job Description

Role Overview

The Applied Machine Learning Ark team builds full‑stack LLM service platforms (MaaS) for US and global markets, developing agents, multimodal models, training, fine‑tuning, inference, alignment, and large‑scale log analytics pipelines.

Responsibilities

  • Design evaluation systems for LLM‑based agents, measuring task success, tool use, reasoning quality, and reliability.
  • Build benchmarks and automated judging pipelines using rule‑based checks, model‑based judging, and human review.
  • Analyze agent execution traces and user feedback to uncover failure patterns and drive system improvements.
  • Support the closed‑loop from experience to capability, collaborating with research, platform, and product teams to productionize methods.

Requirements

  • Currently completing or recently completed a Bachelor’s or Master’s degree in Computer Science, AI, ML, Data Science, or related field.
  • Solid foundation in machine learning and deep learning.
  • Hands‑on experience with LLM‑based systems (agents, tool calling, retrieval, multi‑agent) via research, internships, or projects.
  • Strong Python programming skills and experience with a mainstream ML or agent evaluation framework.
  • Demonstrated research or engineering ability through publications, major projects, internships, or open‑source contributions.
  • Preferred: Publications in top ML/NLP venues (NeurIPS, ICML, ICLR, ACL) on agent learning, self‑evolving systems, or evaluation.
  • Preferred: Experience designing metrics, model‑based judging, annotation, or statistical analysis for evaluation.
  • Preferred: Familiarity with LLM post‑training, reasoning, planning, or continual learning methods.
  • Preferred: Experience with feedback‑driven optimization loops or large‑scale log/trace analysis.

Required Skills

PythonPyTorchHugging Face TransformersELK Stack

Ready to apply for Agent Evaluation & Evolution Machine Learning Engineer Graduate (AML-Ark-US) - 2027 Start?

See how you stack up at ByteDance before you apply, or go straight to the application.

Apply Anyway