Post a job

Senior Software Engineer, Model Evaluation

On-site San Francisco, United States full time USD 180k to 250k/yr

About the Role

Join an early-stage AI team building agents for manufacturing operations. You will create benchmarks and train models against real procurement and supply-chain workflows, helping ensure agents can handle customer-facing tasks reliably.

What You'll Do

  • Design and build evaluation suites for supplier sourcing, qualification, compliance documents, purchase orders, freight, and order tracking.

  • Work with procurement and manufacturing experts to scope tasks, write reference answers and rubrics, and build graders.

  • Train and fine-tune models, using evaluation results to guide experiments.

  • Identify agent failures, improve production systems, and measure whether changes work.

  • Help establish the team's research and evaluation practices.

What We're Looking For

  • At least 3 years of experience, operating at a senior or staff level.

  • Hands-on experience designing benchmarks and evaluations, including authoring tasks, reference solutions or rubrics, and quality checks.

  • Practical experience training and fine-tuning models, not only using models through APIs.

  • Ability to work closely with non-ML domain experts and translate their knowledge into measurable tasks.

  • Supply-chain, procurement, manufacturing, or early-stage startup experience is helpful.

Compensation & Benefits

Base salary is $180,000 to $250,000 annually, plus equity. Visa sponsorship details were not provided.

Location

Hybrid in San Francisco, California, United States.

Source: the employer's careers page. Last checked 2026-11-19. Posted 2026-11-19.

Applications happen on Clera's own site. View role and apply

Get jobs like this by e-mailAt most one e-mail a day, only when something new matches. Unsubscribe in one click.

More from Clera

All jobs at Clera

Similar jobs in United States

All software development jobs in United States

Hiring for a role like this? See how employers fill a role fast.