Skip to content
OpenTrain AIFor AI Companies

Java LLM Code Evaluation Engineer

Apply now

Job details

Pay

$25 USD/label

Engagement

Contract, part-time

Location

Remote

23 countries

Posted

Nov 10, 2026

OpenTrain on Trustpilot

The work

About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role
OpenTrain is recruiting for a Java LLM Code Evaluation Engineer to support AI training through hands-on analysis of public open-source repositories. You will examine software issues, configure and run codebases, assess test quality, and help evaluate how well language models handle realistic bug-fixing scenarios. The work combines practical software engineering with structured evaluation of model performance.

What you will do
Analyze and triage issues across open-source Java libraries. Configure repositories, including Docker-based development environments and basic software pipelines. Evaluate unit-test coverage and overall code quality. Modify and run projects locally to assess LLM performance in bug-fixing tasks. Help identify repositories and issues that present meaningful challenges for language models. Collaborate with researchers and, when needed, guide junior engineers working on related projects.

Required skills
You should have at least three years of overall software engineering experience and strong hands-on experience with Java. You should also be proficient with Git, Docker, and basic software pipeline setup, and be able to understand and navigate complex codebases. Comfort running, modifying, and testing real-world projects locally is important. Experience contributing to or evaluating open-source projects is helpful. Previous participation in LLM research or evaluation projects and experience building or testing developer tools or automation agents are useful additional qualifications.

Schedule, location, and pay
This is a fully remote contractor assignment for freelancers in LATAM. The commitment is at least 40 hours per week, including four hours of overlap with PST, for a one-month contract. Payment is $25 per approved task.

Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.

Requirements

  • Languages: English

Open to applicants in

  • Argentina
  • Belize
  • Bolivia
  • Brazil
  • Chile
  • Colombia
  • Costa Rica
  • Dominican Republic
  • Ecuador
  • El Salvador
  • Guatemala
  • Guyana
  • Haiti
  • Honduras
  • Mexico
  • Nicaragua
  • Panama
  • Paraguay
  • Peru
  • Puerto Rico
  • Suriname
  • Uruguay
  • Venezuela

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmwb4s6qu00q507jpni7pl58u

Keep exploring

Similar open roles

View all AI training jobs

Java LLM Code Evaluation Engineer

About OpenTrain OpenTrain is the unified marketplace where AI training freelancers find and consolidate opportunities across platforms, build an AI training portfolio, and turn specialized knowledge into a durable freel…

Java
Code
Remote · Argentina, Bangladesh, Belize +28 more
English
Part-time · Flexible
Per task · $25/label

Posted Nov 3, 2026

C# Software Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

C#
Code
Remote · India, Pakistan, Nigeria +6 more
English
Part-time · Flexible

Posted Jul 20, 2026

Senior LLM Code Evaluation Engineer

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Go
Code
Remote · India, Pakistan, Nigeria +6 more
English
Part-time · Flexible

Posted Jul 17, 2026

Software Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Software Engineer, LLM Evaluation

OpenTrain is recruiting senior software engineers to evaluate AI-generated code and create training and benchmarking datasets for large language models. This contractor role combines hands-on full-stack engineering with…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026