Senior Java Code Reviewer for AI Evaluation

About OpenTrain

OpenTrain AI is the hiring and contracting organization for this role. OpenTrain is the leading platform for finding and building careers in AI training and data labeling, connecting skilled contributors with projects that shape how modern AI systems work.

Create a free OpenTrain account to build your profile and apply in minutes. This contractor position is available to candidates in India, the Philippines, and Vietnam.

• Contractor and part-time engagement
• 20+ hours per week
• Pay: $25 USD per hour
• English-language work
• Eligible locations: India, the Philippines, and Vietnam

About AI Training and Code Evaluation

AI training is the human side of building artificial intelligence. Specialists review examples, evaluate model responses, and provide structured feedback so AI systems can become more accurate, reliable, secure, and useful.

In this project, your Java expertise will help assess AI-generated programming responses. Your reviews will support the quality of datasets used to train next-generation AI models.

• Work on cutting-edge AI development
• Apply professional software engineering judgment to model outputs
• Help improve correctness, security, and performance in AI-generated code

The Role

OpenTrain AI is seeking a senior Java code reviewer with at least seven years of hands-on experience in modern Java, professional QA, or dedicated code-review roles. You will audit annotator evaluations of AI-generated Java code and determine whether each snippet satisfies its prompt, works correctly, follows secure-coding practices, and meets performance expectations.

The work requires clear written English and disciplined participation in a structured quality-assurance workflow. You will correct inaccurate ratings, provide targeted feedback, and ensure reviews align with the project quality rubric.

• Review AI-generated Java code responses
• Audit and correct annotator evaluations
• Compile and run code in isolated sandbox or container environments
• Validate prompt compliance, functionality, security, and performance
• Provide concise, constructive written feedback

What You'll Evaluate

You will use deep knowledge of modern Java and the JVM to assess implementation quality. Reviews may involve core APIs, standard-library usage, JVM behavior, concurrency, testing strategy, build configuration, and secure coding.

• Modern Java versions 11 through 21
• Standard library and core APIs
• JVM internals, garbage collection, JIT behavior, and profiling
• Thread safety, locks, CompletableFuture, and reactive streams
• JUnit 5, TestNG, Testcontainers, Mockito, and JaCoCo
• Maven, Gradle, Dockerized builds, and CI/CD pipelines
• GitHub or GitLab pull requests and code-review workflows
• OWASP Top 10 issues, deserialization attacks, injection, race conditions, and permission flaws

Requirements

This role is intended for an experienced Java professional who can independently validate code behavior and explain review decisions precisely. You should be comfortable working from rubrics, checklists, and structured annotation or ticketing workflows.

• 7+ years of professional Java development, QA, or dedicated code-review experience
• Deep expertise in Java 11 through 21, core APIs, JVM internals, and performance tuning
• Advanced testing and debugging experience
• Proven ability to analyze multithreaded and concurrent code
• Strong secure-coding knowledge, including OWASP risks and permission flaws
• Experience compiling and running code in sandbox or container environments
• Familiarity with rubric-based scoring, checklist-driven reviews, Jira, Asana, or annotation tools
• Excellent written English at B2 CEFR level or above
• Ability to provide concise feedback and guide peers on best practices

Preferred Experience

Experience with AI evaluation, reinforcement learning from human feedback pipelines, or AI and machine-learning data-labeling projects is welcome. It is listed as a nice-to-have rather than a core requirement.

• LLM evaluation experience
• RLHF pipeline experience
• Previous AI or machine-learning data-labeling work

How to Apply

Apply through OpenTrain AI by creating a free account and completing your profile. If selected, you will use your Java, testing, security, and review expertise to help evaluate the code examples that teach AI systems how to program more effectively.

• Create a free OpenTrain account
• Highlight your Java development, QA, testing, security, and code-review experience
• Apply in minutes
• Be prepared to demonstrate careful, rubric-based technical judgment