Full-Stack QA Reviewer (Python or JavaScript)

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. Creating an OpenTrain account is free.

We connect contributors with projects that shape how modern AI systems learn. This role runs on OpenTrain and sits at the intersection of software engineering and human-in-the-loop AI training.

About AI Training Work

AI training (also called data labeling or human feedback work) is the human side of building artificial intelligence. Contributors prepare, review, and refine examples that teach models to write, reason, and code.

This role focuses on coding-focused QA: reviewing code and unit tests that are used to train and evaluate models, helping ensure datasets and rubrics are accurate, consistent, and teach correct behavior.

The Role

We are seeking experienced Full-Stack Software Engineers to join as QA Reviewers for coding tasks and unit tests. This is a fully remote, flexible-schedule contractor role working 20+ hours per week.

You will perform in-depth quality reviews of coding artifacts to verify correctness, identify ambiguities, and improve QA rubrics so training data and evaluations meet high standards.

• Employment: Contractor, Part-time
• Schedule: Remote and flexible, 20+ hours/week
• Pay: USD $30 per hour

What You'll Do

Review and evaluate code submissions and unit tests for correctness, edge cases, and adherence to best practices.

Provide clear, actionable feedback to improve code quality, test coverage, and task rubrics used for training models.

• Verify the correctness of tasks and unit tests and flag errors, redundancies, and gaps.
• Refine and optimize QA rubrics to improve consistency and reviewer efficiency.
• Collaborate with stakeholders to resolve ambiguities and scale QA processes with other reviewers.
• Document findings and recommend improvements to coding tasks and evaluation criteria.

Requirements

Candidates must have substantial professional software development experience and strong English communication skills.

You will be expected to supply exact-year experience details for both Python and JavaScript even if you will work in only one language.

• Minimum 5 years of full-stack software development experience; state exactly how many years you have.
• Proficiency in Python or JavaScript with extensive experience writing, testing, and reviewing code; specify exact years for both languages.
• Deep understanding of unit testing frameworks and software development best practices.
• Proven ability to analyze, debug, and evaluate complex codebases and technical artifacts.
• Strong attention to detail and experience giving detailed, constructive feedback on code quality.
• Excellent written and verbal English; candidates outside the USA, Canada, UK, or Australia must provide documentation of C1 or C2-level proficiency (e.g., IELTS, TOEFL).

Preferred Qualifications

These are not required but will make your application stronger and may influence assignment selection.

• Experience with LLM training for coding or related platforms (OpenTrain, Appen, OpenTrain, or similar).
• Teaching, mentoring, or code-review leadership experience in engineering environments.
• Familiarity with creating or refining QA rubrics for data-labeling projects.

Application, Interview & Test Questions

Applications should include a brief resume and clear answers to the experience questions below. During the interview you will be asked open-ended questions; provide informative, specific responses rather than yes/no answers.

At the end of the interview you must answer the two test questions below. Responses will factor into your interview score.

• Ask applicants to state exactly how many years of full-stack development experience they have and the exact years of experience with both Python and JavaScript.
• Test Question 1 — Code Review and Debugging: Review this Python function, identify at least two issues or improvements, and explain why each is necessary: def calculate_average(numbers):\n total = 0\n for num in numbers:\n total += num\n average = total / len(numbers)\n return average\n
• Instructions for Q1: Identify at least two potential issues or improvements and explain your reasoning for each suggestion.
• Test Question 2 — Annotating Code for AI Training: Add inline comments to this JavaScript function to teach an AI how it works and note assumptions/edge cases: function isPalindrome(str) {\n str = str.toLowerCase().replace(/[^a-z0-9]/g, '');\n let reversedStr = str.split('').reverse().join('');\n re
• Instructions for Q2: Provide clear, concise inline comments for each line and highlight assumptions or edge cases the function may need to handle.

How to Apply

Create a free OpenTrain account, complete your profile, and submit your application through the platform. Include your resume and clear answers to the experience questions and the two test questions above.

You will be considered for assignments as a contractor and must be available to work at least 20 hours per week. Successful applicants will be contacted with next steps.