Bash/PowerShell Script Reviewer For AI Evaluation

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We connect skilled contributors with projects that shape how state-of-the-art AI systems behave, offering flexible, remote work where your technical expertise directly improves model accuracy and safety.

You’ll join a community focused on real-world AI training: annotating, evaluating, and producing high-quality examples and explanations that help models learn correct, safe, and maintainable behaviors.

About AI training work

AI training (data labeling / RLHF) is the human side of building intelligent systems — people review model outputs, rate alternatives, and create ground-truth solutions. This role focuses on automation scripting: evaluating model-generated Bash/PowerShell, identifying unsafe or incorrect patterns, and producing clear, reproducible model answers.

The role

You will review AI-generated scripting answers and troubleshooting workflows for correctness, safety, portability, and performance, then write high-quality model solutions and explanations that demonstrate best practices for automation-focused shell scripting.

This is a part-time contractor role (less than 20 hours/week), US-based only, paid per hour at $40 USD. Work remotely and set flexible hours that suit your schedule while meeting project deadlines.

• Employment type: Contractor, Part-time
• Time commitment: Less than 20 hours/week
• Pay: $40 USD per hour
• Location: Must be currently located in the United States

What you’ll do

Day-to-day tasks combine technical review, comparative evaluation, and documentation: you’ll inspect AI outputs, detect bugs or unsafe patterns, fact-check technical claims, rank multiple responses, and author canonical solutions and step-by-step explanations.

• Evaluate AI-generated Bash and PowerShell scripts for correctness, readability, performance, safety, and portability.
• Identify security risks (e.g., injection vectors, unsafe deletes), edge cases, and environment-specific failures.
• Compare and rate multiple model responses; provide clear rationale and corrective edits.
• Write high-quality model solutions and explanatory documentation that teach best practices for automation.
• Use LLMs as part of your workflow to prompt, validate, and refine code outputs.

Requirements

You must meet every listed requirement below; we will verify skills during technical screening.

• Currently located in the United States (US-based applicants only).
• 2–3+ years professional experience with Bash and/or PowerShell scripting in DevOps/IT automation/infrastructure roles.
• Bachelor’s degree in Computer Science or a closely related technical field.
• Proven experience debugging and maintaining automation scripts in production.
• Ability to assess scripts for security/safety, portability, and edge cases.
• Experience with Unix tooling (awk, sed, grep) and shell best practices and/or PowerShell ecosystem tooling (PowerShell Core, DSC, Azure PowerShell modules).
• Familiarity with CI/CD pipelines and automation workflows, including Git-based PR review and pipeline scripting.
• Regular, practical use of LLMs for coding, troubleshooting, or reviewing outputs.
• Strong written English (C1+): able to produce step-by-step explanations and clear technical documentation.

Who should apply

Apply if you enjoy reading and improving code, explaining technical tradeoffs in clear English, and shaping how automation knowledge is taught to AI. This role suits experienced scripting engineers who use LLMs as part of their daily toolkit and want flexible, impactful work on model quality.

How it works

After you apply, selected contributors complete a short technical evaluation to verify scripting ability and written explanations. Approved contributors receive tasks through OpenTrain’s platform and are paid as contractors at the stated hourly rate.

• Tasks are delivered via OpenTrain’s project interface ().
• Labeling types include code review, evaluation/rating, prompt-response writing (SFT), and RLHF-style feedback.
• You will supply written rationale for ratings and produce model solutions that become training examples for AI systems.