Senior PowerShell Code Reviewer (Remote, Part-Time)

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We connect skilled contributors with projects that shape how state-of-the-art AI systems behave. Creating an OpenTrain account is free and is the quickest way to start work on real annotation and review projects across the industry.

About AI training and this role

AI training (also called data labeling or human feedback) is the human work that teaches models how to behave. Contributors review, correct, and score examples so models learn from high-quality, secure outputs. This role focuses on validating AI-generated PowerShell scripts and ensuring every review aligns with a strict quality rubric.

The role — what you'll do

You will audit annotator reviews of AI-generated PowerShell scripts and validate each submission in an isolated, sandboxed environment. Your core task is to confirm the script satisfies the prompt, runs correctly, and follows secure scripting and automation best practices.

When a submission is incorrect or mis-rated, you will correct the rating, document the issue, and provide clear, actionable feedback so annotators improve and future model training data is high quality.

• Execute and test AI-generated PowerShell scripts in sandboxed environments to confirm functional compliance.
• Verify prompt compliance: confirm scripts implement the requested behavior and meet acceptance criteria.
• Evaluate scripts for idempotence, robust error handling (try/catch/finally), and parameter validation ([Validate*]).
• Identify security risks such as credential leaks, injection vectors, unescaped inputs, and unnecessary privilege escalation.
• Use rubric-based scoring and checklists to align reviews with QA standards; correct mis-ratings and explain decisions.
• Write concise, constructive feedback and mentoring notes for annotators to improve future submissions.

Requirements & ideal qualifications

This position is aimed at senior-level PowerShell reviewers. You must be comfortable executing code in sandboxed environments and providing rubric-driven QA feedback.

• 5–7+ years professional PowerShell development, automation engineering, or dedicated code-review experience.
• Expert knowledge of PowerShell 5.x–7.x (Core): advanced functions, modules, remoting, and pipeline patterns.
• Proven skills writing idempotent scripts, structured error handling, and input validation using [Validate*] attributes.
• Strong secure-coding awareness: detect/mitigate credential exposure, injection risks, escaping issues, and privilege escalation.
• Testing/debugging experience with Pester, PSScriptAnalyzer, and common debugging tools.
• Familiarity with Windows and cross-platform automation, Azure/AWS modules, and CI/CD pipeline integration.
• Experience with rubric-based scoring, checklist reviews, and ticketing or annotation tools (e.g., Jira, Asana).
• Excellent written English (B2+ CEFR) with the ability to deliver clear, constructive feedback and mentor contributors.

Nice-to-have

These are not required but will help you succeed and may influence assignment to specialized projects.

• Experience evaluating outputs from LLMs, RLHF pipelines, or prior AI/ML data-labeling projects.
• Familiarity with structured QA practices for model training data and prior mentoring of annotators.

Logistics, pay, and commitment

This is a remote contractor, part-time role. Open to applicants worldwide. You will work at least 20 hours per week and follow project schedules and task flows defined by the annotation project.

Compensation is hourly at USD 24 per hour and paid on a per-hour basis as stated in the project. You will use the platform's annotation and review tools to perform validations and record findings.

• Employment type: Contractor, Part-time.
• Time requirement: 20+ hours per week.
• Pay: $24 USD per hour.
• Work location: Remote, worldwide.

Who should apply and how it works

Apply if you are an experienced PowerShell engineer who enjoys hands-on code validation, clear technical writing, and mentoring others. Ideal candidates are meticulous, security-minded, and comfortable reproducing and debugging scripts in a sandbox.

To get started, create a free OpenTrain account, complete any qualification checks or skills tests the project requires, and you may be assigned review tasks. You will work through the platform's review interface and follow the project's rubric for scoring and feedback.

• OpenTrain is free to join and helps you find long-term and short-term annotation and review projects.
• You will be expected to pass a technical qualification or sample review demonstrating secure PowerShell practices and sandbox testing.