LlamaIndex Developer — Code Review & Evaluation

About OpenTrain

OpenTrain is the #1 platform for building careers in AI training and data labeling. We connect experienced contributors with hands-on projects that shape how modern AI systems behave.

Creating an OpenTrain account is free. We hire contractors to perform evaluation, labeling, and review work that directly improves models and tooling across the AI ecosystem.

Why AI Training Work Matters

AI training (data labeling / human feedback) is the human side of building AI: people prepare, correct, and evaluate examples that models learn from. This work is remote, flexible, and lets you contribute directly to model quality and real-world performance.

As a contributor you’ll often work part-time on focused technical tasks—ideal for specialists who want to influence model behavior without full-time engineering commitments.

The Role

We are seeking an experienced LlamaIndex developer to review and evaluate AI-generated prompts, answers, and code related to retrieval-augmented generation (RAG) and document indexing. This is a remote, contract, part-time role (less than 20 hours/week).

Compensation is $25 USD per hour. You will analyze AI outputs, label and categorize responses, identify errors and inefficiencies, and provide structured, actionable feedback to improve LlamaIndex usage and integrations.

Key Responsibilities

• Review AI-generated prompts, responses, and code snippets that use LlamaIndex; assess accuracy, relevance, and adherence to best practices.
• Label and categorize outputs (computer programming / coding) and produce structured feedback that explains issues and suggests precise fixes.
• Identify logical errors, incorrect retrievals, indexing problems, and suboptimal embedding or vector store usage.
• Evaluate integration patterns with LLMs and RAG workflows, including LangChain or standalone LLM setups.
• Conduct AI-driven technical interviews to assess incoming candidates’ LlamaIndex skills per the supplied interview guidelines.

Requirements

• Minimum 5+ years hands-on experience working with LlamaIndex, RAG systems, document indexing, and structured data retrieval for LLMs.
• Deep familiarity with vector databases and stores (examples: FAISS, Pinecone, ChromaDB), embedding models, and query optimization techniques.
• Experience integrating LlamaIndex with LLMs and familiarity with best practices for document chunking, retrieval routing, and index construction.
• Strong English writing and communication skills — you will produce detailed, structured feedback and conduct interviews in English.
• Available to work up to 20 hours/week; this is a contractor, part-time role. Worldwide applicants welcome.
• Willingness to perform code review tasks and to label code-focused outputs (data type: computer code / programming).

Interview & Evaluation Tasks (Guidelines)

You will run focused interviews and technical checks to determine candidate suitability. Follow these task areas when evaluating others and when performing review work.

• Experience assessment: verify 5+ years of LlamaIndex experience; ask for real-world projects using RAG, indexing, and structured retrieval.
• Technical knowledge check: present a short LlamaIndex code snippet with a deliberate issue; have the candidate identify and fix it.
• System design & optimization: ask how they'd structure, chunk, index, and route queries for efficient retrieval and LLM performance.
• AI-evaluation skills: evaluate and improve AI-generated responses — point out errors, inefficiencies, and missing best practices in explanations and code.
• Communication check: require clear, simple English explanations of complex features (e.g., hierarchical retrieval, embedding-based search).
• Final confirmation: confirm prior experience reviewing AI-generated code and willingness to perform structured labeling tasks; reject vague or purely theoretical answers.

How It Works & Compensation

This role is hired and managed by OpenTrain. You’ll be paid $25 USD per hour as a contractor for part-time work (less than 20 hrs/week). Labeling tools and detailed task instructions will be provided when you onboard.

To apply, create an OpenTrain account (free), complete your profile, and submit your application. We’ll assess fit based on your LlamaIndex background and interview performance.

• Employment type: Contractor, Part-time.
• Data type you’ll work with: Computer code / programming; label type: COMPUTER_PROGRAMMING_CODING.
• Worldwide applicants welcome; specific tooling will be provided during onboarding.