SHIP Medicare Counselor AI Evaluator

You will help improve AI systems that answer Medicare questions. Your work will use SHIP counseling expertise to judge whether questions sound realistic and whether generated answers are accurate, complete, useful, and current.

• Review sample Medicare questions and identify questions that do not reflect what real beneficiaries ask.
• Create rubrics for initial enrollment, special enrollment periods, and Part B late penalties.
• Write guidance for comparing Medigap, Medicare Advantage, and Part D plans, including key differences and appropriate use cases.
• Evaluate AI-generated answers against your rubrics and flag errors, missing information, and outdated guidance.
• Provide clear written feedback asynchronously, prioritizing accuracy and usefulness for people from diverse backgrounds.

What it pays and takes

This is a remote contractor project for contributors based in the United States. The project requires consistent participation for a short-term engagement.

• Pay: $10 to $150 per hour.
• Schedule: 20 to 30 hours per week.
• Location: Open to candidates in the United States.
• Language: Fluent in English.
• Required: Active or recent SHIP certification, or former SSA Medicare enrollment experience.
• Required: At least one year advising on initial enrollment, special enrollment periods, Part B late penalties, Medigap versus Medicare Advantage, and Part D.
• Required: Clear, precise written communication.
• No prior AI experience is required; Medicare counseling expertise is the key qualification.
• Helpful: Experience creating Medicare guides, training materials, or counseling rubrics.
• Helpful: Experience counseling diverse beneficiaries, including rural and non-English-speaking clients.

How it works

You apply on OpenTrain with your resume and then complete the application on the hiring site.

About AI training work

AI training is the human work behind systems that answer questions and generate text. People with subject expertise review examples, create evaluation guidance, and identify errors so AI systems can provide more accurate and useful answers.