Coding & QC Expert

About the role

Verita AI is seeking Coding & Rubric QC Experts for a paid pilot evaluating the quality of AI benchmark tasks. Each task includes a question, a proposed gold answer and a rubric of claims describing what a correct answer should contain.

Critically audit existing work rather than create new content. Assess whether rubric claims hold up and whether the reference answer is actually correct.

This is evaluative, judgment-driven work suited to reviewers who enjoy code review, quality assurance and auditing other people's work. Strong performers may be considered for continued or expanded work after the approximately one-week pilot.

Scope of Work

• Review every rubric claim for clarity, specificity and verifiability, identifying vague, redundant or unusable criteria.
• Mark claims as pass, fail or flagged for follow-up, with a short written reason.
• Identify requirements that do not connect to the question or proposed answer.
• Check the correctness of the proposed gold answer before treating it as ground truth.
• Give an overall accept or reject decision on each task, supported by specific findings.

What you’ll bring

• Be based in the United States or Canada under the combined role-specific North America requirement and Verita AI partner policy.
• Completed bachelor's degree in Computer Science, or an equivalent completed degree.
• At least 3 months of human-data, annotation or AI-training experience of any kind.
• At least 1 year of software engineering experience outside human-data work. Any language or stack is acceptable; no specific stack is required.

Preferred Qualifications

• Direct experience grading or auditing rubrics, benchmarks or golden-set answers against references or acceptance criteria.
• Experience critically reviewing others' work and flagging missing requirements, vague criteria or incorrect reference answers.
• A record of clear, evidence-backed feedback rather than unexplained pass/fail decisions.
• Experience as a task reviewer or QC lead on a data-annotation project.
• Experience on coding-related annotation projects.

Benefits

• Paid pilot
• Flexible scheduling
• Remote work within eligible countries

Schedule and availability

Approximately 10+ hours minimum during a paid pilot of about one week, with flexible scheduling. Average task time is expected to be around 20 minutes.

Contract & Payment Terms

• Independent contractor engagement paid per completed, accepted task, not hourly.
• The listing header advertises $30.00/hour, while the detailed compensation terms specify task-based payment. Confirm the per-task rate with Verita AI.
• Paid pilot of approximately one week, with potential for continued or expanded work for strong performers.

Where you can work

Verita AI only accepts applications from candidates based in the United States (USA), United Kingdom (UK) and Canada. This restriction applies to all current and future Verita AI opportunities. This role additionally requires North America residence. Combined with Verita AI's partner-wide country policy, only USA and Canada applicants are eligible for this opportunity; UK applicants are not eligible for this role.

This role is open to people based in: United States, Canada.

Remote does not always mean work from any country. Always check the employer’s location and working-hour requirements.