
AI Rubric Reviewer - Fully Remote | Up to $75/hr
Mercor
Applications are made on Mercor's own site. We never sit in the middle.
Job description
The research teams behind the best-known AI models come to Mercor for data judgment they can't generate on their own. Your standards and expertise become the rubric that frontier AI is graded against. This is a standing listing for analysts and data practitioners who want AI training and evaluation work, not a specific job opening. It stays open whether or not a matching project is running this week. New projects start on short notice, so instead of posting and pulling a listing each time, we keep one open per field and draw from it when one begins. What the work looks like We match data practitioners in the network to projects like: • Writing an analysis problem from work you actually shipped, with the data, the question, and the constraint, plus the reference answer a model's attempt gets graded against • Grading model-written analysis and pipelines: whether the query answers the question that was asked, whether the join fans out, whether the metric means what the model says it means • Flagging where a model produces a technically valid result that would mislead a decision-maker, given selection effects, leakage, or a metric that does not track the outcome What we're looking for • Professional experience in analytics, data engineering, statistics, or machine learning • Clear written communication, since most of the work involves explaining your reasoning • Comfort with ambiguous tasks, and willingness to flag unclear instructions What this covers This pool spans data analysis, including: • Business intelligence • Data engineering • Statistics • Machine learning Where you trained matters less than whether you have shipped analyses someone made a decision on. How it works Apply here with your resume and confirm where you work from, then take a short AI interview, about 20 minutes total. We verify your background and add you to the data analysis pool. When a project needs your specialty, we invite matching people from the pool to that specific listing. It names the rate, the hours, and the client, and any hiring decision is made there. What to expect This listing does not accept or reject candidates, so you will not get a decision on it. We reach out when a project matches your background. Depending on what our clients are running, that can be within a week or several months. Finishing the domain expert interview and joining every network you qualify for both improve your odds of being matched. You only need to sign up once to be eligible across all of them. We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request. Contract and Payment Terms • You will be engaged as an independent contractor. • This is a fully remote role that can be completed on your own schedule. • Projects can be extended, shortened, or concluded early depending on needs and performance. • Your work at Mercor will not involve access to confidential or proprietary information from any employer, client, or institution. • Payments are weekly on Stripe or Wise based on services rendered. • Please note: We are unable to support H1-B or STEM OPT candidates at this time. About Mercor Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will work on projects that focus on training and enhancing AI systems. You will be paid competitively, collaborate with leading researchers, and help shape the next generation of AI systems in your area of expertise. Earn rewards by referring Share the referral link below and earn 20% of your referral's earnings across all their projects on Mercor. Your Earnings Cap is 4× their hourly rate on their first accepted role. Referral limits apply. See the Referral Policy for current limits. Referral limits apply. See the Referral Policy for current limits. Don't know who to refer? Find relevant LinkedIn connections here . One interview, real results AI experts share how Mercor made hiring faster, fairer, and easier — with just one interview. Explore all opportunities → Posted 2 months ago img:first-child]:pt-0 has-[[data-slot=card-footer]]:pb-0 data-[size=lg]:[--card-spacing:1.5rem] data-[size=sm]:[--card-spacing:0.75rem] [&>img:first-child]:rounded-t-xl [&>img:last-child]:rounded-b-xl overflow-visible shadow-lg shadow-gray-200/70" data-test="application-form"> .sr-only]:w-auto"> [data-slot=field]]:rounded-lg has-[>[data-slot=field]]:border has-[[data-checked]]:border-indigo-300 has-[[data-checked]]:bg-indigo-50/50 group-has-[:disabled]/field:opacity-50 group-data-[disabled=true]/field:opacity-50 [&_[data-slot=field]]:p-2.5 has-[>[data-slot=field]]:w-full has-[>[data-slot=field]]:flex-col" for=":Rdb6ulfm:">Full legal name * .sr-only]:w-auto"> [data-slot=field]]:rounded-lg has-[>[data-slot=field]]:border has-[[data-checked]]:border-indigo-300 has-[[data-checked]]:bg-indigo-50/50 group-has-[:disabled]/field:opacity-50 group-data-[disabled=true]/field:opacity-50 [&_[data-slot=field]]:p-2.5 has-[>[data-slot=field]]:w-full has-[>[data-slot=field]]:flex-col" for=":Rdb6ulfmH1:">Email * .sr-only]:w-auto"> [data-slot=field]]:rounded-lg has-[>[data-slot=field]]:border has-[[data-checked]]:border-indigo-300 has-[[data-checked]]:bg-indigo-50/50 group-has-[:disabled]/field:opacity-50 group-data-[disabled=true]/field:opacity-50 [&_[data-slot=field]]:p-2.5 has-[>[data-slot=field]]:w-full has-[>[data-slot=field]]:flex-col" for=":Rbddb6ulfm:">LinkedIn URL * [data-slot=field-content]]:items-start [&>[data-slot=field-label]]:flex-auto [&:has(>[data-slot=field-content])>[role=checkbox]]:mt-px [&:has(>[data-slot=field-content])>[role=radio]]:mt-px mt-2"> [data-slot=field]]:rounded-lg has-[>[data-slot=field]]:border has-[[data-checked]]:border-indigo-300 has-[[data-checked]]:bg-indigo-50/50 group-has-[:disabled]/field:opacity-50 group-data-[disabled=true]/field:opacity-50 [&_[data-slot=field]]:p-2.5 has-[>[data-slot=field]]:w-full has-[>[data-slot=field]]:flex-col font-normal" for="floating-no-linkedin">I don't have a LinkedIn $70-$120 / hr Part-time · Remote
Other jobs you might want
Mercor
Rubric Auditor - Fully Remote | Up to $85/hr
Mercor
Finance Business Partner - Fully Remote | Up to $100/hr
United Kingdom, Ireland, Germany, Spain, Switzerland, US
Mercor
Legal AI Specialist - Fully Remote | Up to $150/hr
Mercor
Marketing Expert - Fully Remote | Up to $85/hr
United Kingdom, Ireland, Germany, Spain, Switzerland, US
Mercor
Finance Business Partner - Fully Remote | Up to $100/hr
United Kingdom, Ireland, Germany, Spain, Switzerland, US
Mercor
AI Evaluation Rubric Auditor