Applied Philosophy Benchmark Specialist | $50-$63/hr Remote
Overview
A remote role focused on creating and vetting high quality philosophy assessment content for an AI research initiative. You will craft and critique multiple choice questions across core philosophy domains, while helping set gold standard benchmarks for advancing AI capabilities. The work blends deep philosophical expertise with rigorous evaluation, and offers flexible, asynchronous engagement.
What You'll Do6
- 1Author original philosophy questions that probe deep conceptual understanding and avoid superficial recall
- 2Ensure each question is clear, self-contained, and includes all necessary details in the prompt
- 3Rate question difficulty as Medium, Hard, or Expert and assign one correct answer plus nine carefully chosen distractors
- 4Provide step by step Chain of Thought solutions with concise reasoning in markdown format
- 5Suggest 1–5 academic references per question from reputable, citable sources
- 6Review pre written questions for accuracy, clarity, and rigor in verification tasks, noting edits and justifications
Requirements5
- 1PhD or doctoral candidate in Philosophy or a closely related field
- 2Strong command of philosophical argumentation, formal logic, and canonical texts across traditions
- 3Experience with academic writing or teaching in philosophy is a plus
- 4Excellent written English with ability to present complex ideas clearly
- 5Ability to work asynchronously and remotely, committing to 10+ hours per week
Who Should Apply
Ideal candidates are seasoned philosophers or doctoral students who thrive on precise analysis, enjoy building rigorous assessment materials, and can articulate complex ideas clearly for AI benchmarking.
Salary Insight
Pay range is $50.00 to $63.00 per hour, with remote, hourly engagement.
Location
Required Skills
Application Tip
Prepare a portfolio snippet showing a sample question, its correct answer, 9 plausible distractors, and a concise justification for the correct option to demonstrate your precision and stylistic clarity.
Similar open positions
Explore active roles that match your skills and interests.
Mercor
VerifiedApplied History & Political Science Benchmark Specialist | $44-$56/hr Remote
This remote role focuses on creating and validating academic assessment content for an AI research initiative. You’ll craft and review high quality multiple choice items in history and political science, help determine gold standard benchmarks, and contribute to improving AI understanding in these fields. The work blends scholarly rigor with practical QA to support advanced AI systems. Expect a flexible, asynchronous arrangement that taps deep expertise in your domains.
Mercor
VerifiedApplied Physics Benchmark Specialist | $61-$77/hr Remote
We are looking for seasoned physicists to craft and review advanced academic assessment content for an AI research project. You’ll develop and validate high quality multiple-choice questions across key physics areas, assess solution quality, and help establish benchmark materials that push AI capabilities forward. The role is remote and hourly, with flexible, asynchronous collaboration. You’ll work in two modes: authoring original questions and verifying pre-written items for accuracy and rigor.
Mercor
VerifiedApplied Engineering Benchmark Specialist | $61-$77/hr Remote
The role focuses on creating and validating rigorous academic assessment content for an AI research effort. You will author and review multiple-choice questions across core engineering topics, judge solution quality, and help build gold-standard benchmarks to push AI capabilities forward. This is a remote, hourly engagement, with tasks that blend technical evaluation and high-quality writing using a clear, structured approach.
Mercor
VerifiedApplied Psychology Benchmark Specialist | $50-$63/hr Remote
A remote role for seasoned psychologists to author and review rigorous assessment items for an AI research effort. You’ll craft original multiple-choice questions and validate existing items to help define benchmarks that advance AI understanding of psychology. The work spans areas like psychometrics, digital health, and consumer psychology, with both creation and verification responsibilities. This opportunity blends scholarly rigor with flexible, asynchronous engagement and compensation on an hourly basis.
Mercor
VerifiedApplied Computer Science Benchmark Specialist | $66-$84/hr Remote
A remote, hourly role focused on building and reviewing high quality AI research benchmark content for computer science. You’ll craft and validate rigorous multiple choice questions across core CS topics, assess answer quality, and contribute to gold-standard datasets that push AI capabilities forward. Expect a split between creating new questions and verifying existing ones, with opportunities to shape benchmarks used by researchers worldwide.
Mercor
VerifiedApplied Mathematics Benchmark Specialist | $61-$77/hr Remote
A remote role focused on creating and validating rigorous math assessment content for an AI research initiative. You’ll write and review advanced multiple choice questions across key mathematics domains, assess solution quality, and help set benchmarks that push AI capabilities forward. The work blends mathematical depth with clear, instructional problem design and documentation. This position stands out for its blend of academic rigor and practical benchmarking in AI research.