Web Research Benchmark Task Author

OpenTrain AI is recruiting a Web Research Benchmark Task Author to develop challenging research tasks that evaluate how well advanced AI models find, interpret and verify information online.

This remote contract opportunity focuses on identifying the limitations of AI-powered research systems. Rather than simply asking models questions, you'll investigate difficult-to-find information, build carefully structured evaluation tasks and examine why AI systems sometimes produce incorrect or unsupported answers.

The work brings together investigative research, fact-checking, prompt development and AI model evaluation. It's particularly suited to people who enjoy solving complex research problems and examining how AI systems reach their conclusions.

The position is open to applicants worldwide and requires a minimum commitment of 20 hours per week.

What You'll Do

Investigate Complex Research Questions

You'll explore specialized subjects and uncover information that isn't readily available through conventional searches.

Your work will involve:

• Investigating niche topics using advanced web research techniques.

• Locating reliable evidence in academic archives, public records and specialized databases.

• Comparing sources and resolving conflicting information.

• Documenting original sources and URLs to support factual conclusions.

• Identifying research questions that expose weaknesses in AI search and reasoning.

Develop AI Evaluation Benchmarks

You'll turn your research findings into structured tasks designed to test advanced language models.

Responsibilities include:

• Writing precise prompts with clearly defined research objectives.

• Creating accurate answer keys supported by credible evidence.

• Developing questions that require multiple research steps.

• Testing benchmark tasks against advanced AI models.

• Ensuring each task is challenging, unambiguous and independently verifiable.

Analyze AI Model Failures

An important part of the position involves understanding why an AI system fails to complete a research task correctly.

You'll examine model outputs, research paths and execution logs to identify problems involving:

• Incorrect or incomplete source selection.

• Weak search strategies.

• Misinterpretation of evidence.

• Limitations in reasoning or information retrieval.

• Ambiguous benchmark instructions.

Your findings will help distinguish genuine model limitations from problems in the evaluation task itself.

Improve Research and Evaluation Quality

You'll also review benchmark tasks developed by other contributors, apply effective research techniques across different subjects and help maintain high standards for accuracy and consistency.

What OpenTrain AI Is Looking For

Candidates should have strong analytical abilities, excellent written English and practical experience investigating complex information.

Required qualifications:

• At least one year of experience in investigative research, fact-checking, AI evaluation, benchmark development, red-teaming or technical quality review.

• Intermediate to advanced web research and OSINT skills.

• Ability to locate and verify information from specialized or difficult-to-access sources.

• Strong understanding of how AI models search for and interpret information.

• Experience reviewing model reasoning, search logs or intermediate outputs.

• Ability to write clear technical instructions, research prompts and evidence-based answer keys.

• Familiarity with AI evaluation benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench or Humanity's Last Exam.

• Fluent written and spoken English.

• A bachelor's degree or higher in a relevant engineering discipline.

Additional Experience That Could Help

You may have an advantage if you've worked on:

• AI red-teaming or adversarial prompt development.

• Prompt-injection or reasoning-trap evaluation tasks.

• Academic or investigative research projects.

• Research involving obscure public records or specialized archives.

• Quality assurance or leadership within AI evaluation projects.

Working Arrangements

This is a fully remote, part-time contractor position open worldwide.

The role requires at least 20 hours of availability each week. Although classified as entry level, applicants are expected to have relevant research experience and technical familiarity with AI evaluation.

Compensation details have not been provided in the listing.

About OpenTrain AI

OpenTrain AI is a platform connecting people with opportunities in artificial intelligence training and evaluation.

AI training work involves human contributors helping develop and assess AI systems through activities such as reviewing model responses, evaluating factual accuracy, testing reasoning capabilities and identifying areas where models need improvement.

This particular opportunity focuses on web research benchmarks that assess whether AI systems can locate trustworthy information and produce evidence-based answers.

Developing Skills in AI Evaluation and Research

AI research evaluation increasingly combines traditional fact-checking with prompt design, source verification and an understanding of how language models behave.

For professionals interested in this field, developing practical experience with AI tools, structured research methods and model evaluation techniques can be valuable preparation.

Resources such as publicly available AI benchmark documentation, research papers and hands-on evaluation projects can help build those skills.

How to Apply

Interested candidates can begin their application through OpenTrain AI.

You'll need to create a free OpenTrain account, submit your resume and complete the application process on the hiring platform.

Apply through OpenTrain AI to be considered for this remote opportunity.