[Full Disclosure: This page contains affiliate links, if you click links and make a purchase, we will earn a commission at no additional cost to you.]
About OpenTrain AI
OpenTrain connects subject-matter experts with projects focused on AI training, model evaluation, and data quality.
For this project, the company is looking for someone with extensive knowledge of the video game industry to help test how accurately advanced AI models understand gaming, from development and design to esports, platforms, publishers, and player communities.
About the Role
As a Video Game AI Evaluation Expert, you'll use your gaming expertise to challenge and assess large language models.
You'll create sophisticated prompts, review AI-generated answers, identify errors and weak reasoning, and contribute to benchmark datasets used to measure model performance.
The work requires more than simply knowing games. You'll need to evaluate information critically, distinguish reliable answers from convincing but inaccurate ones, and clearly document why an AI response succeeds or fails.
What You'll Do
- Develop challenging prompts covering different areas of the video game industry.
- Test AI knowledge across game development, design, genres, platforms, studios, publishers, esports, and gaming communities.
- Review model responses for factual accuracy and depth.
- Assess whether AI-generated reasoning is complete, consistent, and appropriately nuanced.
- Detect hallucinations, outdated claims, contradictions, and difficult edge cases.
- Create benchmark examples that can be used to measure model performance.
- Develop adversarial scenarios designed to expose weaknesses in AI responses.
- Support evaluations with credible evidence and references.
- Write detailed feedback explaining where responses can be improved.
- Work with AI researchers to help improve model behavior and domain knowledge.
- Maintain consistent quality and documentation throughout the evaluation process.
What They're Looking For
OpenTrain is looking for candidates with at least 3 years of professional experience within the gaming ecosystem.
Relevant backgrounds can include:
- Game development
- Game design
- Quality assurance
- Esports
- Gaming journalism
- Gaming research
- Content creation
- Community management
- Related video game industry roles
You should have broad knowledge of the industry, including development practices, major platforms, genres, studios, publishers, competitive gaming, and gaming communities.
Strong analytical skills and excellent written English are also important, as you'll need to explain precisely why particular model responses are accurate, incomplete, misleading, or incorrect.
Education
A master's degree or higher is listed as a requirement, preferably within a gaming-related or otherwise relevant discipline.
Candidates should combine their academic background with practical knowledge of the video game industry.
AI Experience
Previous AI experience can strengthen your application but isn't the primary focus of the position.
Useful additional experience includes:
- Working with large language models.
- Generative AI.
- Prompt engineering.
- AI response evaluation.
- Benchmark or dataset development.
Published research, teaching experience, or professional recognition within gaming may also be valuable.
Project Details
This is an 8-week remote contractor project for candidates located in the United States.
The listing specifies a minimum commitment of 20+ hours per week, while the planned workload is approximately 40 hours per week.
You'll also need at least four hours of working-time overlap with Pacific Time.
Why This Role Is Interesting
This position offers a way to combine deep gaming knowledge with hands-on work in generative AI evaluation.
Rather than developing a game directly, you'll use your understanding of the industry to determine whether AI systems can reason accurately about it — creating difficult tests, finding weaknesses, and helping researchers build models with stronger domain understanding.
For someone already working in gaming who wants practical experience at the intersection of video games and AI, it could be an interesting route into model evaluation and AI training.
.png)
