Python Code Expert Evaluator for AI Training

Period- Project Bases

Location- 100% remote

We are looking for sub-contractors/underkonsult here who can bill their hours.

Project Description

This project involves reviewing AI-generated responses, evaluating technical accuracy and best practices, and providing expert feedback that trains AI systems to write better code. You´ll work with various programming languages and coding challenges, ensuring AI outputs are functional, efficient, and follow industry standards.

Hiring Profile

• BS, MS, or PhD with significant focus on Computer Science (Required). • Expert in Python, with a strong grasp of algorithms, data structures, and debugging practices. • Expert-level English proficiency with excellent writing skills. • Expert at fact-checking information using public sources across multiple domains. • Significant experience using large language models (LLMs). • Strong analytical and structured thinking skills. • Ability to explain complex programming concepts clearly and accurately. • Prior experience with RLHF annotation (preferred). • Prior Software Engineering work experience (strongly preferred). • Expertise in additional programming languages (nice to have): C#, Java, SQL, C++, TypeScript, PHP, C, Go, Bash, PowerShell, Rust, R.

Application & Onboarding Process

• Qualification exam (1–2 hours): evaluates your role-specific expertise. Compensation provided upon passing. • Project exam (1–2 hours): evaluates your adherence to project guidelines and ability to produce Python-specific deliverables. Compensation provided upon passing.

Commitment

• We expect a minimum of 17-20 hours per week, however considering the project scope and timelines the preferred and recommended commitment would be 8 hours/day.