Appen is a leader in AI enablement for critical tasks such as model improvement, supervision, and evaluation. To do this we leverage our global crowd of over one million skilled contractors, speaking over 180 languages and dialects, representing 130 countries. In addition, we utilize the industry's most advanced AI-assisted data annotation platform to collect and label various types of data like images, text, speech, audio, and video.

Our data is crucial for building and continuously improving the world's most innovative artificial intelligence systems and Appen is already trusted by the world's largest technology companies. Now with the explosion of interest in generative AI, Appen is helping leaders in automotive, financial services, retail, healthcare, and governments the confidence to deploy world-class AI products.

At Appen, we are purpose driven. Our fundamental role in AI is to ensure all models are helpful, honest, and harmless, so we firmly believe in unlocking the power of AI to build a better world. We have a learn-it-all culture that values perspective, growth, and innovation. We are customer-obsessed, action-oriented, and celebrate winning together.

At Appen, we are committed to creating an inclusive and diverse workplace. We are an equal opportunity employer that does not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Position Overview:

We are seeking a dedicated and detail-oriented LLM Enablement and Quality Manager to oversee and improve the quality and engagement of contributors across various AI-related projects. The ideal candidate will have a strong background in managing contributor quality through training, feedback loops, and audit processes. This role will be pivotal in ensuring that client expectations are met by driving improvements in prompt generation, review processes, and overall project quality metrics.

Key Responsibilities:

* Training & Webinars:Establish and lead frequent training webinars designed to maintain contributor quality, update prompt approaches, and enhance competencies. Training sessions will be tailored to specific projects or layers within projects and will align with updated client feedback and requirements.

* Feedback Collection; Problem-Solving: Gather and process feedback from contributors related to tasking challenges, such as instruction clarity, expectations, and technical terminology. Provide targeted feedback to project management and clients, identifying viable solutions that maintain engagement without compromising quality.

* 1-on-1 Contributor Support:Host office hours and 1-on-1 training sessions, including “last chance” sessions for contributors flagged for quality issues. Ensure consistency and performance improvements across the contributor base.

* Documentation & Rubric Updates:Maintain and regularly update project instruction documents and rubrics for contributors, reviewers, and QA staff. Ensure these materials support clear and consistent execution across all layers of the project.

* Project Metrics Reporting:Analyze and condense project-wide quality metrics, such as prompt rejection rates and response rating accuracy, into actionable insights. Use these insights to create story points for clients and project management to visualize trends and identify training focus areas.

* Quality Audits & Spot Checks:Conduct random spot audits on contributors to measure prompt quality, detect potential spammers, and flag contributors for time theft or noncompliance with project requirements.

Additional Opportunities:Take on responsibilities that emphasize technical requirements, specialized terminology, and evolving project needs. Contribute to broader project development and the definition of KPIs to refine contributor quality management processes.

Qualifications:

* Proven experience managing contributor quality and engagement in AI or data-focused projects.

* Strong ability to create and lead effective training sessions, both group-based and individual.

* Experience in quality auditing, feedback processing, and reporting key metrics.

* Excellent communication skills, with the ability to work across diverse teams.

* Ability to adapt and manage multiple projects or project layers concurrently.

* Familiarity with prompt generation, rating systems, and AI model training is a plus. 

$16 - $26 an hour