ActiveJobs
Workster.jobs

English-Speaking AI Data Evaluation Specialist | Large Language Models

Workster.jobs · Lisbon, PT

Full-timeOn-sitePosted 21 November 2026
Continue to Workster.jobs's application →

Applications are made on Workster.jobs's own site. We never sit in the middle.

Job description

Workster is partnering with a leading global customer experience company to recruit English-speaking AI Data Evaluation Specialists for an innovative artificial intelligence project based in Lisbon, Portugal. If you’re passionate about AI, language technology, and data quality, this is a unique opportunity to contribute to the development of next-generation Large Language Models (LLMs). You’ll work with cutting-edge AI systems by reviewing, evaluating, and improving AI-generated responses, helping create safer, more accurate, and more reliable conversational experiences for millions of users worldwide. Your Role • Review and evaluate AI-generated responses using detailed annotation and quality guidelines • Label and annotate language model outputs with accuracy and consistency • Assess AI-generated content for quality, relevance, factual accuracy, safety, and policy compliance • Apply complex annotation guidelines across a wide variety of scenarios and content types • Identify inconsistencies, edge cases, and potential model issues, escalating when appropriate • Analyze structured text, datasets, and AI-generated content to support model improvement • Deliver high-quality work while consistently meeting productivity, quality, and accuracy targets • Maintain strict confidentiality and comply with all project standards, security requirements, and internal procedures • Contribute directly to the ongoing training and improvement of advanced AI systems through detailed evaluations and feedback • 24/7 rotating schedule (Monday to Sunday) with 2 rotational days off Your Qualifications • Native-level fluency in English • Good command of English (B2 level or above) • Strong understanding of Generative AI, Large Language Models (LLMs), and AI evaluation methodologies • Previous experience in AI data annotation, LLM evaluation, Reinforcement Learning from Human Feedback (RLHF), Supervised Fine-Tuning (SFT), Quality Assurance, Content Labeling, Trust & Safety, or similar analytical roles is highly desirable • Excellent analytical thinking and attention to detail • Ability to interpret and consistently apply complex written guidelines and policies • Strong reading comprehension and written communication skills • Comfortable working with structured text, datasets, and AI-generated content • Ability to work efficiently in a fast-paced, quality-driven environment while meeting performance targets • Professional, objective, and capable of handling confidential information with discretion The Offer • Office-based position in Lisbon, Portugal • Comprehensive onboarding and specialized training in AI evaluation processes • Opportunity to work on one of the world’s most advanced artificial intelligence projects • International and multicultural working environment • Career development opportunities within the rapidly growing AI industry • Private health insurance • Meal allowance • Ongoing learning and professional development • Modern office facilities with a collaborative team culture • Unique opportunity to shape the future of generative AI and language technologies

Other jobs you might want

Browse all jobs on ActiveJobs →