Arabic Data Annotator – Remote, $11/hr

About the role

We are seeking an Arabic‑speaking Data Annotator to support the evaluation of large language model (LLM) outputs. This short‑term remote contract focuses on personalisation quality, ensuring AI responses are relevant, grounded, and respectful of user data.

Key responsibilities

• Design multi‑turn conversational prompts that incorporate personal context.
• Evaluate AI responses for relevance, grounding, helpfulness, and correct use of personal data.
• Perform side‑by‑side (SxS) ranking of AI outputs and identify grounding errors, poor inferences, or forced personalisation.
• Write clear, structured rationales referencing specific conversation turns.
• Extract and verify model debug information and data source usage.
• Maintain strict data hygiene by deleting evaluation conversations after review.

Required profile

• Fluent Arabic reading and writing skills.
• Experience in data annotation, AI quality evaluation, content moderation, or related fields.
• Strong analytical thinking, attention to detail, and ability to assess nuanced AI responses.
• BS/BA degree or equivalent analytical experience.
• Excellent written communication and structured feedback abilities.
• Self‑motivated, able to work independently with reliable internet and a desktop/laptop.
• Full‑time availability (8 hours/day) with a 4‑hour overlap with Pacific Standard Time.

Required skills