Head of Data Labelling

Our client is a VC-backed startup. They’re developing a live AI-powered job matching platform with 50k monthly paying active users. The goal is simple: job search shouldn’t feel like work. The team operates with clear ownership, measurable outcomes and fast iteration loops. Role Overview: Head of Data Labelling owns the end-to-end data labelling function - people, process, tooling, quality and predictable delivery - ensuring high-quality datasets for our AI models. You will partner closely with Product, ML and Engineering teams to translate ambiguous problems into consistent labels that unlock model improvements and reduce regressions. What You Will Label: Job ↔ profile relevance for matching/ranking (incl. constraints). ATS/application form fields taxonomy (required/optional, standard/custom, conditional). Auto-apply/workflow failure taxonomy (fail reasons, manual vs retry). Specific requirements from Job Descriptions. Responsibilities: Act as the single point of contact for all data labelling requests from product teams. Design labelling guidelines and acceptance criteria for each dataset. Own and maintain labelling taxonomy and decision rules (versioned guidelines, schema consistency, change logs). Set up and customise data labelling tools and infrastructure (e.g. Label Studio). Build and run a QA system (sampling, double-labelling on subsets, adjudication, audits) and improve labelling quality metrics. Define and track delivery predictability (turnaround, SLAs) to reduce acceptance iterations. Maintain an edge-case library and run regular calibration (weekly reviews + guideline updates). Lead and scale the team end-to-end (hiring, onboarding, workload, time tracking, payroll). Expected Results (First 3 Months): Audit the current data labelling process and identify clear improvement areas. Make a data-driven decision on in-house labelling vs outsourcing scaling. Reduce average dataset turnaround time through process and tooling improvements. Achieve high satisfaction from product managers during dataset acceptance. Launch a golden set + recurring QA routine and demonstrate fewer acceptance reworks and higher label consistency. Ensure scaling (in-house or via a vendor) with a concrete QA plan, a calibration approach, and clear cost–quality trade-offs. Must-Have Requirements: 3+ years of hands-on experience with text-based datasets (not image or video). 3+ years of experience leading a data labelling team. Experience with projects requiring industry-specific knowledge beyond common-sense. Strong experience with labeling tools (e.g., Label Studio) and workflow setup. Proven ability to build QA in labelling (audits, adjudication, error analysis). Strong guideline-writing skills and ability to drive consistent labelling across annotators. Strong process thinking and operational discipline. Experience working with multiple stakeholders (product teams). English level B2+ (written and spoken). Nice to Have: Domain experience in HR, recruiting. Experience with NLP-style tasks (classification, extraction, ranking judgments) Basic analytics/automation skills (Sheets/SQL; simple validators/export scripts are a plus). Experience scaling teams in fast-growing startups. Experience working with outsourced data labelling teams.