Trust & Safety

A Trust & Safety professional works to protect users and platforms from harmful content, abuse, fraud, and policy violations. The role involves reviewing escalated content, developing and enforcing platform policies, investigating bad actors, and working with product and engineering teams to build safer systems. Analysts must apply consistent judgement to ambiguous situations — often involving graphic, sensitive, or legally complex material — while balancing user experience with harm prevention. Trust & Safety teams operate at scale, using a combination of human review, automated detection, and machine learning classifiers. Depending on the organisation, the role may specialise in areas such as child safety, misinformation, violent extremism, account integrity, or financial fraud. It requires strong analytical skills, emotional resilience, and a thorough understanding of platform policies and relevant law.

An AI Quality / Testing Analyst designs and executes the evaluation processes that ensure AI systems — particularly those using large language models or machine learning models — behave reliably, safely, and as intended before and after deployment. Day-to-day work involves writing test cases that probe AI system behaviour across a wide range of inputs including edge cases and adversarial examples, running structured evaluations to measure output quality against defined criteria, documenting failure modes and unexpected behaviours, collaborating with prompt engineers and ML engineers to investigate root causes, and maintaining evaluation datasets and benchmarks. The role requires a combination of systematic QA discipline, curiosity about AI system behaviour, and enough technical literacy to understand what is being tested and why it might fail. AI Quality Testing is one of the newest specialisms in the technology sector, emerging from the realisation that conventional software testing methods — which verify deterministic outputs — are insufficient for AI systems whose outputs are probabilistic, context-dependent, and difficult to fully specify in advance. Entry-level positions typically focus on test case design, manual evaluation, and maintaining test sets, with progression toward automated evaluation frameworks, red-teaming, and evaluation methodology ownership. The role exists primarily at AI companies, technology firms with significant AI deployments, financial services organisations using ML in regulated decision-making, and consulting firms advising on AI governance.