The Hidden Toll of Content Moderation: An Essential Yet Traumatizing Role. Online content moderators are increasingly important workers on the darkest corners of the internet as demand for safe and accessible online space intensifies.
A recent VOP Investigates series reveals the often-unseen lives of people charged with keeping online users from what can be disturbing content through platforms such as TikTok, Facebook, and Instagram by moderating flagged material in real-time.
Even as AI is set to improve the moderation of harmful content, most of the heavy-lifting work will still fall to human moderators who must use nuanced judgment. Long-term exposure to distressing images and videos is known to pose serious mental health challenges for moderators.
Responses have included a union formed by ex-moderators in East Africa as well as lawsuits that underline the toll these jobs take on mental health. In a first, Meta paid out $52 million to moderators in 2020 and recognized the trauma those individuals endured. The companies also have begun to establish wellness programs, including on-site support and therapeutic resources, to look after moderators’ mental well-being.
Role of AI in Moderation: Hope and Limitations
AI moderation tools are advancing very rapidly, such as OpenAI’s ChatGPT, said to approach an accuracy of 90% in detecting harmful content. Experts opine that AI remains not well-equipped to understand nuances in subtle meanings and in contexts, possibly ending up blocking situations and infringement on freedom of speech.
Former head of trust and safety at OpenAI, Dave Willner acknowledges AI’s robustness in dealing with disturbing content but says human beings will always be required for nuanced moderation.
Industry Statements on Welfare of Moderator
There have been efforts from the major tech players. TikTok comes in, putting money together for clinician support and implementing features that blur graphic material for moderators to review them. Meta implemented its features to blur graphic material for moderators.
Lastly, OpenAI reinforced the commitment it makes to its human workforce in ensuring that they are well-equipped for the task ahead.