The psychological impacts of content moderation on content moderators: a qualitative study
Article
Spence, R., Bifulco, A., Bradbury, P., Martellozzo, E. and DeMarco, J. 2023. The psychological impacts of content moderation on content moderators: a qualitative study. Cyberpsychology: Journal of Psychosocial Research on Cyberspace. 17 (4). https://doi.org/10.5817/cp2023-4-8
Type | Article |
---|---|
Title | The psychological impacts of content moderation on content moderators: a qualitative study |
Authors | Spence, R., Bifulco, A., Bradbury, P., Martellozzo, E. and DeMarco, J. |
Abstract | Content moderation generally refers to the practice of monitoring user generated content and applying a set of rules defining what is acceptable. As a result, Content Moderators (CMs) are responsible for reviewing and removing harmful online material, which has the potential to cause psychological harm. However, little is known about the perceived impacts of the job. Semi-structured interviews were conducted with 11 commercial CMs who were exposed to child sexual abuse material (CSAM) as part of their job. Interviews were analysed using framework analysis to investigate any psychological impacts. Results demonstrated that CMs manifested with a range of symptoms consistent with experiencing repeated trauma. This included experiencing intrusive thoughts of CSAM, triggered by situations with similar contexts to those encountered at work, avoidance of children and negative cognitive and emotional effects such as cynicism, anxiety and detachment. These symptoms fit into a framework of post traumatic and secondary traumatic stress and suggest CMs wellbeing and exposure to stressful stimuli may be comparable to professionals working in the emergency services or caring professions, such as social workers. These personnel are at heightened risk of various problems including compassion fatigue, vicarious trauma and burnout. Companies which employ moderators should learn from these comparable professions and provide psychoeducation and trauma-informed care to CMs which have been demonstrated to help mitigate these challenges. |
Keywords | child sexual abuse material; content moderation; trauma; post-traumatic stress disorder; work stress |
Sustainable Development Goals | 3 Good health and well-being |
Middlesex University Theme | Health & Wellbeing |
Research Group | Centre for Abuse and Trauma Studies (CATS) |
Publisher | Masaryk University |
Journal | Cyberpsychology: Journal of Psychosocial Research on Cyberspace |
ISSN | |
Electronic | 1802-7962 |
Publication dates | |
Online | 18 Sep 2023 |
18 Sep 2023 | |
Publication process dates | |
Deposited | 14 Sep 2023 |
Submitted | 27 Nov 2022 |
Accepted | 07 Aug 2023 |
Output status | Published |
Publisher's version | License |
Copyright Statement | Copyright © 2023 Ruth Spence, Antonia Bifulco, Paula Bradbury, Elena Martellozzo, Jeffrey DeMarco |
Digital Object Identifier (DOI) | https://doi.org/10.5817/cp2023-4-8 |
Web of Science identifier | WOS:001097407300006 |
Language | English |
https://repository.mdx.ac.uk/item/9y745
Download files
1680
total views621
total downloads122
views this month6
downloads this month
Export as
Related outputs
Content moderator mental health and associations with coping styles: replication and extension of previous studies
Spence, R. and DeMarco, J. 2025. Content moderator mental health and associations with coping styles: replication and extension of previous studies. https://doi.org/10.20944/preprints202502.0160.v1Exploring the use of Virtual Reality to enhance rapid decompression from occupational stress during short wellness breaks for content moderators
Spence, R., Campbell, N., Woofenden, T. and DeMarco, J. 2025. Exploring the use of Virtual Reality to enhance rapid decompression from occupational stress during short wellness breaks for content moderators. https://doi.org/10.21203/rs.3.rs-5684444/v1'Understanding domestic violence and abuse: victim, suspect and crime predictors of police outcomes'
Barbin, A., Crivatu, I., Davies, K., Horvath, M.A.H. and Spence, R. 2025. 'Understanding domestic violence and abuse: victim, suspect and crime predictors of police outcomes'. Journal of Family Violence. https://doi.org/10.1007/s10896-024-00802-zIssues with using police data to investigate offending: a research perspective
Spence, R. and Crivatu, I 2025. Issues with using police data to investigate offending: a research perspective. Police Journal: Theory, Practice and Principles. https://doi.org/10.1177/0032258x251313944How does news coverage of suicide affect suicidal behaviour at a high-frequency location? A 7-year time series analysis
Marzano, L., Spence, R., Marsh, I, Barbin, A and Kruger, I 2024. How does news coverage of suicide affect suicidal behaviour at a high-frequency location? A 7-year time series analysis. BMJ Public Health. 3 (1). https://doi.org/10.1136/bmjph-2024-002136‘I had no choice’: adult neutralisation of online sexual engagement with children
Bradbury, P., Bleakley, P. and Martellozzo, E. 2024. ‘I had no choice’: adult neutralisation of online sexual engagement with children. Journal of Police and Criminal Psychology. https://doi.org/10.1007/s11896-024-09645-2Content moderator coping strategies: associations with psychological distress, secondary trauma and wellbeing
Spence, R., Martellozzo, E. and DeMarco, J. 2024. Content moderator coping strategies: associations with psychological distress, secondary trauma and wellbeing. Journal of Media Psychology: Theories, Methods, and Applications. https://doi.org/10.1027/1864-1105/a000454Limerence, hidden obsession, fixation, and rumination: a scoping review of human behaviour
Bradbury, P., Short, E. and Bleakley, P. 2024. Limerence, hidden obsession, fixation, and rumination: a scoping review of human behaviour. Journal of Police and Criminal Psychology. https://doi.org/10.1007/s11896-024-09674-xContent moderator mental health, secondary trauma, and well-being: a cross-sectional study
Spence, R., Bifulco, A., Bradbury, P., Martellozzo, E. and DeMarco, J. 2024. Content moderator mental health, secondary trauma, and well-being: a cross-sectional study. Cyberpsychology, Behavior, and Social Networking. 27 (2), pp. 149-155. https://doi.org/10.1089/cyber.2023.0298Supporting digital key workers: Addressing the challenges faced by content moderators during and after the COVID-19 pandemic
Martellozzo, E., Bleakley, P., Bradbury, P., Spence, R. and DeMarco, J. 2024. Supporting digital key workers: Addressing the challenges faced by content moderators during and after the COVID-19 pandemic. International Journal of Technology, Policy and Management. 24 (2), pp. 212-228. https://doi.org/10.1504/IJTPM.2024.137818Identity awoken in second-generation British Poles in the UK—personal journeys
Bifulco, A. and Smojkis, M. 2023. Identity awoken in second-generation British Poles in the UK—personal journeys. Genealogy. 7 (3). https://doi.org/10.3390/genealogy7030059Moderating online child sexual abuse material (CSAM): Does self-regulation work, or is greater state regulation needed?
Bleakley, P., Martellozzo, E., Spence, R. and DeMarco, J. 2023. Moderating online child sexual abuse material (CSAM): Does self-regulation work, or is greater state regulation needed? European Journal of Criminology. 21 (2), pp. 231-250. https://doi.org/10.1177/14773708231181361Content moderators’ strategies for coping with the stress of moderating content online
Spence, R., Harrison, A., Bradbury, P., Bleakley, P., Martellozzo, E. and DeMarco, J. 2023. Content moderators’ strategies for coping with the stress of moderating content online. Journal of Online Trust and Safety. 1 (5), pp. 1-18. https://doi.org/10.54501/jots.v1i5.91The right tool for the job: evaluating police experiences of a pilot tool for responding to stalking
Bleakley, P., Frost, S., Bradbury, P., Short, E. and Martellozzo, E. 2023. The right tool for the job: evaluating police experiences of a pilot tool for responding to stalking. Policing: a Journal of Policy and Practice. 17. https://doi.org/10.1093/police/paac022Understanding sexual violence and factors related to police outcomes
Davies, K., Spence, R., Cummings, E., Cross, M. and Horvath, M. 2022. Understanding sexual violence and factors related to police outcomes. Frontiers in Psychology. 13, pp. 1-13. https://doi.org/10.3389/fpsyg.2022.977318Police responses to cyberstalking during the Covid-19 pandemic in the UK
Martellozzo, E., Bleakley, P., Bradbury, P., Frost, S. and Short, E. 2022. Police responses to cyberstalking during the Covid-19 pandemic in the UK. Police Journal: Theory, Practice and Principles. https://doi.org/10.1177/0032258X221113452Impacts of the COVID-19 pandemic on reporting of rape, serious sexual offences, and domestic abuse in one English police force
Spence, R., Horvath, M., Dalton, C. and Davies, K. 2022. Impacts of the COVID-19 pandemic on reporting of rape, serious sexual offences, and domestic abuse in one English police force. Journal of Gender-Based Violence. 6 (2), pp. 242-260. https://doi.org/10.1332/239868021X16425806897258Criminal court sentencing: the case for specialist ‘young adult’ courts
Ward, J. and Spence, R. 2022. Criminal court sentencing: the case for specialist ‘young adult’ courts. The British Journal of Criminology: An International Review of Crime and Society. 63 (4), pp. 1041-1057. https://doi.org/10.1093/bjc/azac076Frontline response: exploring the impact of COVID‐19 on stalking behaviours
Short, E., Bradbury, P., Martellozzo, E., Frost, S. and Bleakley, P. 2022. Frontline response: exploring the impact of COVID‐19 on stalking behaviours. Journal of Police and Criminal Psychology. 37 (3), pp. 540-548. https://doi.org/10.1007/s11896-022-09514-wThe moderation effect of secure attachment on the relationship between positive events and wellbeing
Spence, R., Kagan, L., Nunn, S., Bailey-Rodriguez, D., Fisher, H., Hosang, G. and Bifulco, A. 2022. The moderation effect of secure attachment on the relationship between positive events and wellbeing. PsyCh Journal. 11 (4), pp. 541-549. https://doi.org/10.1002/pchj.546Life events, depression and supportive relationships affect academic achievement in university students
Spence, R., Kagan, L., Nunn, S., Bailey-Rodriguez, D., Fisher, H., Hosang, G. and Bifulco, A. 2022. Life events, depression and supportive relationships affect academic achievement in university students. Journal of American College Health. 70 (7), pp. 1931-1935. https://doi.org/10.1080/07448481.2020.1841776“Lucky Boy!”; Public perceptions of child sexual offending committed by women
Bradbury, P. and Martellozzo, E. 2021. “Lucky Boy!”; Public perceptions of child sexual offending committed by women. Journal of Victimology and Victim Justice. 4 (2), pp. 160-178. https://doi.org/10.1177/25166069211060091