Moderating online child sexual abuse material (CSAM): Does self-regulation work, or is greater state regulation needed?

Article


Bleakley, P., Martellozzo, E., Spence, R. and DeMarco, J. 2023. Moderating online child sexual abuse material (CSAM): Does self-regulation work, or is greater state regulation needed? European Journal of Criminology. 21 (2), pp. 231-250. https://doi.org/10.1177/14773708231181361
TypeArticle
TitleModerating online child sexual abuse material (CSAM): Does self-regulation work, or is greater state regulation needed?
AuthorsBleakley, P., Martellozzo, E., Spence, R. and DeMarco, J.
Abstract

Social media platforms serve a role in the Internet age as crucial public forums connecting users around the world through a decentralised cyberspace. These platforms host high volumes of content and, as such, the role of content moderators (CMs) employed to safeguard users against harmful content like child sexual abuse material and gore is critical — however, despite how essential CMs are to the social media landscape, their work as “first responders” is complicated by legal and systemic debates over whether policing cyberspace should be left to the self-regulation of tech companies, or if greater state-regulation is required. In this scoping review, major debates in this area are identified and evaluated. This includes the issue of territorial jurisdiction, and how it obstructs traditional policing online; concerns over free speech and privacy if CMs are given greater powers; debates over whether tech companies should be legally liable for user-generated content and; the impacts (mental and professional) on the very CMs now operating as the new frontline against harmful, often traumatic, materials shared on social media. In outlining these issues, our objective is to highlight issues requiring further attention in order to best support CMs, and to enhance responses to harmful online content.

Keywordscontent moderation; social media; online harms; child sexual abuse material; cybercrime; technology industry
Sustainable Development Goals16 Peace, justice and strong institutions
Middlesex University ThemeHealth & Wellbeing
Research GroupCentre for Abuse and Trauma Studies (CATS)
PublisherSage
JournalEuropean Journal of Criminology
ISSN1477-3708
Publication dates
Online02 Jul 2023
PrintMar 2024
Publication process dates
Deposited24 May 2023
Accepted16 May 2023
Output statusPublished
Accepted author manuscript
Copyright Statement

Bleakley, P., Martellozzo, E., Spence, R., & DeMarco, J. (2024). Moderating online child sexual abuse material (CSAM): Does self-regulation work, or is greater state regulation needed? European Journal of Criminology, 21(2), 231-250. https://doi.org/10.1177/14773708231181361
The author accepted manuscript is made available in this repository in accordance with the publisher's (SAGE) self-archiving policy for an Institutional Repository https://uk.sagepub.com/en-gb/eur/posting-to-an-institutional-reposit...

Web address (URL)https://journals.sagepub.com/home/EUC
Digital Object Identifier (DOI)https://doi.org/10.1177/14773708231181361
LanguageEnglish
Permalink -

https://repository.mdx.ac.uk/item/8q61x

Download files


Accepted author manuscript
  • 104
    total views
  • 163
    total downloads
  • 10
    views this month
  • 14
    downloads this month

Export as

Related outputs

Content moderator mental health, secondary trauma, and well-being: a cross-sectional study
Spence, R., Bifulco, A., Bradbury, P., Martellozzo, E. and DeMarco, J. 2024. Content moderator mental health, secondary trauma, and well-being: a cross-sectional study. Cyberpsychology, Behavior, and Social Networking. 27 (2), pp. 149-155. https://doi.org/10.1089/cyber.2023.0298
Supporting digital key workers: Addressing the challenges faced by content moderators during and after the COVID-19 pandemic
Martellozzo, E., Bleakley, P., Bradbury, P., Spence, R. and DeMarco, J. 2024. Supporting digital key workers: Addressing the challenges faced by content moderators during and after the COVID-19 pandemic. International Journal of Technology, Policy and Management. 24 (2), pp. 212-228. https://doi.org/10.1504/IJTPM.2024.137818
The psychological impacts of content moderation on content moderators: a qualitative study
Spence, R., Bifulco, A., Bradbury, P., Martellozzo, E. and DeMarco, J. 2023. The psychological impacts of content moderation on content moderators: a qualitative study. Cyberpsychology: Journal of Psychosocial Research on Cyberspace. 17 (4). https://doi.org/10.5817/cp2023-4-8
Content moderators’ strategies for coping with the stress of moderating content online
Spence, R., Harrison, A., Bradbury, P., Bleakley, P., Martellozzo, E. and DeMarco, J. 2023. Content moderators’ strategies for coping with the stress of moderating content online. Journal of Online Trust and Safety. 1 (5), pp. 1-18. https://doi.org/10.54501/jots.v1i5.91
The right tool for the job: evaluating police experiences of a pilot tool for responding to stalking
Bleakley, P., Frost, S., Bradbury, P., Short, E. and Martellozzo, E. 2023. The right tool for the job: evaluating police experiences of a pilot tool for responding to stalking. Policing: a Journal of Policy and Practice. 17. https://doi.org/10.1093/police/paac022
Understanding sexual violence and factors related to police outcomes
Davies, K., Spence, R., Cummings, E., Cross, M. and Horvath, M. 2022. Understanding sexual violence and factors related to police outcomes. Frontiers in Psychology. 13, pp. 1-13. https://doi.org/10.3389/fpsyg.2022.977318
Police responses to cyberstalking during the Covid-19 pandemic in the UK
Martellozzo, E., Bleakley, P., Bradbury, P., Frost, S. and Short, E. 2022. Police responses to cyberstalking during the Covid-19 pandemic in the UK. Police Journal: Theory, Practice and Principles. https://doi.org/10.1177/0032258X221113452
Impacts of the COVID-19 pandemic on reporting of rape, serious sexual offences, and domestic abuse in one English police force
Spence, R., Horvath, M., Dalton, C. and Davies, K. 2022. Impacts of the COVID-19 pandemic on reporting of rape, serious sexual offences, and domestic abuse in one English police force. Journal of Gender-Based Violence. 6 (2), pp. 242-260. https://doi.org/10.1332/239868021X16425806897258
Criminal court sentencing: the case for specialist ‘young adult’ courts
Ward, J. and Spence, R. 2022. Criminal court sentencing: the case for specialist ‘young adult’ courts. The British Journal of Criminology: An International Review of Crime and Society. 63 (4), pp. 1041-1057. https://doi.org/10.1093/bjc/azac076
Frontline response: exploring the impact of COVID‐19 on stalking behaviours
Short, E., Bradbury, P., Martellozzo, E., Frost, S. and Bleakley, P. 2022. Frontline response: exploring the impact of COVID‐19 on stalking behaviours. Journal of Police and Criminal Psychology. 37 (3), pp. 540-548. https://doi.org/10.1007/s11896-022-09514-w
The moderation effect of secure attachment on the relationship between positive events and wellbeing
Spence, R., Kagan, L., Nunn, S., Bailey-Rodriguez, D., Fisher, H., Hosang, G. and Bifulco, A. 2022. The moderation effect of secure attachment on the relationship between positive events and wellbeing. PsyCh Journal. 11 (4), pp. 541-549. https://doi.org/10.1002/pchj.546
Life events, depression and supportive relationships affect academic achievement in university students
Spence, R., Kagan, L., Nunn, S., Bailey-Rodriguez, D., Fisher, H., Hosang, G. and Bifulco, A. 2022. Life events, depression and supportive relationships affect academic achievement in university students. Journal of American College Health. 70 (7), pp. 1931-1935. https://doi.org/10.1080/07448481.2020.1841776
“Lucky Boy!”; Public perceptions of child sexual offending committed by women
Bradbury, P. and Martellozzo, E. 2021. “Lucky Boy!”; Public perceptions of child sexual offending committed by women. Journal of Victimology and Victim Justice. 4 (2), pp. 160-178. https://doi.org/10.1177/25166069211060091
Understanding trauma in children and young people in the school setting
Spence, R., Kagan, L., Kljakovic, M. and Bifulco, A. 2021. Understanding trauma in children and young people in the school setting. Educational and Child Psychology. 38 (1), pp. 87-98.
The child attachment style interview (Child-ASI) and depression: preliminary findings
Bifulco, A., Jacobs, C., Oskis, A., Bunn, A. and Spence, R. 2021. The child attachment style interview (Child-ASI) and depression: preliminary findings. Educational and Child Psychology. 38 (1), pp. 62-74.
Shared practice, learning, and goals between police and young people: a qualitative analysis of the National Volunteer Police Cadets
DeMarco, J. and Bifulco, A. 2021. Shared practice, learning, and goals between police and young people: a qualitative analysis of the National Volunteer Police Cadets. Policing: a Journal of Policy and Practice. 15 (4), pp. 2095-2110. https://doi.org/10.1093/police/paaa010
Life events and emotional disorder revisited: research and clinical applications
Bifulco, A., Spence, R. and Kagan, L. 2020. Life events and emotional disorder revisited: research and clinical applications. London, UK Routledge.
Exploring the removal of online child sexual abuse material in the United Kingdom: processes and practice
Martellozzo, E. and DeMarco, J. 2020. Exploring the removal of online child sexual abuse material in the United Kingdom: processes and practice. Crime Prevention & Community Safety. 22 (4), pp. 331-350. https://doi.org/10.1057/s41300-020-00099-2
Racial differences in Police experiences: investigation into the factors influencing citizen-police co-operation
DeMarco, J. 2016. Racial differences in Police experiences: investigation into the factors influencing citizen-police co-operation. International Journal of Research in Social Sciences and Humanities. 6 (2), pp. 42-63.
Exploring policing and industry practice in the prevention of online child sexual abuse
Davidson, J., Schimmenti, A., Caretti, V., Puccia, A., Corbari, E., Bogaerts, S., Schilder, J., Scally, M., Bifulco, A. and DeMarco, J. 2019. Exploring policing and industry practice in the prevention of online child sexual abuse. in: Bryce, I. and Petherick, W. (ed.) Child Sexual Abuse: Forensic Issues in Evidence, Impact, and Management Academic Press, Elsevier. pp. 657-678
Partner violence in women - associations with childhood maltreatment, attachment style and major depression
Bifulco, A., Damiani, R., Jacobs, C., Bunn, A. and Spence, R. 2019. Partner violence in women - associations with childhood maltreatment, attachment style and major depression. Maltrattamento E Abuso All’infanzia: Rivista Interdisciplinare. 21 (2), pp. 13-28. https://doi.org/10.3280/MAL2019-002002
Researching the affects that on online pornography has on U.K. adolescents aged 11 to 16
Martellozzo, E., Monaghan, A., Davidson, J. and Adler, J. 2020. Researching the affects that on online pornography has on U.K. adolescents aged 11 to 16. SAGE Open. 10 (1), pp. 1-11. https://doi.org/10.1177/2158244019899462
Online child sexual abuse
Martellozzo, E. 2019. Online child sexual abuse. in: Bryce, I., Robinson, Y. and Petherick, W. (ed.) Child Abuse and Neglect: Forensic Issues in Evidence, Impact, and Management Elsevier.
Online sexual grooming: children as victims of online abuse
Martellozzo, E. 2017. Online sexual grooming: children as victims of online abuse. in: Martellozzo, E. and Jane, E. (ed.) Cybercrime and its victims Routledge, Taylor & Francis. pp. 108-128
An adapted REA exploring online child sexual abuse victimisation across the EU
Scally, M. and DeMarco, J. 2016. An adapted REA exploring online child sexual abuse victimisation across the EU. British Psychological Society Annual Conference. Nottingham 26 - 28 Apr 2016
ISEC illegal use of the Internet: policing, collaboration and victimisation—A review of the literature. Document prepared for the European Commission initiative in dealing with cybercrime and cybersecurity
DeMarco, J., Davidson, J., Scally, M. and Long, E. 2015. ISEC illegal use of the Internet: policing, collaboration and victimisation—A review of the literature. Document prepared for the European Commission initiative in dealing with cybercrime and cybersecurity. Middlesex University.
Lifetime trauma, adversity and emotional disorder in older age women
Bifulco, A., Jacobs, C., Oskis, A., Cavana, F. and Spence, R. 2019. Lifetime trauma, adversity and emotional disorder in older age women. Maltrattamento E Abuso All’infanzia: Rivista Interdisciplinare. 21 (2), pp. 29-43. https://doi.org/10.3280/mal2019-002003
A contextual approach to trauma experience: lessons from life events research
Spence, R., Kagan, L. and Bifulco, A. 2019. A contextual approach to trauma experience: lessons from life events research. Psychological Medicine. 49 (9), pp. 1409-1413. https://doi.org/10.1017/s0033291719000850
The long-term effects of childhood financial hardship mediated by physical abuse, shame, and stigma on depression in women
Spence, R., Nunn, S. and Bifulco, A. 2019. The long-term effects of childhood financial hardship mediated by physical abuse, shame, and stigma on depression in women. Maltrattamento E Abuso All’infanzia: Rivista Interdisciplinare. 21 (1), pp. 55-72. https://doi.org/10.3280/MAL2019-001005
Characteristics of severe life events, attachment style, and depression – Using a new online approach
Bifulco, A., Kagan, L., Spence, R., Nunn, S., Bailey-Rodriguez, D., Hosang, G., Taylor, M. and Fisher, H. 2019. Characteristics of severe life events, attachment style, and depression – Using a new online approach. British Journal of Clinical Psychology. 58 (4), pp. 427-439. https://doi.org/10.1111/bjc.12221
Web-based measure of life events using Computerized Life Events and Assessment Record (CLEAR): preliminary cross-sectional study of reliability, validity, and association with depression
Bifulco, A., Spence, R., Nunn, S., Kagan, L., Bailey-Rodriguez, D., Hosang, G., Taylor, M. and Fisher, H. 2019. Web-based measure of life events using Computerized Life Events and Assessment Record (CLEAR): preliminary cross-sectional study of reliability, validity, and association with depression. JMIR Mental Health. 6 (1). https://doi.org/10.2196/10675
Factors associated with variation in hospital use at the end of life in England
Bardsley, M., Georghiou, T., Spence, R. and Billings, J. 2019. Factors associated with variation in hospital use at the end of life in England. BMJ Supportive & Palliative Care. 9 (2), pp. 167-174. https://doi.org/10.1136/bmjspcare-2015-000936
Attachment style, loneliness and depression in older age women
Spence, R., Jacobs, C. and Bifulco, A. 2020. Attachment style, loneliness and depression in older age women. Aging and Mental Health. 24 (5), pp. 837-839. https://doi.org/10.1080/13607863.2018.1553141
Digital dangers and cyber-victimisation: a study of European adolescent online risky behaviour for sexual exploitation
DeMarco, J., Cheevers, C., Davidson, J., Bogaerts, S., Pace, U., Aiken, M., Caretti, V., Schimmenti, A. and Bifulco, A. 2017. Digital dangers and cyber-victimisation: a study of European adolescent online risky behaviour for sexual exploitation. Clinical Neuropsychiatry. 14 (1), pp. 104-112.
Organised and Terrorist Networks: Empirical Research Report
Ruggiero, V., DeMarco, J., Monaghan, A., Huber, F., Jäger, B., Musotto, R., Wall, D., Bonfanti, M. and Meyer- Daetsch, L. 2017. Organised and Terrorist Networks: Empirical Research Report. European Commission.
Child online protection in the MENA region
Patrick, B., Monica, B., Davidson, J., Ali, M., Davison, P., Day, E., DeMarco, J., Rasul, M., Razai, S., Scally, M. and Wex, B. 2016. Child online protection in the MENA region. Centre for Justice and Crime Prevention.
EU online child safety: what does the literature say?
DeMarco, J., Davidson, J., Bifulco, A., Scally, M., Cheevers, C., Schimmenti, A., Caretti, V., Puccia, A., Corbari, E., Schilder, J. and Bogaerts, S. 2016. EU online child safety: what does the literature say? Crimen et Delictum: International Journal of Criminological and Investigative Sciences. 11 (1), pp. 1-19.
Adolescent attachment style in residential care: the attachment style interview and vulnerable attachment style questionnaire
Bifulco, A., Jacobs, C., Ilan-Clarke, Y., Spence, R. and Oskis, A. 2017. Adolescent attachment style in residential care: the attachment style interview and vulnerable attachment style questionnaire. The British Journal of Social Work. 47 (7), pp. 1870-1883. https://doi.org/10.1093/bjsw/bcw117
Help-seeking in emerging adults with and without a history of mental health referral: a qualitative study
Spence, R., Owens-Solari, M. and Goodyer, I. 2016. Help-seeking in emerging adults with and without a history of mental health referral: a qualitative study. BMC Research Notes. 9 (1). https://doi.org/10.1186/s13104-016-2227-8
"I wasn’t sure it was normal to watch it…”A quantitative and qualitative examination of the impact of online pornography on the values, attitudes, beliefs and behaviours of children and young people
Martellozzo, E., Monaghan, A., Adler, J., Davidson, J., Leyva, R. and Horvath, M. 2016. "I wasn’t sure it was normal to watch it…”A quantitative and qualitative examination of the impact of online pornography on the values, attitudes, beliefs and behaviours of children and young people. Middlesex University, NSPCC, OCC. https://doi.org/10.6084/m9.figshare.3382393
Enhancing police and industry practice
Davidson, J., DeMarco, J., Bifulco, A., Bogaerts, S., Caretti, V., Aiken, M., Chevers, C., Corbari, E., Scally, M., Schimmenti, A. and Puccia, A. 2016. Enhancing police and industry practice. Middlesex University.
Policing online child sexual abuse - the British experience
Martellozzo, E. 2015. Policing online child sexual abuse - the British experience. European Journal of policing Studies. 3 (1), pp. 32-52.
Measuring life events and their association with clinical disorder: a protocol for development of an online approach
Spence, R., Bunn, A., Nunn, S., Hosang, G., Kagan, L., Fisher, H., Taylor, M. and Bifulco, A. 2015. Measuring life events and their association with clinical disorder: a protocol for development of an online approach. JMIR Research Protocols. 4 (3), p. e83. https://doi.org/10.2196/resprot.4085
Focus on: antidepressant prescribing trends in the prescribing of antidepressants in primary care
Spence, R., Roberts, A., Ariti, C. and Bardsley, M. 2014. Focus on: antidepressant prescribing trends in the prescribing of antidepressants in primary care. Nuffield Trust.
Exploring young people's use of social networking sites and digital media in the internet safety context: a comparison of the UK and Bahrain
Davidson, J. and Martellozzo, E. 2013. Exploring young people's use of social networking sites and digital media in the internet safety context: a comparison of the UK and Bahrain. Information, Communication & Society. 16 (9), pp. 1456-1476. https://doi.org/10.1080/1369118X.2012.701655
Item response theory and validity of the NEO-FFI in adolescents
Spence, R., Owens, M. and Goodyer, I. 2012. Item response theory and validity of the NEO-FFI in adolescents. Personality and Individual Differences. 53 (6), pp. 801-807. https://doi.org/10.1016/j.paid.2012.06.002
Exploring young people's use of social networking sites and digital media in the Internet safety context: a comparison of the UK and Bahrain
Davidson, J. and Martellozzo, E. 2013. Exploring young people's use of social networking sites and digital media in the Internet safety context: a comparison of the UK and Bahrain. Information, Communication & Society. 16 (9), pp. 1456-1476. https://doi.org/10.1080/1369118X.2012.701655
Protecting vulnerable young people in cyberspace from sexual abuse: raising awareness and responding globally
Davidson, J. and Martellozzo, E. 2008. Protecting vulnerable young people in cyberspace from sexual abuse: raising awareness and responding globally. Police Practice and Research. 9 (4), pp. 277-289. https://doi.org/10.1080/15614260802349965
Setting up a youth violence prevention project in a London hospital emergency department
Ilan-Clarke, Y., Bunn, A., DeMarco, J., Bifulco, A., Criddle, J. and Holdsworth, G. 2013. Setting up a youth violence prevention project in a London hospital emergency department. Journal of Public Mental Health. 12 (2), pp. 80-92. https://doi.org/10.1108/JPMH-09-2012-0005
The longitudinal psychometric properties of the EAS temperament survey in adolescence
Spence, R., Owens, M. and Goodyer, I. 2013. The longitudinal psychometric properties of the EAS temperament survey in adolescence. Journal of Personality Assessment. 95 (6), pp. 633-639. https://doi.org/10.1080/00223891.2013.819513
Cuidado y protección de menores contra el abuso sexual on line
Martellozzo, E. 2009. Cuidado y protección de menores contra el abuso sexual on line. Cuadernos de Seguridad. 11, pp. 151-181.
Online child sexual abuse by female offenders: an exploratory study
Martellozzo, E., Nehring, D. and Taylor, H. 2010. Online child sexual abuse by female offenders: an exploratory study. International Journal of Cyber Criminology. 4 (1&2), pp. 592-609.
Protecting children online: towards a safer internet
Davidson, J. and Martellozzo, E. 2008. Protecting children online: towards a safer internet. in: Wetherby, G., Williams, K., Birch, P. and Cain, M. (ed.) Sex as crime? Willan. pp. 338-355
Internet sex offenders: individual autonomy, ‘folk devils’ and state control
Martellozzo, E., Davidson, J. and Cambridge Socio-Legal Group 2009. Internet sex offenders: individual autonomy, ‘folk devils’ and state control. in: Sclater, S., Ebtehaj, F., Jackson, E. and Richards, M. (ed.) Regulating autonomy: sex, reproduction and family Oxford, UK Hart Publishing. pp. 129-146
Understanding the perpetrators' online behaviour
Martellozzo, E. 2010. Understanding the perpetrators' online behaviour. in: Davidson, J. and Gottschalk, P. (ed.) Internet child abuse: current research and policy Abingdon, UK Routledge-Cavendish. pp. 104-125
Online child sexual abuse: grooming, policing and child protection in a multi-media world
Martellozzo, E. 2012. Online child sexual abuse: grooming, policing and child protection in a multi-media world. London, UK Routledge.
Globalization, urbanization & security
Davidson, J. and Martellozzo, E. 2009. Globalization, urbanization & security. in: Cordner, G., Cordner, A. and Das, D. (ed.) Urbanization, policing & security: global perspectives CRC Press.
Clinical and economic outcomes from the UK pilot psychiatric services for personality-disordered offenders
Fortune, Z., Barrett, B., Armstrong, D., Coid, J., Crawford, M., Mudd, D., Rose, D., Slade, M., Spence, R., Tyrer, P. and Moran, P. 2011. Clinical and economic outcomes from the UK pilot psychiatric services for personality-disordered offenders. International Review of Psychiatry. 23 (1), pp. 61-69. https://doi.org/10.3109/09540261.2010.545989
Urbanization and security: moving forward, key themes, and challenges
Davidson, J. and Martellozzo, E. 2009. Urbanization and security: moving forward, key themes, and challenges. in: Cordner, G., Das, D. and Cordner, A. (ed.) Urbanization, policing, and security: global perspectives. CRC Press. pp. 435-440
Sex offenders' use of the Internet.
Martellozzo, E. 2010. Sex offenders' use of the Internet. in: Davidson, J. and Gottschalk, P. (ed.) Internet child abuse: current research and policy. Routledge-Cavendish. pp. 104-125
Evaluation of CEOP ThinkUKnow internet safety programme and exploration of young people's internet safety knowledge.
Davidson, J., Martellozzo, E. and Lorenz, M. 2009. Evaluation of CEOP ThinkUKnow internet safety programme and exploration of young people's internet safety knowledge. Centre for Abuse & Trauma Studies and Kingston University.
Cycle of abuse
Martellozzo, E. and Taylor, H. 2009. Cycle of abuse. Index on Censorship. 38 (1), pp. 117-122. https://doi.org/10.1080/03064220902761106
An evaluation of new services for personality-disordered offenders: staff and service user perspectives
Fortune, Z., Rose, D., Crawford, M., Slade, M., Spence, R., Mudd, D., Barrett, B., Coid, J., Tyrer, P. and Moran, P. 2010. An evaluation of new services for personality-disordered offenders: staff and service user perspectives. International Journal of Social Psychiatry. 56 (2), pp. 186-195. https://doi.org/10.1177/0020764009105281