Representative design: a realistic alternative to (systematic) integrative design

Article

Holleman, G., Dhami, M., Hooge, I. and Hessels, R. 2024. Representative design: a realistic alternative to (systematic) integrative design. Behavioral and Brain Sciences: An International Journal of Current Research and Theory with Open Peer Commentary. 47. https://doi.org/10.1017/S0140525X23002200
TypeArticle
TitleRepresentative design: a realistic alternative to (systematic) integrative design
AuthorsHolleman, G., Dhami, M., Hooge, I. and Hessels, R.
Abstract

We disagree with Almaatouq et al. that no realistic alternative exists to the "one-at-a-time" paradigm. Seventy years ago, Egon Brunswik introduced representative design which offers a clear path to commensurability and generality. Almaatouq et al.’s integrative design cannot guarantee the external validity and generalizability of results which is sorely needed, while representative design tackles the problem head on.

Keywordsintegrative design; representative design; generalizability; external validity
Sustainable Development Goals4 Quality education
9 Industry, innovation and infrastructure
PublisherCambridge University Press (CUP)
JournalBehavioral and Brain Sciences: An International Journal of Current Research and Theory with Open Peer Commentary
ISSN0140-525X
Electronic1469-1825
Publication dates
Online05 Feb 2024
Print2024
Publication process dates
Accepted14 Apr 2023
Deposited24 Apr 2023
Output statusPublished
Accepted author manuscript
License
File Access Level
Open
Digital Object Identifier (DOI)https://doi.org/10.1017/S0140525X23002200
PubMed ID38311450
LanguageEnglish
Permalink -

https://repository.mdx.ac.uk/item/8q5q2

Log in to edit

Download files

Accepted author manuscript
BBS-S-23-00409.pdf
License: CC BY-NC-ND 4.0
File access level: Open

  • 224
    total views
  • 115
    total downloads
  • 11
    views this month
  • 4
    downloads this month

Export as

Related outputs

Beyond a reasonable doubt: ordinary people’s views of an extraordinary concept
Dhami, M. and Lundrigan, S. 2026. Beyond a reasonable doubt: ordinary people’s views of an extraordinary concept. Psychology, Public Policy and Law.
Margin-aware active learning for user-adaptive text classification
Tirunagari, S., Dhami, M. and Windridge, D. 2026. Margin-aware active learning for user-adaptive text classification. 9th International Conference on Natural Language Processing and Information Retrieval. Kyushu University, Fukuoka, Japan 12 - 14 Dec 2025 Springer.
Judges on trial: the future of research on judicial decision-making
Dhami, M., Kajdanowicz, T., Windridge, D. and Boukacem-Zeghmouri, C. 2026. Judges on trial: the future of research on judicial decision-making. in: Shortland, N. and Alison, L. (ed.) Big Ideas in Forensic Psychology: Visions of a Forensic Future From Leading Voices in the Field Wiley.
The application of pre-trained transformer models to UK Court of Appeal legal judgments
Abbas, W., Zia, T., Tirunagari, S., Chennareddy, V., Dhami, M. and Windridge, D. 2025. The application of pre-trained transformer models to UK Court of Appeal legal judgments. Quan, T.T., Sombattheera, C., Pham, H.-A. and Tran, N.T. (ed.) 18th International Conference on Multi-disciplinary Trends in Artificial Intelligence. Ho Chi Minh City, Vietnam 03 - 05 Dec 2025 Singapore Springer. pp. 252-264 https://doi.org/10.1007/978-981-95-4963-4_21
Adapting scenario planning to create an expectation for surprises: going beyond probability and plausibility in risk assessment
Derbyshire, J., Dhami, M., Belton, I., Önkal, D. and Aven, T. 2025. Adapting scenario planning to create an expectation for surprises: going beyond probability and plausibility in risk assessment. Risk Analysis. 45 (11), pp. 3737-3757. https://doi.org/10.1111/risa.70112
Visualizing versus verbalizing uncertainty in intelligence analysis
Dhami, M., Witt, J.K. and De Werd, P. 2025. Visualizing versus verbalizing uncertainty in intelligence analysis. Intelligence and National Security. 40 (2), pp. 302-327. https://doi.org/10.1080/02684527.2025.2468049
Effects of task structure and confirmation bias in alternative hypotheses evaluation
Dhami, M., Belton, I., de Werd, P., Hadzhieva, V. and Wicke, L. 2024. Effects of task structure and confirmation bias in alternative hypotheses evaluation. Cognitive Research: Principles and Implications. 9 (1). https://doi.org/10.1186/s41235-024-00560-y
Chapter 30: Which are the main characteristics determining sentence severity? An empirical exploration of shoplifting offences using spike-and-slab models
Pina-Sanchez, J., Dhami, M. and Gosling, J. 2024. Chapter 30: Which are the main characteristics determining sentence severity? An empirical exploration of shoplifting offences using spike-and-slab models. in: Fix, P. and Montgomery, M. (ed.) Research Handbook on Judicial Politics Cheltenham, UK Edward Elgar Publishing. pp. 450-464
Possibilities for decision science in the metaverse
Dhami, M. and Zhu, Y. 2024. Possibilities for decision science in the metaverse. Decision. 11 (4), pp. 523-536. https://doi.org/10.1037/dec0000235
The role of character-based personal mitigation in sentencing judgments
Belton, I. and Dhami, M. 2024. The role of character-based personal mitigation in sentencing judgments. Journal of Empirical Legal Studies. 21 (1), pp. 208-239. https://doi.org/10.1111/jels.12376
On conducting ethically-sound psychological science in the metaverse
Cockerton, T., Zhu, Y. and Dhami, M. 2024. On conducting ethically-sound psychological science in the metaverse. American Psychologist. 79 (1), pp. 92-108. https://doi.org/10.1037/amp0001211
Health, wellbeing, and social interaction: an international and demographic analysis of perceived life changes and the positives and negatives of the COVID-19 lockdown
Murray, J., Dhami, M., McClatchey, K., Weiss-Cohen, L. and Ayton, P. 2023. Health, wellbeing, and social interaction: an international and demographic analysis of perceived life changes and the positives and negatives of the COVID-19 lockdown. Europe’s Journal of Psychology. 19 (2), pp. 143-157. https://doi.org/10.5964/ejop.7751
Insights into accuracy of social scientists' forecasts of societal change
Grossman, I., Rotella, A., Hutcherson, C., Sharpinskyi, K., Varnum, M., Achter, S., Dhami, M. and The Forecasting Collaborative 2023. Insights into accuracy of social scientists' forecasts of societal change. Nature Human Behaviour. https://doi.org/10.1038/s41562-022-01517-1
The value of experiments in futures and foresight science: a reply
Derbyshire, J., Dhami, M., Belton, I. and Onkal, D. 2023. The value of experiments in futures and foresight science: a reply. Futures & Foresight Science. 5 (2). https://doi.org/10.1002/ffo2.156
Meta-informational cue inconsistency and judgment of information accuracy: Spotlight on intelligence analysis
Mandel, D., Irwin, D., Dhami, M. and Budescu, D. 2023. Meta-informational cue inconsistency and judgment of information accuracy: Spotlight on intelligence analysis. Journal of Behavioral Decision Making. 36 (3). https://doi.org/10.1002/bdm.2307
Patterns and predictors of stranger rape locations
Lundrigan, S., Weir, R., Newton, A., Agudelo, K. and Dhami, M. 2022. Patterns and predictors of stranger rape locations. European Journal on Criminal Policy and Research. https://doi.org/10.1007/s10610-022-09535-5
The value of experiments in futures and foresight science as illustrated by the case of scenario planning
Derbyshire, J., Dhami, M., Belton, I. and Onkal, D. 2022. The value of experiments in futures and foresight science as illustrated by the case of scenario planning. Futures & Foresight Science. 5 (2). https://doi.org/10.1002/ffo2.146
Scenario generation and scenario quality using the cone of plausibility
Dhami, M., Wicke, L. and Onkal, D. 2022. Scenario generation and scenario quality using the cone of plausibility. Futures. 142, pp. 1-11. https://doi.org/10.1016/j.futures.2022.102995
A multi-disciplinary and comparative approach to evaluating pre-trial detention decisions: towards evidence-based reform
Dhami, M. and van den Brink, Y. 2022. A multi-disciplinary and comparative approach to evaluating pre-trial detention decisions: towards evidence-based reform. European Journal on Criminal Policy and Research. 28 (3), pp. 381-395. https://doi.org/10.1007/s10610-022-09510-0
Communicating uncertainty using words and numbers
Dhami, M. and Mandel, D. 2022. Communicating uncertainty using words and numbers. Trends in Cognitive Sciences. 26 (6), pp. 514-526. https://doi.org/10.1016/j.tics.2022.03.002
Intelligence analysis support guide: development and validation
Dhami, M. and Careless, K. 2023. Intelligence analysis support guide: development and validation. International Journal of Intelligence and Counter Intelligence. 36 (1), pp. 241-259. https://doi.org/10.1080/08850607.2021.1966589
Sentencing multiple- versus single-offence cases: does more crime mean less punishment?
Dhami, M. 2022. Sentencing multiple- versus single-offence cases: does more crime mean less punishment? The British Journal of Criminology: An International Review of Crime and Society. 62 (1), pp. 55-72. https://doi.org/10.1093/bjc/azab030
Arithmetic computation with probability words and numbers
Mandel, D., Dhami, M., Tran, S. and Irwin, D. 2021. Arithmetic computation with probability words and numbers. Journal of Behavioral Decision Making. 34 (4), pp. 593-608. https://doi.org/10.1002/bdm.2232
Words or numbers? Communicating probability in intelligence analysis
Dhami, M. and Mandel, D.R. 2021. Words or numbers? Communicating probability in intelligence analysis. American Psychologist. 76 (3), pp. 549-560. https://doi.org/10.1037/amp0000637
Disengaging and rehabilitating high-value detainees: a small scale qualitative study
Dhami, M., Goodman-Delahunty, J., Martschuk, N., Cheung, S. and Belton, I. 2020. Disengaging and rehabilitating high-value detainees: a small scale qualitative study. Journal for Deradicalization.
Using scenarios to forecast outcomes of a refugee crisis
Wicke, L., Dhami, M., Onkal, D. and Belton, I. 2022. Using scenarios to forecast outcomes of a refugee crisis. International Journal of Forecasting. 38 (3), pp. 1175-1184. https://doi.org/10.1016/j.ijforecast.2019.05.017
Are people experiencing the ‘pains of imprisonment’ during the Covid-19 lockdown?
Dhami, M., Weiss-Cohen, L. and Ayton, P. 2020. Are people experiencing the ‘pains of imprisonment’ during the Covid-19 lockdown? Frontiers in Psychology. 11. https://doi.org/10.3389/fpsyg.2020.578430
Factors predicting conviction in child stranger rape
Lundrigan, S., Dhami, M. and Agudelo, K. 2020. Factors predicting conviction in child stranger rape. Child Abuse and Neglect. 101. https://doi.org/10.1016/j.chiabu.2019.104242
Criminal sentencing by preferred numbers
Dhami, M., Belton, I., Merrall, E., McGrath, A. and Bird, S. 2020. Criminal sentencing by preferred numbers. Journal of Empirical Legal Studies. 17 (1), pp. 139-163. https://doi.org/10.1111/jels.12246
A survey of intelligence analysts’ strategies for solving analytic tasks
Dhami, M. and Careless, K. 2019. A survey of intelligence analysts’ strategies for solving analytic tasks. Military Psychology. 31 (2), pp. 117-127. https://doi.org/10.1080/08995605.2018.1561105
The "analysis of competing hypotheses" in intelligence analysis
Dhami, M., Belton, I. and Mandel, D. 2019. The "analysis of competing hypotheses" in intelligence analysis. Applied Cognitive Psychology. 33 (6), pp. 1080-1090. https://doi.org/10.1002/acp.3550
Factors predicting jury convictions in stranger rape cases
Lundrigan, S., Dhami, M. and Agudelo, K. 2019. Factors predicting jury convictions in stranger rape cases. Frontiers in Psychology. 10. https://doi.org/10.3389/fpsyg.2019.00526
Improving procedural fairness in housing possession cases
Whitehouse, L., Bright, S. and Dhami, M. 2019. Improving procedural fairness in housing possession cases. Civil justice quarterly. 38 (3), pp. 351-375.
A survey of intelligence analysts’ perceptions of analytic tools
Dhami, M. 2017. A survey of intelligence analysts’ perceptions of analytic tools. 2017 European Intelligence and Security Informatics Conference. Athens, Greece 11 - 13 Sep 2017 Institute of Electrical and Electronics Engineers (IEEE). pp. 131-134 https://doi.org/10.1109/EISIC.2017.26
Boosting intelligence analysts’ judgment accuracy: what works, what fails?
Mandel, D., Karvetski, C. and Dhami, M. 2018. Boosting intelligence analysts’ judgment accuracy: what works, what fails? Judgment and Decision Making. 13 (6), pp. 607-621.
Police discretion in rape cases
Dhami, M., Lundrigan, S. and Thomas, S. 2018. Police discretion in rape cases. Journal of Police and Criminal Psychology. https://doi.org/10.1007/s11896-018-9299-4
A re-examination of the acquittal biasing effect of offence seriousness
Lundrigan, S., Dhami, M. and Mueller-Johnson, K. 2018. A re-examination of the acquittal biasing effect of offence seriousness. Psychiatry, Psychology and Law. 25 (5), pp. 769-778. https://doi.org/10.1080/13218719.2018.1478334
Towards an evidence-based approach to communicating uncertainty in intelligence analysis
Dhami, M. 2018. Towards an evidence-based approach to communicating uncertainty in intelligence analysis. Intelligence and National Security. 33 (2), pp. 257-272. https://doi.org/10.1080/02684527.2017.1394252
Kenneth R. Hammond’s contributions to the study of human judgment and decision making
Dhami, M. and Mumpower, J. 2018. Kenneth R. Hammond’s contributions to the study of human judgment and decision making. Judgment and Decision Making. 13 (1), pp. 1-22.
Effects of judicial instructions and juror characteristics on interpretations of beyond reasonable doubt
Mueller-Johnson, K., Dhami, M. and Lundrigan, S. 2018. Effects of judicial instructions and juror characteristics on interpretations of beyond reasonable doubt. Psychology, Crime and Law. 24 (2), pp. 117-133. https://doi.org/10.1080/1068316X.2017.1394461
An empirical note on perceptions of partial apologies
Dhami, M. 2017. An empirical note on perceptions of partial apologies. Onati Socio-Legal Series. 7 (3), pp. 408-420.
Development of an information sheet providing rapport advice for interpreters in police interviews
Dhami, M., Goodman-Delahunty, J. and Desai, S. 2017. Development of an information sheet providing rapport advice for interpreters in police interviews. Police Practice and Research. 18 (3), pp. 291-305. https://doi.org/10.1080/15614263.2017.1291580
Critical review of analytic techniques
Dhami, M., Belton, I. and Careless, K. 2016. Critical review of analytic techniques. European Intelligence and Security Informatics Conference (EISIC) 2016. Uppsala, Sweden 17 - 19 Aug 2016 Institute of Electrical and Electronics Engineers (IEEE). pp. 152-155 https://doi.org/10.1109/EISIC.2016.33
Male youth perceptions of violent extremism: towards a test of rational choice theory
Dhami, M. and Murray, J. 2016. Male youth perceptions of violent extremism: towards a test of rational choice theory. The Spanish Journal of Psychology. 19. https://doi.org/10.1017/sjp.2016.49
On getting inside the judge’s mind
Dhami, M. and Belton, I. 2017. On getting inside the judge’s mind. Translational Issues in Psychological Science. 3 (2), pp. 214-226. https://doi.org/10.1037/tps0000115
Statistical analyses of court decisions: an example of multilevel models of sentencing
Dhami, M. and Belton, I. 2016. Statistical analyses of court decisions: an example of multilevel models of sentencing. Law and Method. 10, pp. 247-266. https://doi.org/10.5553/REM/.000019
Towards a pedagogy for critical security studies: politics of migration in the classroom
Bilgic, A., Dhami, M. and Onkal, D. 2018. Towards a pedagogy for critical security studies: politics of migration in the classroom. International Studies Perspectives. 19 (3), pp. 250-266. https://doi.org/10.1093/isp/ekx016
Using court records for sentencing research: pitfalls and possibilities
Dhami, M. and Belton, I. 2015. Using court records for sentencing research: pitfalls and possibilities. in: Roberts, J. (ed.) Exploring Sentencing Practice in England and Wales Springer.
Effects of a victim’s response to an offender’s apology: when the victim becomes the bad guy
Dhami, M. 2016. Effects of a victim’s response to an offender’s apology: when the victim becomes the bad guy. European Journal of Social Psychology. 46 (1), pp. 110-123. https://doi.org/10.1002/ejsp.2145
Improving intelligence analysis with decision science
Dhami, M., Mandel, D., Mellers, B. and Tetlock, P. 2015. Improving intelligence analysis with decision science. Perspectives on Psychological Science. 10 (6), pp. 753-757. https://doi.org/10.1177/1745691615598511
Ordinal structure of the generic analytic workflow: a survey of intelligence analysts
Dhami, M. and Careless, K. 2015. Ordinal structure of the generic analytic workflow: a survey of intelligence analysts. 2015 European Intelligence and Security Informatics Conference. Manchester, UK 07 - 09 Sep 2015 IEEE. pp. 141-144 https://doi.org/10.1109/EISIC.2015.37
Editorial: The aging decision-maker: advances in understanding the impact of cognitive change on decision-making
Ward, E. and Dhami, M. 2016. Editorial: The aging decision-maker: advances in understanding the impact of cognitive change on decision-making. Frontiers in Psychology. 7. https://doi.org/10.3389/fpsyg.2016.01622
Apology in victim-offender mediation
Dhami, M. 2016. Apology in victim-offender mediation. Contemporary Justice Review. 19 (1), pp. 31-42. https://doi.org/10.1080/10282580.2015.1101686
A scientific approach to countering the terrorist threat
Dhami, M. 2014. A scientific approach to countering the terrorist threat. in: Cole, J. and Pantucci, R. (ed.) STFC/RUSI.
Racial disparities, judge characteristics, and standards of review in sentencing: comment
Dhami, M. 2015. Racial disparities, judge characteristics, and standards of review in sentencing: comment. Journal of Institutional and Theoretical Economics (JITE). 171 (1), pp. 48-52.
Predicting verdicts using pre-trial attitudes and standard of proof
Lundrigan, S., Dhami, M. and Mueller-Johnson, K. 2013. Predicting verdicts using pre-trial attitudes and standard of proof. Legal and Criminological Psychology. https://doi.org/10.1111/lcrp.12043
Normative and descriptive models of military decisions to deploy precision strike capabilities
Banks, A. and Dhami, M. 2014. Normative and descriptive models of military decisions to deploy precision strike capabilities. Military Psychology. 26 (1), pp. 33-43. https://doi.org/10.1037/mil0000028
Lawyer and nonlawyer susceptibility to framing effects in out-of-court civil litigation settlement
Belton, I., Thomson, M. and Dhami, M. 2014. Lawyer and nonlawyer susceptibility to framing effects in out-of-court civil litigation settlement. Journal of Empirical Legal Studies. 11 (3), pp. 578-600. https://doi.org/10.1111/jels.12050
Quasirational models of sentencing
Dhami, M., Belton, I. and Goodman-Delahunty, J. 2015. Quasirational models of sentencing. Journal of Applied Research in Memory and Cognition. 4 (3), pp. 239-247. https://doi.org/10.1016/j.jarmac.2014.07.009
Improving the communication of uncertainty in climate science and intelligence analysis
Ho, E., Budescu, D., Dhami, M. and Mandel, D. 2015. Improving the communication of uncertainty in climate science and intelligence analysis. Behavioral Science & Policy. 1 (2), pp. 43-55. https://doi.org/10.1353/bsp.2015.0015
Instructions on reasonable doubt: defining the standard of proof and the juror's task
Dhami, M., Lundrigan, S. and Mueller-Johnson, K. 2015. Instructions on reasonable doubt: defining the standard of proof and the juror's task. Psychology, Public Policy and Law. 21 (2), pp. 169-178. https://doi.org/10.1037/law0000038
Standards and best practices in digital and multimedia forensics
Li, S., Dhami, M. and Ho, A. 2015. Standards and best practices in digital and multimedia forensics. in: Ho, A. and Li, S. (ed.) Handbook of Digital Forensics of Multimedia Data and Devices Wiley. pp. 38-93
Psychological perspectives on military decisions to deployprecision strike capabilities
Banks, A. and Dhami, M. 2014. Psychological perspectives on military decisions to deployprecision strike capabilities. in: Aaronson, M., Aslam, W., Dyson, T. and Rauxloh, R. (ed.) Precision Strike Warfare and International Intervention: Strategic, Ethico-Legal and Decisional Implications Routledge. pp. 173-187
Interviewing high value detainees: securing cooperation and disclosures
Goodman-Delahunty, J., Martschuk, N. and Dhami, M. 2014. Interviewing high value detainees: securing cooperation and disclosures. Applied Cognitive Psychology. 28 (6), pp. 883-897. https://doi.org/10.1002/acp.3087
Simply criminal: predicting burglars' occupancy decisions with a simple heuristic
Snook, B., Dhami, M. and Kavanagh, J. 2011. Simply criminal: predicting burglars' occupancy decisions with a simple heuristic. Law and Human Behavior. 35 (4), pp. 316-326. https://doi.org/10.1007/s10979-010-9238-0
Challenges to establishing volunteer‐run, community‐based restorative justice programs
Dhami, M. and Joy, P. 2007. Challenges to establishing volunteer‐run, community‐based restorative justice programs. Contemporary Justice Review. 10 (1), pp. 9-22. https://doi.org/10.1080/10282580601157455
Crime as risk taking
Dhami, M. and Mandel, D. 2012. Crime as risk taking. Psychology, Crime and Law. 18 (4), pp. 389-403. https://doi.org/10.1080/1068316X.2010.498423
Take-the-best in expert-novice decision strategies for residential burglary
Garcia-Retamero, R. and Dhami, M. 2009. Take-the-best in expert-novice decision strategies for residential burglary. Psychonomic Bulletin & Review. 16 (1), pp. 163-169. https://doi.org/10.3758/PBR.16.1.163
On avoiding framing effects in experienced decision makers
Garcia-Retamero, R. and Dhami, M. 2013. On avoiding framing effects in experienced decision makers. Quarterly Journal of Experimental Psychology. 66 (4), pp. 829-842. https://doi.org/10.1080/17470218.2012.727836
Pictures speak louder than numbers: on communicating medical risks to immigrants with limited non-native language proficiency
Garcia-Retamero, R. and Dhami, M. 2011. Pictures speak louder than numbers: on communicating medical risks to immigrants with limited non-native language proficiency. Health Expectations. 14, pp. 46-57. https://doi.org/10.1111/j.1369-7625.2011.00670.x
First-time and recurrent inmates' experiences of imprisonment
Souza, K. and Dhami, M. 2010. First-time and recurrent inmates' experiences of imprisonment. Criminal Justice and Behavior. 37 (12), pp. 1330-1342. https://doi.org/10.1177/0093854810379969
On measuring quantitative interpretations of reasonable doubt
Dhami, M. 2008. On measuring quantitative interpretations of reasonable doubt. Journal of Experimental Psychology: Applied. 14 (4), pp. 353-363. https://doi.org/10.1037/a0013344