Meta-informational cue inconsistency and judgment of information accuracy: Spotlight on intelligence analysis

Article

Mandel, D., Irwin, D., Dhami, M. and Budescu, D. 2023. Meta-informational cue inconsistency and judgment of information accuracy: Spotlight on intelligence analysis. Journal of Behavioral Decision Making. 36 (3). https://doi.org/10.1002/bdm.2307
TypeArticle
TitleMeta-informational cue inconsistency and judgment of information accuracy: Spotlight on intelligence analysis
AuthorsMandel, D., Irwin, D., Dhami, M. and Budescu, D.
Abstract

Meta-information is information about information that can be used as cues to guide judgments and decisions. Three types of meta-information that are routinely used in intelligence analysis are source reliability, information credibility and classification level. The first two cues are intended to speak to information quality (in particular, the probability that the information is accurate), and classification level is intended to describe the information’s security sensitivity. Two experiments involving professional intelligence analysts (N = 25 and 27, respectively) manipulated meta-information in a 6 (source reliability) by 6 (information credibility) by 2 (classification) repeated-measures design. Ten additional items were retested to measure intra-individual reliability. Analysts judged the probability of information accuracy based on its meta-informational profile. In both experiments, the judged probability of information accuracy was sensitive to ordinal position on the scales and the directionality of linguistic terms used to anchor the levels of the two scales. Directionality led analysts to group the first three levels of each scale in a positive group and the fourth and fifth levels in a negative group, with the neutral term “cannot be judged” falling between these groups. Critically, as reliability and credibility cue inconsistency increased, there was a corresponding decrease in intra-analyst reliability, inter-analyst agreement, and effective cue utilization. Neither experiment found a significant effect of classification on probability judgments.

Keywordsinconsistency; information accuracy; information credibility; intelligence analysis; meta-information; secrecy; source reliability
Sustainable Development Goals16 Peace, justice and strong institutions
PublisherWiley
JournalJournal of Behavioral Decision Making
ISSN0894-3257
Electronic1099-0771
Publication dates
Online08 Nov 2022
PrintJul 2023
Publication process dates
Deposited24 Oct 2022
Submitted17 May 2022
Accepted17 Oct 2022
Output statusPublished
Publisher's version
License
File Access Level
Open
Accepted author manuscript
File Access Level
Restricted
Copyright Statement

© 2022 The Authors. Journal of Behavioral Decision Making published by John Wiley & Sons Ltd.
This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.

Digital Object Identifier (DOI)https://doi.org/10.1002/bdm.2307
Web of Science identifierWOS:000880040300001
LanguageEnglish
Permalink -

https://repository.mdx.ac.uk/item/8q1z4

Log in to edit

Download files

Publisher's version
Behavioral Decision Making - 2022 - Mandel - Meta‐informational cue inconsistency and judgment of information accuracy .pdf
License: CC BY-NC-ND 4.0
File access level: Open

  • 83
    total views
  • 22
    total downloads
  • 3
    views this month
  • 2
    downloads this month

Export as

Related outputs

Effects of task structure and confirmation bias in alternative hypotheses evaluation
Dhami, M., Belton, I., de Werd, P., Hadzhieva, V. and Wicke, L. 2024. Effects of task structure and confirmation bias in alternative hypotheses evaluation. Cognitive Research: Principles and Implications. 9 (1). https://doi.org/10.1186/s41235-024-00560-y
Chapter 30: Which are the main characteristics determining sentence severity? An empirical exploration of shoplifting offences using spike-and-slab models
Pina-Sanchez, J., Dhami, M. and Gosling, J. 2024. Chapter 30: Which are the main characteristics determining sentence severity? An empirical exploration of shoplifting offences using spike-and-slab models. in: Fix, P. and Montgomery, M. (ed.) Research Handbook on Judicial Politics Cheltenham, UK Edward Elgar Publishing. pp. 450-464
Possibilities for decision science in the metaverse
Dhami, M. and Zhu, Y. 2024. Possibilities for decision science in the metaverse. Decision. 11 (4), pp. 523-536. https://doi.org/10.1037/dec0000235
The role of character-based personal mitigation in sentencing judgments
Belton, I. and Dhami, M. 2024. The role of character-based personal mitigation in sentencing judgments. Journal of Empirical Legal Studies. https://doi.org/10.1111/jels.12376
On conducting ethically-sound psychological science in the metaverse
Cockerton, T., Zhu, Y. and Dhami, M. 2024. On conducting ethically-sound psychological science in the metaverse. American Psychologist. 79 (1), pp. 92-108. https://doi.org/10.1037/amp0001211
Representative design: a realistic alternative to (systematic) integrative design
Holleman, G., Dhami, M., Hooge, I. and Hessels, R. 2024. Representative design: a realistic alternative to (systematic) integrative design. Behavioral and Brain Sciences: An International Journal of Current Research and Theory with Open Peer Commentary. 47. https://doi.org/10.1017/S0140525X23002200
Health, wellbeing, and social interaction: an international and demographic analysis of perceived life changes and the positives and negatives of the COVID-19 lockdown
Murray, J., Dhami, M., McClatchey, K., Weiss-Cohen, L. and Ayton, P. 2023. Health, wellbeing, and social interaction: an international and demographic analysis of perceived life changes and the positives and negatives of the COVID-19 lockdown. Europe’s Journal of Psychology. 19 (2), pp. 143-157. https://doi.org/10.5964/ejop.7751
The value of experiments in futures and foresight science: A reply
Derbyshire, J., Dhami, M., Belton, I. and Onkal, D. 2023. The value of experiments in futures and foresight science: A reply. Futures & Foresight Science. 5 (2). https://doi.org/10.1002/ffo2.156
Insights into accuracy of social scientists' forecasts of societal change
Grossman, I., Rotella, A., Hutcherson, C., Sharpinskyi, K., Varnum, M., Achter, S., Dhami, M. and The Forecasting Collaborative 2023. Insights into accuracy of social scientists' forecasts of societal change. Nature Human Behaviour. https://doi.org/10.1038/s41562-022-01517-1
Patterns and predictors of stranger rape locations
Lundrigan, S., Weir, R., Newton, A., Agudelo, K. and Dhami, M. 2022. Patterns and predictors of stranger rape locations. European Journal on Criminal Policy and Research. https://doi.org/10.1007/s10610-022-09535-5
The value of experiments in futures and foresight science as illustrated by the case of scenario planning
Derbyshire, J., Dhami, M., Belton, I. and Onkal, D. 2022. The value of experiments in futures and foresight science as illustrated by the case of scenario planning. Futures & Foresight Science. https://doi.org/10.1002/ffo2.146
Scenario generation and scenario quality using the cone of plausibility
Dhami, M., Wicke, L. and Onkal, D. 2022. Scenario generation and scenario quality using the cone of plausibility. Futures. 142, pp. 1-11. https://doi.org/10.1016/j.futures.2022.102995
A multi-disciplinary and comparative approach to evaluating pre-trial detention decisions: towards evidence-based reform
Dhami, M. and van den Brink, Y. 2022. A multi-disciplinary and comparative approach to evaluating pre-trial detention decisions: towards evidence-based reform. European Journal on Criminal Policy and Research. 28 (3), pp. 381-395. https://doi.org/10.1007/s10610-022-09510-0
Communicating uncertainty using words and numbers
Dhami, M. and Mandel, D. 2022. Communicating uncertainty using words and numbers. Trends in Cognitive Sciences. 26 (6), pp. 514-526. https://doi.org/10.1016/j.tics.2022.03.002
Intelligence analysis support guide: development and validation
Dhami, M. and Careless, K. 2023. Intelligence analysis support guide: development and validation. International Journal of Intelligence and Counter Intelligence. 36 (1), pp. 241-259. https://doi.org/10.1080/08850607.2021.1966589
Sentencing multiple- versus single-offence cases: does more crime mean less punishment?
Dhami, M. 2022. Sentencing multiple- versus single-offence cases: does more crime mean less punishment? The British Journal of Criminology: An International Review of Crime and Society. 62 (1), pp. 55-72. https://doi.org/10.1093/bjc/azab030
Arithmetic computation with probability words and numbers
Mandel, D., Dhami, M., Tran, S. and Irwin, D. 2021. Arithmetic computation with probability words and numbers. Journal of Behavioral Decision Making. 34 (4), pp. 593-608. https://doi.org/10.1002/bdm.2232
Words or numbers? Communicating probability in intelligence analysis
Dhami, M. and Mandel, D.R. 2021. Words or numbers? Communicating probability in intelligence analysis. American Psychologist. 76 (3), pp. 549-560. https://doi.org/10.1037/amp0000637
Are people experiencing the ‘pains of imprisonment’ during the Covid-19 lockdown?
Dhami, M., Weiss-Cohen, L. and Ayton, P. 2020. Are people experiencing the ‘pains of imprisonment’ during the Covid-19 lockdown? Frontiers in Psychology, section Personality and Social Psychology. 11. https://doi.org/10.3389/fpsyg.2020.578430
Disengaging and rehabilitating high-value detainees: a small scale qualitative study
Dhami, M., Goodman-Delahunty, J., Martschuk, N., Cheung, S. and Belton, I. 2020. Disengaging and rehabilitating high-value detainees: a small scale qualitative study. Journal for Deradicalization.
Using scenarios to forecast outcomes of a refugee crisis
Wicke, L., Dhami, M., Onkal, D. and Belton, I. 2022. Using scenarios to forecast outcomes of a refugee crisis. International Journal of Forecasting. 38 (3), pp. 1175-1184. https://doi.org/10.1016/j.ijforecast.2019.05.017
Factors predicting conviction in child stranger rape
Lundrigan, S., Dhami, M. and Agudelo, K. 2020. Factors predicting conviction in child stranger rape. Child Abuse and Neglect. 101. https://doi.org/10.1016/j.chiabu.2019.104242
Criminal sentencing by preferred numbers
Dhami, M., Belton, I., Merrall, E., McGrath, A. and Bird, S. 2020. Criminal sentencing by preferred numbers. Journal of Empirical Legal Studies. 17 (1), pp. 139-163. https://doi.org/10.1111/jels.12246
A survey of intelligence analysts’ strategies for solving analytic tasks
Dhami, M. and Careless, K. 2019. A survey of intelligence analysts’ strategies for solving analytic tasks. Military Psychology. 31 (2), pp. 117-127. https://doi.org/10.1080/08995605.2018.1561105
Factors predicting jury convictions in stranger rape cases
Lundrigan, S., Dhami, M. and Agudelo, K. 2019. Factors predicting jury convictions in stranger rape cases. Frontiers in Psychology. 10. https://doi.org/10.3389/fpsyg.2019.00526
The "analysis of competing hypotheses" in intelligence analysis
Dhami, M., Belton, I. and Mandel, D. 2019. The "analysis of competing hypotheses" in intelligence analysis. Applied Cognitive Psychology. 33 (6), pp. 1080-1090. https://doi.org/10.1002/acp.3550
Improving procedural fairness in housing possession cases
Whitehouse, L., Bright, S. and Dhami, M. 2019. Improving procedural fairness in housing possession cases. Civil justice quarterly. 38 (3), pp. 351-375.
A survey of intelligence analysts’ perceptions of analytic tools
Dhami, M. 2017. A survey of intelligence analysts’ perceptions of analytic tools. 2017 European Intelligence and Security Informatics Conference. Athens, Greece 11 - 13 Sep 2017 Institute of Electrical and Electronics Engineers (IEEE). pp. 131-134 https://doi.org/10.1109/EISIC.2017.26
Towards an evidence-based approach to communicating uncertainty in intelligence analysis
Dhami, M. 2018. Towards an evidence-based approach to communicating uncertainty in intelligence analysis. Intelligence and National Security. 33 (2), pp. 257-272. https://doi.org/10.1080/02684527.2017.1394252
Boosting intelligence analysts’ judgment accuracy: what works, what fails?
Mandel, D., Karvetski, C. and Dhami, M. 2018. Boosting intelligence analysts’ judgment accuracy: what works, what fails? Judgment and Decision Making. 13 (6), pp. 607-621.
Police discretion in rape cases
Dhami, M., Lundrigan, S. and Thomas, S. 2018. Police discretion in rape cases. Journal of Police and Criminal Psychology. https://doi.org/10.1007/s11896-018-9299-4
A re-examination of the acquittal biasing effect of offence seriousness
Lundrigan, S., Dhami, M. and Mueller-Johnson, K. 2018. A re-examination of the acquittal biasing effect of offence seriousness. Psychiatry, Psychology and Law. 25 (5), pp. 769-778. https://doi.org/10.1080/13218719.2018.1478334
Kenneth R. Hammond’s contributions to the study of human judgment and decision making
Dhami, M. and Mumpower, J. 2018. Kenneth R. Hammond’s contributions to the study of human judgment and decision making. Judgment and Decision Making. 13 (1), pp. 1-22.
Effects of judicial instructions and juror characteristics on interpretations of beyond reasonable doubt
Mueller-Johnson, K., Dhami, M. and Lundrigan, S. 2018. Effects of judicial instructions and juror characteristics on interpretations of beyond reasonable doubt. Psychology, Crime and Law. 24 (2), pp. 117-133. https://doi.org/10.1080/1068316X.2017.1394461
An empirical note on perceptions of partial apologies
Dhami, M. 2017. An empirical note on perceptions of partial apologies. Onati Socio-Legal Series. 7 (3), pp. 408-420.
Development of an information sheet providing rapport advice for interpreters in police interviews
Dhami, M., Goodman-Delahunty, J. and Desai, S. 2017. Development of an information sheet providing rapport advice for interpreters in police interviews. Police Practice and Research. 18 (3), pp. 291-305. https://doi.org/10.1080/15614263.2017.1291580
Critical review of analytic techniques
Dhami, M., Belton, I. and Careless, K. 2016. Critical review of analytic techniques. European Intelligence and Security Informatics Conference (EISIC) 2016. Uppsala, Sweden 17 - 19 Aug 2016 Institute of Electrical and Electronics Engineers (IEEE). pp. 152-155 https://doi.org/10.1109/EISIC.2016.33
Male youth perceptions of violent extremism: towards a test of rational choice theory
Dhami, M. and Murray, J. 2016. Male youth perceptions of violent extremism: towards a test of rational choice theory. The Spanish Journal of Psychology. 19. https://doi.org/10.1017/sjp.2016.49
On getting inside the judge’s mind
Dhami, M. and Belton, I. 2017. On getting inside the judge’s mind. Translational Issues in Psychological Science. 3 (2), pp. 214-226. https://doi.org/10.1037/tps0000115
Statistical analyses of court decisions: an example of multilevel models of sentencing
Dhami, M. and Belton, I. 2016. Statistical analyses of court decisions: an example of multilevel models of sentencing. Law and Method. 10, pp. 247-266. https://doi.org/10.5553/REM/.000019
Towards a pedagogy for critical security studies: politics of migration in the classroom
Bilgic, A., Dhami, M. and Onkal, D. 2018. Towards a pedagogy for critical security studies: politics of migration in the classroom. International Studies Perspectives. 19 (3), pp. 250-266. https://doi.org/10.1093/isp/ekx016
Using court records for sentencing research: pitfalls and possibilities
Dhami, M. and Belton, I. 2015. Using court records for sentencing research: pitfalls and possibilities. in: Roberts, J. (ed.) Exploring Sentencing Practice in England and Wales Springer.
Effects of a victim’s response to an offender’s apology: when the victim becomes the bad guy
Dhami, M. 2016. Effects of a victim’s response to an offender’s apology: when the victim becomes the bad guy. European Journal of Social Psychology. 46 (1), pp. 110-123. https://doi.org/10.1002/ejsp.2145
Improving intelligence analysis with decision science
Dhami, M., Mandel, D., Mellers, B. and Tetlock, P. 2015. Improving intelligence analysis with decision science. Perspectives on Psychological Science. 10 (6), pp. 753-757. https://doi.org/10.1177/1745691615598511
Ordinal structure of the generic analytic workflow: a survey of intelligence analysts
Dhami, M. and Careless, K. 2015. Ordinal structure of the generic analytic workflow: a survey of intelligence analysts. 2015 European Intelligence and Security Informatics Conference. Manchester, UK 07 - 09 Sep 2015 IEEE. pp. 141-144 https://doi.org/10.1109/EISIC.2015.37
First-time and recurrent inmates' experiences of imprisonment
Souza, K. and Dhami, M. 2010. First-time and recurrent inmates' experiences of imprisonment. Criminal Justice and Behavior. 37 (12), pp. 1330-1342. https://doi.org/10.1177/0093854810379969
Editorial: The aging decision-maker: advances in understanding the impact of cognitive change on decision-making
Ward, E. and Dhami, M. 2016. Editorial: The aging decision-maker: advances in understanding the impact of cognitive change on decision-making. Frontiers in Psychology. 7. https://doi.org/10.3389/fpsyg.2016.01622
Apology in victim-offender mediation
Dhami, M. 2016. Apology in victim-offender mediation. Contemporary Justice Review. 19 (1), pp. 31-42. https://doi.org/10.1080/10282580.2015.1101686
A scientific approach to countering the terrorist threat
Dhami, M. 2014. A scientific approach to countering the terrorist threat. in: Cole, J. and Pantucci, R. (ed.) STFC/RUSI.
Racial disparities, judge characteristics, and standards of review in sentencing: comment
Dhami, M. 2015. Racial disparities, judge characteristics, and standards of review in sentencing: comment. Journal of Institutional and Theoretical Economics (JITE). 171 (1), pp. 48-52.
Predicting verdicts using pre-trial attitudes and standard of proof
Lundrigan, S., Dhami, M. and Mueller-Johnson, K. 2013. Predicting verdicts using pre-trial attitudes and standard of proof. Legal and Criminological Psychology. https://doi.org/10.1111/lcrp.12043
Normative and descriptive models of military decisions to deploy precision strike capabilities
Banks, A. and Dhami, M. 2014. Normative and descriptive models of military decisions to deploy precision strike capabilities. Military Psychology. 26 (1), pp. 33-43. https://doi.org/10.1037/mil0000028
Lawyer and nonlawyer susceptibility to framing effects in out-of-court civil litigation settlement
Belton, I., Thomson, M. and Dhami, M. 2014. Lawyer and nonlawyer susceptibility to framing effects in out-of-court civil litigation settlement. Journal of Empirical Legal Studies. 11 (3), pp. 578-600. https://doi.org/10.1111/jels.12050
Quasirational models of sentencing
Dhami, M., Belton, I. and Goodman-Delahunty, J. 2015. Quasirational models of sentencing. Journal of Applied Research in Memory and Cognition. 4 (3), pp. 239-247. https://doi.org/10.1016/j.jarmac.2014.07.009
Improving the communication of uncertainty in climate science and intelligence analysis
Ho, E., Budescu, D., Dhami, M. and Mandel, D. 2015. Improving the communication of uncertainty in climate science and intelligence analysis. Behavioral Science & Policy. 1 (2), pp. 43-55. https://doi.org/10.1353/bsp.2015.0015
Instructions on reasonable doubt: defining the standard of proof and the juror's task
Dhami, M., Lundrigan, S. and Mueller-Johnson, K. 2015. Instructions on reasonable doubt: defining the standard of proof and the juror's task. Psychology, Public Policy and Law. 21 (2), pp. 169-178. https://doi.org/10.1037/law0000038
Standards and best practices in digital and multimedia forensics
Li, S., Dhami, M. and Ho, A. 2015. Standards and best practices in digital and multimedia forensics. in: Ho, A. and Li, S. (ed.) Handbook of Digital Forensics of Multimedia Data and Devices Wiley. pp. 38-93
Psychological perspectives on military decisions to deployprecision strike capabilities
Banks, A. and Dhami, M. 2014. Psychological perspectives on military decisions to deployprecision strike capabilities. in: Aaronson, M., Aslam, W., Dyson, T. and Rauxloh, R. (ed.) Precision Strike Warfare and International Intervention: Strategic, Ethico-Legal and Decisional Implications Routledge. pp. 173-187
Interviewing high value detainees: securing cooperation and disclosures
Goodman-Delahunty, J., Martschuk, N. and Dhami, M. 2014. Interviewing high value detainees: securing cooperation and disclosures. Applied Cognitive Psychology. 28 (6), pp. 883-897. https://doi.org/10.1002/acp.3087
Simply criminal: predicting burglars' occupancy decisions with a simple heuristic
Snook, B., Dhami, M. and Kavanagh, J. 2011. Simply criminal: predicting burglars' occupancy decisions with a simple heuristic. Law and Human Behavior. 35 (4), pp. 316-326. https://doi.org/10.1007/s10979-010-9238-0
Challenges to establishing volunteer‐run, community‐based restorative justice programs
Dhami, M. and Joy, P. 2007. Challenges to establishing volunteer‐run, community‐based restorative justice programs. Contemporary Justice Review. 10 (1), pp. 9-22. https://doi.org/10.1080/10282580601157455
Crime as risk taking
Dhami, M. and Mandel, D. 2012. Crime as risk taking. Psychology, Crime and Law. 18 (4), pp. 389-403. https://doi.org/10.1080/1068316X.2010.498423
Take-the-best in expert-novice decision strategies for residential burglary
Garcia-Retamero, R. and Dhami, M. 2009. Take-the-best in expert-novice decision strategies for residential burglary. Psychonomic Bulletin & Review. 16 (1), pp. 163-169. https://doi.org/10.3758/PBR.16.1.163
Pictures speak louder than numbers: on communicating medical risks to immigrants with limited non-native language proficiency
Garcia-Retamero, R. and Dhami, M. 2011. Pictures speak louder than numbers: on communicating medical risks to immigrants with limited non-native language proficiency. Health Expectations. 14, pp. 46-57. https://doi.org/10.1111/j.1369-7625.2011.00670.x
On avoiding framing effects in experienced decision makers
Garcia-Retamero, R. and Dhami, M. 2013. On avoiding framing effects in experienced decision makers. Quarterly Journal of Experimental Psychology. 66 (4), pp. 829-842. https://doi.org/10.1080/17470218.2012.727836
On measuring quantitative interpretations of reasonable doubt
Dhami, M. 2008. On measuring quantitative interpretations of reasonable doubt. Journal of Experimental Psychology: Applied. 14 (4), pp. 353-363. https://doi.org/10.1037/a0013344