Using machine learning to infer reasoning provenance from user interaction log data: based on the data/frame theory of sensemaking

Article


Kodagoda, N., Pontis, S., Simmie, D., Attfield, S., Wong, B., Blandford, A. and Hankin, C. 2017. Using machine learning to infer reasoning provenance from user interaction log data: based on the data/frame theory of sensemaking. Journal of Cognitive Engineering and Decision Making. 11 (1), pp. 23-41. https://doi.org/10.1177/1555343416672782
TypeArticle
TitleUsing machine learning to infer reasoning provenance from user interaction log data: based on the data/frame theory of sensemaking
AuthorsKodagoda, N., Pontis, S., Simmie, D., Attfield, S., Wong, B., Blandford, A. and Hankin, C.
Abstract

The reconstruction of analysts’ reasoning processes (reasoning provenance) during complex sensemaking tasks can support reflection and decision making. One potential approach to such reconstruction is to automatically infer reasoning from low-level user interaction logs. We explore a novel method for doing this using machine learning. Two user studies were conducted in which participants performed similar intelligence analysis tasks. In one study, participants used a standard web browser and word processor; in the other, they used a system called INVISQUE (Interactive Visual Search and Query Environment). Interaction logs were manually coded for cognitive actions based on captured think-aloud protocol and posttask interviews based on Klein, Phillips, Rall, and Pelusos’s data/frame model of sensemaking as a conceptual framework. This analysis was then used to train an interaction frame mapper, which employed multiple machine learning models to learn relationships between the interaction logs and the codings. Our results show that, for one study at least, classification accuracy was significantly better than chance and compared reasonably to a reported manual provenance reconstruction method. We discuss our results in terms of variations in feature sets from the two studies and what this means for the development of the method for provenance capture and the evaluation of sensemaking systems.

LanguageEnglish
PublisherSage
JournalJournal of Cognitive Engineering and Decision Making
ISSN1555-3434
Publication dates
Online24 Oct 2016
Print01 Mar 2017
Publication process dates
Deposited07 Mar 2018
Accepted08 Sep 2016
Output statusPublished
Accepted author manuscript
Copyright Statement

Neesha Kodagoda, Sheila Pontis, Donal Simmie, Simon Attfield, B. L. William Wong, Ann Blandford, Chris Hankin, Using Machine Learning to Infer Reasoning Provenance From User Interaction Log Data: Based on the Data/Frame Theory of Sensemaking, Journal of Cognitive Engineering and Decision Making, Vol 11, Issue 1, pp. 23 - 41. Copyright © 2016 (Human Factors and Ergonomics Society). Reprinted by permission of SAGE Publications.

Digital Object Identifier (DOI)https://doi.org/10.1177/1555343416672782
Permalink -

https://repository.mdx.ac.uk/item/870wy

  • 9
    total views
  • 0
    total downloads
  • 2
    views this month
  • 0
    downloads this month

Export as