A study of different representation conventions during investigatory sensemaking

Background: During the process of conducting investigations, users structure information externally to help them make sense of what they know, and what they need to know. Software-based visual representations may be a natural place for doing this, but there are a number of types of information structuring that might be supported and hence designed for. Further, there might be important differences in how well different representational conventions support sensemaking. There are questions about what type of representational support might allow these users to be more effective when interacting with information. Aim: To explore the impact that different types of external representational structuring have on performance and user experience during intelligence type investigations. Intelligence analysis represents a difficult example domain were sensemaking is needed. We have a particular interest in the role that timeline representations might play given evidence that people are naturally predisposed to make sense of complex social scenarios by constructing narratives. From this we attempt to quantify possible benefits of timeline representation during investigatory sensemaking, compared with argumentation representation. Method: Participants performed a small investigation using the IEEE 2011 VAST challenge dataset in which they structured information either as a timeline, an argumentation or as they wished (freeform). 30 participants took part in the study. The study used three levels of a between participants independent variable of representation type. The dependent variables were performance (in terms of recall, precision efficiency and understanding) and user experience (in terms of cognitive load, engagement and confidence in understanding). Result: The result shows that the freeform condition experienced a lower cognitive load than the other two: timeline and argument respectively. A post hoc exploratory analysis was conducted to better understand the information behaviour and structuring activities across conditions and to better understand the types of structuring that participants perform in the freeform condition. The analysis resulted in an Embedded Representational Structuring Theory (ERST) that helps to characterise and describe representations primarily in terms of their elements and their relations. Conclusion: The results suggest that: (a) people experienced lower cognitive load when they are free to structure information as they wish, (b) during their investigations, they create complex heterogeneous representations consisting of various entities and multiple relation types and (c) their structuring activities can be described by a finite set of structuring conventions.