The centrality of user modeling to high recall with high precision search

The objective of search is to find documents relevant to a particular user's notion of relevance. However, relevance is often a moving target: imperfectly defined and subject to change as more documents are seen. In this paper, we report on systematic user modeling (UM) and the use of a system-internal agent (proxy) to produce a hybrid human-computer system that achieves extraordinarily high performance on mediated search tasks. We present details of our UM-approach and its four main components: (i) use case (ii) scope (iii) nuance and (iv) linguistic variability. We illustrate how these components provide a framework with which a user and a proxy co-construct a shared representation of information needs and mutual knowledge. This representation serves as the common ground through which external knowledge is shared, mediated, negotiated, synthesized and made accessible to the system. We evaluated the performance of our system on the Legacy Tobacco Documents Library, a corpus of advertising, manufacturing, marketing, sales and scientific research activities of major US tobacco companies. Independently adjudicated results from NIST's 2008 TREC legal track demonstrate that our approach to UM yields high performance on search tasks.

Date of Conference: 11-14 Oct. 2009
Conference Location: San Antonio, TX.