Characterizing and Evaluating Whole Session Interactive Information Retrieval

Investigators: Prof. Nick Belkin and Prof. Chirag Shah

Sponsored by the National Science Foundation (NSF) Award 1423239
About the Project
This research addresses a newly important issue in contemporary life. As people become more accustomed to using the Web for finding information, they are increasingly using it for addressing ever more complex and personally important information problems. However, current Web search engines have been developed and specifically tuned to helping people find simple, mostly factual information, usually as a single response list to a single, simple query. But when they try to address the new types of problems, people need to engage in longer information seeking episodes than the one query-one response paradigm assumes. They may also need to engage in many activities other than just clicking on a search result, such as reading, evaluating, comparing and using information. Current Web search engines do not sufficiently support this model of information seeking and use. This research addresses this problem by studying why people engage in such complex information seeking (that is, the reasons that motivate them to do this), and what they try to accomplish during the course of an information seeking episode (their search intentions). The end-goal of this research is to design and evaluate new types of search engines for supporting people in accomplishing the goals that have led them to engage in information seeking. This means, in essence, being able to personalize system support to the individual, and the individual's goals and context. Specifically, this research will establish relationships between people's behaviors during an information seeking episode, the motivating goals that led them to engage in information seeking, and their specific intentions at any point in an information seeking episode. This will enable development of systems that will be able to predict how best to support the individual person in addressing their information problem. For example, the findings from this project could help build a system that automatically identifies that a searcher is shopping for a car, and help him/her compare cost-benefits of new vs. used cars, buying vs. leasing, and eventually making an informed decision. Research will be integrated with educational activities via developing modules to supplement courses in iSchools and library/information science programs, etc. This is important, since a broad range of students would learn about new methods of searching and related user studies and evaluation.

Progress and Planned Activities
Summer 2015With newly appointed graduate assistant Matt Mitsui, we are working on designing a new user study to understand people's intentions for different search segments.
Fall 2015Lab study to extract search intentions.
Submitted a short paper to SIGIR 2016.
Spring 2016Continuation of the lab study to extract search intentions.
Started discussions with GESIS in Cologne, Germany for possible partnership.
Got SIGIR 2016 short paper accepted.
Summer 2016Continuation of the lab study to extract search intentions.
Fall 2016Continuation of the lab study to extract search intentions.
Spring 2017Continuation of the lab study to extract search intentions.
Got SIGIR 2017 short paper accepted.
Summer 2017Designing of a field study to extract search intentions.
Got ASIST 2017 full paper accepted.
Fall 2017IRB application and pilots for the in situ study. A paper presented at ASIST 2017 conference.
Spring 2018In situ study commenced. A paper presented at ACM CHIIR 2018 conference.
Summer 2018In situ study continued.
Fall 2018Design and piloting of a new lab study.
Spring 2019Running a lab study.

Publications
  1. Mitsui, M., Shah, C. & Belkin, N.J. (2016). Extracting Information Seeking Intentions for Web Search Sessions. In: Proceedings of the 2016 ACM SIGIR International Conference on Research and Development in Information Retrieval (SIGIR 2016). New York: ACM. [PDF]
  2. Rha, E.Y., Mitsui, M., Belkin, N.J., Shah, C. (2016) Exploring the Relationships Between Search Intentions and Query Reformulations. In: Proceedings of the 78th Annual Meeting of the Association for Information Science and Technology, Copenhagen, Denmark, October 2016. [PDF]
  3. Mitsui, M., Liu, J., Belkin, N., & Shah, C. (2017). Predicting Information Seeking Intentions from Search Behaviors. In Proceedings of ACM SIGIR 2016 Conference. 4 pp. August 7-11, 2017. Tokyo, Japan.
  4. Belkin, N., Hienert, D., Philipp, M., & Shah, C. (2017). Data Requirements for Evaluation of Personalization of Information Retrieval. A Position Paper. Personalized Search Evaluation Lab at CLEF 2017. September 11-14, 2017. Dublin, Ireland.
  5. Rha, E. Y., Shi, W., & Belkin, N. J. (2017). An exploration of reasons for query reformulations. Proceedings of the Association for Information Science and Technology, 54(1), 337-346.
  6. Heinert, D., Mitsui, M., Mayr, P., Shah, C., & Belkin, N. (2018). The Role of the Task Topic in Web Search of Different Task Types. In Proceedings of ACM Conference on Human Information Interaction and Retrieval (CHIIR). March 11-15, 2018. New Brunswick, NJ.
  7. Liu, J., Mitsui, M., Belkin, N., & Shah, C. (2019). Task, Information Seeking Intentions, and User Behavior: Toward A Multi-level Understanding of Web Search. Proceedings of ACM Conference on Human Information Interaction and Retrieval (CHIIR). March 10-14, 2019. Glasgow, Scotland, UK.
  8. Hienert, D., Kern, D., Mitsui, M., Shah, C., & Belkin, N. (2019). Reading Protocol: Understanding what has been read in Interactive Information Retrieval Tasks. Proceedings of ACM Conference on Human Information Interaction and Retrieval (CHIIR). March 10-14, 2019. Glasgow, Scotland, UK.

Presentations
  1. Belkin, N. (2015). Usefulness as the Criterion of Evaluation of Interactive Information Retrieval. GESIS, The Leibniz Institute for the Social Sciences, Cologne, Germany. June, 2015. [PDF]
  2. Shah, C. (2016). Social and Collaborative Information Seeking: Bringing Synergy in Search. GESIS, The Leibniz Institute for the Social Sciences, Cologne, Germany. June, 2016.
  3. Shah, C. (2016). Information Fostering: Proactively Complementing Information Seeking. GESIS, The Leibniz Institute for the Social Sciences, Cologne, Germany. June, 2016.
  4. Belkin, N. (2017) Goals, Tasks, and Information Retrieval. Keynote Address, 2017 Annual Conference of the Patent Information User Group, Atlanta, GA, May 22, 2017. [PDF]
  5. Mitsui, M. (2017). Extracting Information Seeking Intentions from Web Search Sessions. Radboud University Nijmegen, Cologne, Netherlands. October, 2016.
  6. Belkin, N. (2017). Exploring Information Seeking and Searching Intentions: An Overview of Recent Research at Rutgers University. Tsinghua, Peking and Nankai Universities, China. December, 2017. [PDF]
  7. Belkin, N. (2018). The Potential for Personalizing Search According to Interactive Search Intentions. Keynote presentation at: Beyond the Simple Search Box? Mini-Symposium on Process Support in Information Seeking, University of Amsterdam, The Netherlands, April 18, 2018. [PDF]
  8. Belkin, N. (2018). Evaluation of Personalization of Information Interaction in an Era of Information Ubiquity. CLEF 2018. Avignon, France. [PDF]
  9. Shah, C. (2019). Going from 'What' and 'How' to 'Who' and 'Why': Task Analysis in Online and Physical Contexts. University College London, UK. March 15, 2019.

Tools
Browser plug-ins
Data collection tools