Bibliography: From Ultra-light to Naturalistic IR Interaction. By - - PDF document

Bibliography: From ’Ultra-light’ to Naturalistic IR Interaction.

By Peter Ingwersen Royal School of LIS Birketinget 6, DK 2300 Copenhagen S – Denmark pi@db.dk

Methodological Issues – Research Design

Baeza-Yates, R. & Ribeiro, R. (1999), Eds. Modern Information Retrieval. Wokingham, UK: Addison-Wesley. Belew, R.K. (2000). Finding Out About: A Cognitive Perspective on Search Engine Technology and the WWW. Cambridge, UK: Cambridge University Press. Borlund, P. (2003b): The IIR evaluation model: A framework for evaluation of interactive information retrieval systems. Information Research, 8(3), paper no. 152. [http://informationr.net/ir/8-3/paper152.html] Ericsson, K. A. & Simon, H. A. (1996). Protocol analysis: Verbal reports as data Cambridge, MA: The MIT Press. (2nd Rev.ed.) Fidel, R. (1993). Qualitative methods in information retrieval research. Library and Information Science Research, 15(3): 219-247. Fidel, R. & Soergel, D. (1983). Factors affecting online bibliographic retrieval: A conceptual framework for research. Journal of the American Society for Information Science, 34(3): 163-180. Frankfort-Nachmias, C. & Nachmias, D. (2000). Research Methods in the Social Sciences. 6th ed. New York: Worth Publishers. xxi, 550 p Hawking, D., Voorhees, E., Craswell, N. & Bailey, P. (2000). Overview of TREC-8 Web Track. In: Voorhees, E.M. & Harman, D.K. (Eds.) TREC 8 Proc.: 131-150. http://trec.nist.gov/pubs/ . Hersh, W., Pentecost, J. & Hickam, D. (1996). A task oriented approach to information retrieval

• evaluation. JASIS, 47(1): 50-56.

Ingwersen, P. & Järvelin, K. (2005) The Turn: Integration of Information Seeking and Retrieval in

• Context. Berlin, Germany: Springer.

Ingwersen, P. & Järvelin, K. (2007). On the holistic cognitive theory for information retrieval Drifting Outside the Cave of the Laboratory Framework. In: Dominich, S & Kisss, F. (eds.): Studies in Theory of Information Retrieval. Budapest: Foundation for Information Society:135-147. Ingwersen, P. & Willett, P. (1995). An introduction to algorithmic and cognitive approaches for information retrieval. Libri, 45: 160-177. Järvelin, K. (2007). An analysis of two approaches in Information Retrieval: From frameworks to study designs. Journal of the American Society for Information Science and Technology, 58(7): 971–986. Kekäläinen, J. & Järvelin, K. (2002). User-oriented evaluation methods for information retrieval: A case study based on conceptual models for query expansion. In: Lakemeyer, G. & Nebel, B. (Eds.) Exploring Artificial Intelligence in the New Millennium. San Francisco: Morgan Kaufmann Publishers: 355-379. Saracevic, T., Kantor, P., Chamis, A. & Trivison, D. (1988). A study of information seeking and

• retrieving. I. Background and methodology. JASIS, 39(3): 161-176.

Tague, J. & Schultz, R. (1988). Some measures and procedures for evaluation of the userinterface in an information retrieval system. ACM SIGIR 1988 Proc.: 371-385. Teevan, J., Alvarado, C., Ackerman, M. S. & Karger, D. R. (2004). The perfect search engine is not enough: A study of orienteering behavior in directed search. ACM-CHI 2004: 415-422.

Relevance & Performance Measure Issues in Interactive IR

Barry, C. & Schamber, L. (1998). User’s criteria for relevance evaluation: A cross situational

• comparison. Information Processing & Management, 34(2/3): 219-236.

Borlund, P. (2003a). The concept of relevance in IR. JASIST, 54(10): 913-925. Cosijn, E. & Ingwersen, P. (2000). Dimensions of relevance. Information Processing & Management, 36: 533-550. Czerwinski, M., Horvitz, E. & Cutrell, E. (2001). Subjective duration assessment: An implicit probe for software usability. IHM-HCI 2001 Conf. Proc.: 167-170. Frokjaer, E., Hertzum, M. & Hormbaek, K. (2000). Measuring usability: Are effectiveness, efficiency, and satisfaction really correlated? ACM CHI 2000: 345-352. Järvelin, K. & Kekäläinen, J. (2000). IR evaluation methods for highly relevant documents. In: Belkin, N.J., Ingwersen, P. & Leong, M.-K. (Eds.), ACM SIGIR 2000 Proc.: 41-48. Järvelin, K. & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (ACM TOIS), 20(4): 422-446.

Käki, M. (2004). Proportional search interface usability measures.NordiCHI 2004 Proc.: 365-372.

Kekäläinen, J. (2005). Binary and graded relevance in IR evaluations - Comparison of the effects on ranking of IR systems. Information Processing & Management, 41(5): 1019-1033. Kekäläinen, J. & Järvelin, K. (2002). Using graded relevance assessments in IR evaluation. Journal

• f the American Society for Information Science and Technology, 53(13): 1120-1129.

Larsen, B., Malik, S. & Tombros, A. (2006). The interactive track at INEX2005. In: Fuhr, N., Lalmas, M., Malik, S. and Kazai, G. (Eds.), INEX 2005. Berlin: Springer: 398-410. Lun, A. M. (2001). Measuring usability with the USE Questionnaire. STC Usability SIG Newsletter, 8(2), http://www.stcsig.org/usability/newsletter/0110_measuring_with_use.html (Checked June 15, 08). Nielsen, J. (2003). Usability 101: Introduction to usability. http://www.useit.com/alertbox/20030825.html Checked June 15, 08). Salojärvi, J., Kojo, I., Simola J. & Kaski, S. (2003). Can relevance be inferred from eye movements in information retrieval? In: Proc. of WSOM’03, Workshop on Self-Org. Maps. Kyushu Institute

• f Technology, Kitakyushu, Japan: 261-266.

Spink, A., Greisdorf, H. & Bateman, J. (1998). From highly relevant to not relevant: Examining different regions of relevance. Information Processing & Management, 34(5): 599-621. Vakkari, P. & Sormunen, E. (2004). The influence of relevance levels on the effectiveness of interactive information retrieval. JASIST, 55(11): 963-969. Voorhees, E.M. (1998). Variations in relevance judgments and the measurement of retrieval

• effectiveness. In: ACM-SIGIR1998 Proc.: 315-323.

Laboratory Experiments – IR Simulations

Croft, W.B. & Thomson, R.H. (1987). I3R: A new approach to the design of document retrieval

• systems. Journal of the American Society for Information Science, 38(6): 389-404.

Dennis, S., Bruza, P. & McArthur, R. (2001). Web searching: A progress-oriented experimental study of three interactive search paradigms. JASIST, 53(2): 120-133. Magennis, M. & van Rijsbergen, C.J. (1997). The potential and actual effectiveness of interactive query expansion. In: Belkin, N. J., Narasimhalu, A. D. & Willett, P. (Eds.), ACM SIGIR 1997 Proc.: 324-332. White, R.W. (2006). Using searcher simulations to redesign a polyrepresentative implicit feedback

• interface. Information Processing & Management, vol. 42(5): 1185-1202.

White, R.W., Ruthven, I. & Jose, J.M. & van Rijsbergen, C.J. (2005). Evaluating implicit feedback models using searcher simulations. ACM Transactions on Information Systems, 23(3): 325-361.

IR Interaction ‘Ultra-Light’

Belkin, N.J., Oddy, R.N. & Brooks, H. (1982). ASK for information retrieval: Part 2. Journal of Documentation, 38(3): 145-164. Harper, D. J., Koychev, I., Sun, Y. & Pirie, I. (2004). Within-Document retrieval: A user-centred evaluation of relevance profiling. Information Retrieval, 7: 265-290. Hsieh-Yee, I. (1998). Search tactics of Web users in searching for texts, graphics, known items and subjects: a search simulation study. Reference Librarian, 60: 61-85. Kelly, D., Dollu, V.D. & Xin Fu (2005). The loquacious user: A document-independent source of terms for query expansion. In: ACM SIGIR2005 Proc.: 457-464. Kelly, D. & Xin Fu (2007). Eliciting better information need descriptions from users of information search systems Information Processing & Management, 43(1): 30-46. Papaeconomou, P. Zijlema, A.F. & Ingwersen, P. (2008). Searchers’ relevance judgments and criteria in evaluating Web pages in a learning style perspective. In: Borlund, P. et al. (eds.), Proceedings of the Information Interaction in Context Conference (IIiX) 2008. London. NY: ACM Press, ACM International Conference Proceeding Series; Vol. 348: 123-132. White, R.W., Jose, J.M. & Ruthven, I. (2006). An implicit feedback approach for interactive information retrieval. Information Processing & Management, 42(1): 166-190. White, R.W., Ruthven, I. & Jose, J.M. (2005a). A study of factors affecting the utility of implicit relevance feedback. In: ACM SIGIR 2005 Proc.: 57-64.

IR Interaction ‘Light’

Beaulieu, M. (1997). Experiments with interfaces to support query expansion. Journal of Documentation, 53(1): 8-19. Beaulieu, M. (2000). Interaction in information searching and retrieval. Journal of Documentation, 56(4): 431-439. Beaulieu, M. & Jones, S. (1998). Interactive searching and interface issues in the Okapi best match probabilistic retrieval system. Interacting with Computers, 10(3): 237-248. Belkin, N.J., Cool, C., Croft, W.B. & Callan, J.P. (1993). The effect of multiple query representations on information retrieval performance. In: Korfhage, R., Rasmussen, E. & Willet,

• P. (Eds.) ACM SIGIR 1993 Proc.: 339-346.

Belkin, N.J., Cool, C., Koenemann, J., Ng, W.B. & Park, S. (1996). Using relevance feedback and ranking in interactive searching. In: Harman, D. (Ed.) TREC 4 Proc.: 181-210. Belkin, N.J., Cool, C., Koenemann, J., Park, S. & Ng, W.B. (1996). Information seeking behaviour in new searching environments. In: Ingwersen, P. & Pors, N.O. (Eds.), CoLIS 2 Conf. Proc.: 403- 416. Belkin, N. J., Cool, C., Kelly, D., Lee, H.-J., Muresan, G., Tang, M.-C. & Yuan, X.-J. (2003). Query length in interactive information retrieval. ACM SIGIR 2003 Proc.: 205-212. Bilal, D. (2000). Children’s use of the Yahooligans! Web search engine: I. Cognitive, physical and affective behaviors on fact-based search tasks. Journal of the American Society for Information Science, 51(7): 646-665. Borlund, P. 2000. Experimental components for the evaluation of interactive information retrieval

• systems. Journal of Documentation, 56 (1): 71-90. (Also Research Design).

Byström, K. & Järvelin K. 1995. Task complexity affects information seeking and use. Inf. Proc.& Man., 31(2): 191-213. Borlund, P. & Ingwersen, P. (1998). Measures of relative relevance and ranked half-life: Performance indicators for interactive IR. In: ACM SIGIR 1998 Proc.: 324-331. Campbell, I. (2000). Interactive evaluation of the ostensive model using a new test collection of images with multiple relevance assessments. Information Retrieval, 2(1): 87-114.

Cothey, V. (2002). A longitudinal study of World Wide Web users’ information-searching

• behavior. JASIST, 53(2): 67-78.

Efthimiadis, E.N. (1993). A user-centered evaluation of ranking algorithms for interactive query

• expansion. In: ACM SIGIR 1993 Proc.: 146-156.

Joachims, T., Granka, L., Pan, B., Hembrooke, H. & Gay, G. (2005). Accurately interpreting click- through data as implicit feedback. In: ACM-SIGIR 2005 Proc.: 154-161. Koenemann, J. & Belkin, N. J. (1996). A case for interaction: A study of interactive information retrieval behavior and effectiveness. SIGCHI 1996 Conference Proc.: 205-212. Maglaughlin, K.L. & Sonnenwald, D.H. (2002). User perspectives on relevance criteria: A comparison among relevant, partially relevant, and not-relevant judgments. Journal of the American Society for Information Science and Technology, 53(5): 327-342. Palmquist, R.A. & Kim, K.S. (2000). Cognitive style and on line database search experience as predictors of web search performance. JASIST, 51(6): 558-566. Petrelli, D., Beaulieu, M., Sanderson, M. Demetriou, G., Herring, P. & Hansen, P. (2004). Observing users, designing clarity: A case study on the user-centered design of a cross-language information retrieval system. JASIST, 55(10): 923-934. Puolamäki, K., Salojärvi, J. & Savia, E. (2005). Combining eye movements and collaborative filtering for proactive information retrieval. In: ACM-SIGIR 2005 Proc.: 146-153. Ruthven, I., Lalmas, M. & van Rijsbergen, K. (2002). Ranking expansion terms with partial and

• stensive evidence. In: Bruce, H. et al. (Eds.), CoLIS4 Conf. Proc.: 199-220.

Ruthven, I., Lalmas, M. & van Rijsbergen, K. (2003). Incorporating user search behaviour into relevance feedback. JASIST, 54(6): 529-549. Tombros, A., Ruthven, I., Jose, J.M. (2003). Searchers’ criteria for assessing web pages. In: ACM- SIGIR 2003 Proc.: 96-104. White, R., Jose, J. & Ruthven, I. (2003). A task-oriented study of the influencing effects of query- biased summarization in web searching. Information Processing and Management, 39(5): 707- 733.

Naturalistic IR Interaction

Anick, P. (2003). Using terminological feedback for web search refinement: A log based study. In: ACM-SIGIR 2003 Proc.:88-95. Barry, C.L. (1994). User-defined relevance criteria: An exploratory study. Journal of the American Society for Information Science, 45(3): 149-159. Bellotti, V., Ducheneaut, N., Howard, M. & Smith, I. (2003). Taking email to task: The design and evaluation of a task management centered email tool. ACM SIGCHI 2003 Proc.: 345-352. Bilal, D. (2002). Children’s use of the Yahooligans! Web search engine: III. Cognitive and physical behaviors on fully self-generated search tasks. Journal of the American Society for Information Science and Technology, 53(13): 1170-1183. Dumais, S., Cutrell, E., Cadiz, J. J., Jancke, G., Sarin, R. & Robbins, D. C. (2003). Stuff I’ve Seen: A system for personal information retrieval and re-use. ACM SIGIR 2003: 72-79. Fidel, R. et al. (1999). A visit to the information mall: Web searching behavior of high school

• students. JASIS, 50(1): 24-37.

Ford, N., Miller, D. & Moss, N. (2001). The role of individual differences in Internet searching: An empirical study. JASIST, 52(12): 1049-1066. Hirsh, S.G. (1999). Children’s relevance criteria and information-seeking on electronic resources. JASIS, 50(14): 1265-1283. Kelly, D. & Belkin, N. J. (2004). Display time as implicit feedback: Understanding task effects. ACM SIGIR 2004 Proc.: 377-384. (Longitudinal study).

Nielsen, M L (2001). A framework for work task based thesaurus design. Journal of Documentation, 57 (6), 774-797. Nielsen, M L (2004). Task-based evaluation of associative thesaurus in real-life environment. Proceedings of the ASIST 2004 Annual Meeting: 437-447. Saracevic, T. & Kantor, P. (1988). A study of information seeking and retrieving. II. Users, questions, and effectiveness. JASISe, 39(3): 177-196. Saracevic, T. & Kantor, P. (1988). A study of information seeking and retrieving. III. searchers, searches, and overlaps. JASIS, 39(3): 197-216. Vakkari, P. (2001). Changes in search tactics and relevance judgments in preparing a research proposal: A summary of findings of a longitudinal study. Information Retrieval, 4(3/4): 295-310. Vakkari, P. & Hakala, N. (2000). Changes in relevance criteria and problem stages in task

• performance. Journal of Documentation, 56(5): 540-562.

Wang, P. (1997). Users’ information needs at different stages of a research project: A cognitive

• view. In: Vakkari, P. et al. (Ed.) Information Seeking in Context (ISIC 1) Conf. Proc.: 307-318.

Wang, P., Hawk, W.B. & Tenopir, C. (2000). Users’ interaction with World Wide Web resources: an exploratory study using a holistic approach. Information Processing & Management, 36(2): 229-251.