CLEF: Ongoing Activities and Plans for the Future Maristella Agosti Giorgio Di Nunzio Nicola Ferro Carol Peters University of Padua IS TI-CNR, Area di Ricerca Italy Italy {agosti, dinunzio, ferro}@dei.unipd.it carol.peters@isti.cnr.it The NTCIR Workshop 6 Meeting – – Tokyo, May 15- - 18, 2007 18, 2007 The NTCIR Workshop 6 Meeting Tokyo, May 15
CLEF Objectives CLEF Objectives � Promote research and stimulate development of multilingual � Promote research and stimulate development of multilingual IR systems for European (and non- -European) languages, European) languages, IR systems for European (and non through through � Creation of evaluation infrastructure and organisation of regula � Creation of evaluation infrastructure and organisation of regular r evaluation campaigns for system testing evaluation campaigns for system testing � Building of an MLIA/ CLIR research community � Building of an MLIA/ CLIR research community � Construction of publicly available test � Construction of publicly available test - -suites suites � Major Goal � Major Goal � To anticipate the emerging needs of the R&D community, and � To anticipate the emerging needs of the R&D community, and � Encourage development of truly multilingual, multimodal systems � Encourage development of truly multilingual, multimodal systems 2 2 The NTCIR Workshop 6 Meeting The NTCIR Workshop 6 Meeting M. Agosti M. Agosti, G.M. Di Nunzio, N. Ferro, C. Peters , G.M. Di Nunzio, N. Ferro, C. Peters Tokyo, May 15- Tokyo, May 15 -18, 2007 18, 2007
CLEF History History CLEF � 1997 � 1997 – – First CLIR system evaluation campaigns in US First CLIR system evaluation campaigns in US and Japan: Text and Japan: REtrieval Conference (TREC) and NTCIR TREC) and NTCIR � CLEF actually began life in 1997 as a track for Cross Language Information Retrieval (CLIR) within TREC. Mainly, English centered tasks (EN -> X, X -> EN). � 2000 � 2000 – – CLIR evaluation in Europe: CLEF (extension of CLIR track at TRE CLIR evaluation in Europe: CLEF (extension of CLIR track at TREC) C) � Fully multilingual, multimodal information retrieval systems � Fully multilingual, multimodal information retrieval systems � capable of processing a query in any medium and any language � capable of processing a query in any medium and any language � finding relevant information from a multilingual multimedia coll � finding relevant information from a multilingual multimedia collection containing ection containing documents in any language and form, documents in any language and form, � and presenting it in the style most likely to be useful to the u � and presenting it in the style most likely to be useful to the user ser � The European Network of Excellence on Digital Libraries (DELOS � The European Network of Excellence on Digital Libraries (DELOS ) has has ) supported CLEF since 2000 (however, this support will end this year 2007) ear 2007) supported CLEF since 2000 (however, this support will end this y � DELOS � DELOS under FP5 2000 – – 2003 ( 2003 (http:/ / delos http:/ / delos- -noe.isti.cnr.it noe.isti.cnr.it / ) / ) under FP5 2000 � DELOS � DELOS under FP6 2004 – – 2007 (http:/ / 2007 (http:/ / www.delos.info www.delos.info/ ) / ) under FP6 2004 3 3 The NTCIR Workshop 6 Meeting The NTCIR Workshop 6 Meeting M. Agosti M. Agosti, G.M. Di Nunzio, N. Ferro, C. Peters , G.M. Di Nunzio, N. Ferro, C. Peters Tokyo, May 15- Tokyo, May 15 -18, 2007 18, 2007
CLEF Technical Infrastructure History CLEF Technical Infrastructure History � 1997 � 1997 – – 1999: CLEF as CLIR Track at TREC 1999: CLEF as CLIR Track at TREC � same infrastructure developed at NIS � same infrastructure developed at NIS T T � 2000 � 2000 – – 2004: CLEF as CLIR stand 2004: CLEF as CLIR stand- -alone European initiative alone European initiative � Martin � Martin Braschler Braschler developed and managed the CLEF infrastructure, developed and managed the CLEF infrastructure, starting from the TREC one starting from the TREC one � 2005 � 2005 – – 2007: CLEF as CLIR stand 2007: CLEF as CLIR stand- -alone European initiative alone European initiative � University of Padua designed and developed a new infrastructure, � University of Padua designed and developed a new infrastructure, applying concepts and approaches of the digital library field applying concepts and approaches of the digital library field 4 4 The NTCIR Workshop 6 Meeting The NTCIR Workshop 6 Meeting M. Agosti M. Agosti, G.M. Di Nunzio, N. Ferro, C. Peters , G.M. Di Nunzio, N. Ferro, C. Peters Tokyo, May 15- Tokyo, May 15 -18, 2007 18, 2007
CLEF Organisation Organisation CLEF � CLEF is mainly a voluntary activity: the following Institutions � CLEF is mainly a voluntary activity: the following Institutions contributed to contributed to the organisation the organisation of the different tracks of the CLEF 2006 campaign: of the different tracks of the CLEF 2006 campaign: � � Information and and Language Language Processing Processing S S ystems, , U. U. Information ystems � � Centre for the for the Evaluation Evaluation of of Human Human Language Language and and Centre Amsterdam, Amsterdam, Netherlands Netherlands Multimodal Multimodal Communication Communication Technologies (CELCT), Technologies (CELCT), � � Trento, Italy Italy IZ Bonn, IZ Bonn, Germany Germany Trento, � � � � Centro per la Ricerca S cientifica e Tecnologica, Inst . Inst . Information Information Technology Technology, , Hyderabad Hyderabad, India , India Centro per la Ricerca S cientifica e Tecnologica, Istituto Trentino di Cultura, Trento, Italy Istituto Trentino di Cultura, Trento, Italy � � LS LS I- I -UNED, Madrid, UNED, Madrid, S S pain pain � � College of Information College of Information S S tudies and tudies and Institute Institute for for � � Linguateca, Linguateca , S S intef , Oslo, intef , Oslo, Norway Norway Advanced Computer Computer S S tudies, , U. U. Maryland, US Maryland, US A Advanced tudies A � � Linguistic Linguistic Modelling Modelling Lab., Lab., Bulgarian Bulgarian Acad Acad S S ci ci � � Dept . of Computer . of Computer S S cience, , U. U. Indonesia Indonesia Dept cience � � NIS NIS T, US T, US A A � � Depts. of Computer . of Computer S S cience & & Medical Medical Informatics Informatics, , Depts cience � � Biomedial Biomedial Informatics Informatics, Oregon , Oregon Health Health and and S S cience cience RWTH Aachen RWTH Aachen U., U., Germany Germany University, US A University, US A � � Dept . of Computer S Dept . of Computer S cience and cience and Information Information S S ystems ystems, , � � Research Computing Cente Cente, , Moscow Moscow S S tate U. U. Research Computing tate U. Limerick Limerick, , Ireland Ireland U. � � � Research Institute for Linguistics, Hungarian Hungarian � Research Institute for Linguistics, Dept . of Computer . of Computer S S cience and and Information Information Dept cience Academy of Academy of S S ciences ciences Engineering Engineering, , National National U. U. Taiwan Taiwan � � � � S S chool of Computer chool of Computer S S cience and cience and Mathematics Mathematics, , Dept . of Dept . of Information Information Engineering Engineering, , U. U. Padua, Padua, Italy Italy Victoria U., Australia Victoria U., Australia � � Dept Dept . of . of Information Information S S ci, U. ci, U. Hildesheim, Hildesheim, Germany Germany � � S chool of Computing, DCU, of Computing, DCU, Ireland Ireland S chool � � Dept Dept . of . of Information Information S S tudies tudies, , U. U. S S heffield, UK heffield, UK � � UC Data Archive Archive and and S S chool of of Information Information UC Data chool � � Evaluations and Language Resources Evaluations and Language Resources Distribution Distribution Management and Management and S S ystems ystems, UC , UC Berkeley Berkeley, US , US A A Agency S S arl, Paris, , Paris, France France Agency arl � � University "Alexandru University " Alexandru Ioan Ioan Cuza Cuza", IAS ", IAS I, Romania I, Romania � � German Research Research Centre Centre for for Artificial Artificial Intelligence, Intelligence, German � � aarbr ü ü cken U. U. Hospitals Hospitals and U and U.of .of Geneva Geneva, , S S witzerland witzerland DFKI, DFKI, S S aarbr cken, , Germany Germany 5 5 The NTCIR Workshop 6 Meeting The NTCIR Workshop 6 Meeting M. Agosti M. Agosti, G.M. Di Nunzio, N. Ferro, C. Peters , G.M. Di Nunzio, N. Ferro, C. Peters Tokyo, May 15- Tokyo, May 15 -18, 2007 18, 2007
Recommend
More recommend
