seminar
An International Evaluation Platform for Information Access Technologies
This talk outlines the activities and progress of NTCIR (NII Testbeds and Communities in Information access Research, http://research.nii.ac.jp/ntcir/), an international evaluation of information access technologies such as information retrieval (IR), question answering and cross-lingual information access. It has used the datasets in East Asian languages including Chinese, Korean, Japanese and English, but has attracted participation from all over the world. The project started from late 1997 and has run in 18-month cycles. More than 150 groups from all over the world registered to each cycle and in total more than 1000 groups participated in NTCIR. More than 2500 groups have used NTCIR test collections for their research. Evaluation methodologies and test collections in NTCIR constitute a fundamental infrastructure for information access technology research, through which many new research challenges have been produced and enhanced internationally.
The three major international IR evaluation platforms, TREC, CLEF, and NTCIR have worked collaboratively and each has played unique roles to contribute to deepen and diversify the research. For example, NTCIR initiated the Cross-Language Information access across Asian language and English in specialized domains such as patents, mathematics and entrance examination. New challenging tasks include interactive question answering, extracting and visualizing trends from text, module-based evaluation, bridging between machine translation and retrieval, spoken query and documents. This seminar will talk about evaluation methodologies for these tasks and discuss future directions.
<p>Noriko Kando is a professor in the Information-society Research Division of the National</p>
<p>Institute of Informatics (NII), Tokyo, Japan, and has been co-appointed as a professor in the Department of Informatics at the Graduate University of Advanced Studies, Japan. She initiated NTCIR, an evaluation of information-access technologies, using East Asian languages and English documents, and has been leading various projects including ones for exploratory-search systems. She received her Ph.D. from Keio University in Library and Information Science in 1995 and has been teaching and conducting research at NII since 1994. She has had more than 100 of her refereed scientific articles published in journals and by international conferences, and has been an invited speaker at many international conferences and workshops. Recently she has been involved in a grand-challenge called “Todai Robot Project”, which aiming to build a computer system enable to pass the university entrance exams as a base QA-system part leader. She was an Asian representative for the ACM-SIGIR Executive Committee from 2003 to 2007, and a Program Co-Chair of ACM-SIGIR 2007. She is an Associate Editor of ACM Transaction for Asian Language Information Processing from 2000 to 2007 and for Information Processing and Management (Elsevier) from 2007 to 2013.</p>