Lecture Notes in Computer Science, 2003, Volume 2633/2003, 545, DOI: 10.1007/3-540-36618-0_14

Question Answering System for Incomplete and Noisy Data
Methods and Measures for Its Evaluation

Lili Aunimo, Oskari Heinonen, Reeta Kuuskoski, Juha Makkonen, Renaud Petit and Otso Virtanen

View Related Documents

Abstract

We present a question answering system that can handle noisy and incomplete natural language data, and methods and measures for the evaluation of question answering systems. Our question answering system is based on the vector space model and linguistic analysis of the natural language data. In the evaluation procedure, we test eight different preprocessing schemes for the data, and come to the conclusion that lemmatization combined with breaking compound words into their constituents gives significantly better results than the baseline. The evaluation process is based on stratified random sampling and bootstrapping. To measure the correctness of an answer, we use partial credits as well as full credits.

Fulltext Preview

Image of the first page of the fulltext document