|Title||Unsupervised Evaluation of Parser Robustness|
|Publication Type||Book Chapter|
|Year of Publication||2005|
|Authors||Bigert, J, Sjöbergh, J, Knutsson, O, Sahlgren, M|
|Book Title||Computational Linguistics and Intelligent Text Processing|
|Series Title||Lecture Notes in Computer Science|
|City||Berlin / Heidelberg|
This article describes an automatic evaluation procedure for NLP system robustness under the strain of noisy and ill-formed input. The procedure requires no manual work or annotated resources. It is language and annotation scheme independent and produces reliable estimates on the robustness of NLP systems. The only requirement is an estimate on the NLP system accuracy. The procedure was applied to five parsers and one part-of-speech tagger on Swedish text. To establish the reliability of the procedure, a comparative evaluation involving annotated resources was carried out on the tagger and three of the parsers.