You are here

Unsupervised Evaluation of Parser Robustness

TitleUnsupervised Evaluation of Parser Robustness
Publication TypeBook Chapter
Year of Publication2005
AuthorsBigert J, Sjöbergh J, Knutsson O, Sahlgren M
EditorGelbukh A
Book TitleComputational Linguistics and Intelligent Text Processing
Series TitleLecture Notes in Computer Science
CityBerlin / Heidelberg
ISBN Number978-3-540-24523-0

This article describes an automatic evaluation procedure for NLP system robustness under the strain of noisy and ill-formed input. The procedure requires no manual work or annotated resources. It is language and annotation scheme independent and produces reliable estimates on the robustness of NLP systems. The only requirement is an estimate on the NLP system accuracy. The procedure was applied to five parsers and one part-of-speech tagger on Swedish text. To establish the reliability of the procedure, a comparative evaluation involving annotated resources was carried out on the tagger and three of the parsers.