You are here

Automatic Semantic Subject Indexing of Web Documents in Highly Inflected Languages

TitleAutomatic Semantic Subject Indexing of Web Documents in Highly Inflected Languages
Publication TypeMiscellaneous
Year of Publication2010
AuthorsSinkkilä, R, Suominen, O, Hyvönen, E
Keywordsindexing
Abstract

Structured semantic metadata about unstructured web documents can be created using automatic subject indexing methods, avoiding laborious manual indexing. A succesful automatic subject indexing tool for the web should work with texts in multiple languages and be independent of the domain of discourse of the documents and controlled vocabularies. However, analyzing text written in a highly in ected language requires word form normalization that goes beyond rule-based stemming algorithms. We have tested the state-of-the art automatic indexing tool Maui on Finnish texts using three stemming and lemmatization algorithms and tested it with documents and vocabularies of di erent domains. Both of the lemmatization algorithms we tested performed signi cantly better than a rule-based stemmer, and the subject indexing quality was found to be comparable to that of human indexers.

URLhttp://www.seco.tkk.fi/publications/submitted/sinkkila-et-al-subject-indexing-2010.pdf