You are here

Automatic Semantic Subject Indexing of Web Documents in Highly Inflected Languages

TitleAutomatic Semantic Subject Indexing of Web Documents in Highly Inflected Languages
Publication TypeBook Chapter
Year of Publication2011
AuthorsSinkkilä, R, Suominen, O, Hyvönen, E
EditorAntoniou, G, Grobelnik, M, Simperl, E, Parsia, B, Plexousakis, D, De Leenheer, P, Pan, J
Book TitleThe Semantic Web: Research and Applications
Series TitleLecture Notes in Computer Science
CityBerlin / Heidelberg

Structured semantic metadata about unstructured web documents can be created using automatic subject indexing methods, avoiding laborious manual indexing. A succesful automatic subject indexing tool for the web should work with texts in multiple languages and be independent of the domain of discourse of the documents and controlled vocabularies. However, analyzing text written in a highly inflected language requires word form normalization that goes beyond rule-based stemming algorithms. We have tested the state-of-the art automatic indexing tool Maui on Finnish texts using three stemming and lemmatization algorithms and tested it with documents and vocabularies of different domains. Both of the lemmatization algorithms we tested performed significantly better than a rule-based stemmer, and the subject indexing quality was found to be comparable to that of human indexers.