Content Extraction Studies for Multilingual Unstructured Web Documents

Abstract

With increased usage of web and mobile communication, language, form and content of present day web documents have tended to become more and more complex with inclusion of different kinds of media files and multi-lingual texts. The mnemonics way of mapping consonants for digits, adding vowels suitably so that words could be formed is a multilingual generator and if properly done can generate universal language. Thus, in Telugu language all words end with vowels, in Tamil language each basis character is spelt same, irrespective of its joints. Extracting content and information to reach the web surfer in an easy way, has become important and with this as basis, a generic method is developed from basic pixel level so that applicability can be effective. Converting pixel matrix into selected attributes and later a modified form of pattern matching is used to assess the content. The development, being generic, results are given for character and word level performance, with four different languages used in the data.