Automatic alphabet recognition

Maayan Geffet, Yair Wiseman, Dror Feitelson

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

The last step of the Information Retrieval process is to display the found documents to the user. However, some difficulties might occur at that point. English texts are usually written in the ASCII standard. Unlike the English language, many languages have different character sets, and do not have one standard. This plurality of standards causes problems, especially in a web environment, where one may download a document with an unknown standard. This paper suggests a purely automatic way of finding the standard which was used by the document writer based on the statistical letters distribution in the language. We developed a vector-space-based method that creates frequencies vectors for each letter of the language and then matches a new document's vectors to the pre-computed templates. The algorithm was applied on various types of corpora in Hebrew, Russian and English, and provides an efficient solution to the stated problem in most cases.

Original languageAmerican English
Pages (from-to)25-40
Number of pages16
JournalInformation Retrieval
Volume8
Issue number1
DOIs
StatePublished - 2005

Keywords

  • Characters set
  • Letters' mapping
  • Natural language alphabet

Fingerprint

Dive into the research topics of 'Automatic alphabet recognition'. Together they form a unique fingerprint.

Cite this