Zombse

The Zombie Stack Exchanges That Just Won't Die

View the Project on GitHub anjackson/zombse

What's the current state of the art for OCR for handwriting, especially for historical documents?

Just a few years ago, it seemed handwriting recognition for historic documents was way out of reach. I've seen various news and informal reports recently that suggest to me that great strides have been made in this area of late.

Jenn Riley

Comments

Answer by Trevor Owens

Nowhere and it is likely not to go anywhere for the foreseeable future.

As far as studies go, I think the best work in this area remains the machine learning work done for the post office. The postal service has some tools in place to read handwritten addresses. This is a much smaller problem space, as addresses appear in a rather structured format. Working with handwriting, in particularly with anyone's handwriting, and with historical handwriting and scripts is really really hard. Our ability to read handwritten text piggy backs on our very sophisticated capabilities for facial recognition which is some of the most computationally complex things humans have going for them. (For info on that, see Reading in the Brain.

In short, as far as I understand, OCR of handwritten text is still an active problem for computer science research.

For more info you might check out the resources on the Handwriting Recognition Wikipedia page.

Comments

Answer by Simon Spero

The main conference/workshop series addressing this issue is the International Conference on Frontiers in Handwriting Recognition (ICFHR). This was previously known as the International Workshop on Frontiers in Handwriting Recognition (IWFHR).

There are some improving results on handwriting recognition for historical documents; moving into the mid 90s in some cases.

An example of recent work on High German Mediaeval documents i HisDoc: HisDoc: Historical Document Analysis, Recognition, and Retrieval (Presented at DH2012)\ Project Home Page

Comments

Answer by Joe

I admit, I'm not an expert on the topic, but the Zooniverse project has two crowd sourcing projects : Old Weather is extracting information out of British ship logs, while Ancient Lives is transcrbing Greek papyrus scraps.

I also remember seeing a poster talking about reading historical scientist's journals, but I can't find it. I think it was recently; I thought it was ASIS&T 2011, but it might've been IA Summit 2012 (which doesn't have a list of posters that I can find). I want to say it was exploration logs (similar in vein to Louis & Clark), but I can't remember if they were just classifying what was in it vs. actively trying to transcribe the writing. It might have been the Junius Henderson notebooks, but to my knowledge, that was annotating the transcription and not working from images of the originals.

In trying to find that poster and/or project, I came across a list by Ben Brumfield of crowd sourcing transcription projects in early 2011, and his blog seems to be a great resource on the topic in general, but of note is that list is not all were successful:

According to the New York Times article, there was an attempt to crowdsource the Papers of Abraham Lincoln. The article quotes project director Daniel Stowell explaining that nonacademic transcribers "produced so many errors and gaps in the papers that 'we were spending more time and money correcting them as creating them from scratch.'" The prototype transcription tool (created by NCSA at UIUC) has been abandoned.

... I would assume that it would be possible to use the crowd source efforts to train an automated parser (train it on one page, then have it read the rest of the handwriting from that person), so that you can get the most productive use out of the volunteers ... but I don't know if anyone's doing it. (or if it works as well in practice as in theory; I remember seeing an example from a war journal where the handwriting abrubtly changes due to an injury)

Comments

Answer by Joan Andreu Sánchez

For handwritten text recognition of historical documents, the existing Optical Character Recognition (OCR) systems are not suitable. This is due to the fact that in images of handwritten text, characters cannot be isolated automatically in a reliable way. Adequately tackling the problem of transcribing this type of text images requires holistic approaches often referred to as “segmentation free off-line HTR”. In these approaches all text elements (sentences, words and characters) need to be recognized ashole, without any prior segmentation of the image into these elements. Recent technology for HTR borrows concepts and methods from the field of Automatic Speech Recognition. For the application of this technology to handwritten document images, document image processing steps are needed, such as layout analysis and text line extraction. Given the large amount of data, it is necessary to automate these steps as much as possible. This is a challenge because, so far, standard rules for handwritten documents have been lacking. Once the text line images are available, the HTR models can be fully automatically obtained using powerful, well known training techniques, which only require the (whole) transcripts of a reasonably small number of these (unsegmented) line images.

Despite recent significant improvements, currently available HTR technologies are still far from offering fully automated solutions for transcription.

If you interested in knowing more about HTR for transcription of historical documents you can visit the transcriptorium project web page, and the HTR web page of the PRHLT research group

Comments