Showing posts with label artificial intelligence. Show all posts
Showing posts with label artificial intelligence. Show all posts

Saturday, August 22, 2026

Handwriting Recognition - A 2026 status check

Background: For years, I had a habit of writing whenever I traveled. No laptop, no apps, just a pen, a notebook, and pages of cursive observations about airports, flights, fellow travelers, and the emotions that come with being away from home. Two decades later, I decided to digitize those notebooks and make them available on Kindle, eventually publishing them as Transit Tales: Between Airports and Emotions: Travel Blog 20 Years Too Late (View on Amazon). As a first step, I assumed modern LLM-powered handwriting recognition would effortlessly convert the pages into text. Instead, the journey led to an unexpected discovery: handwriting recognition is still far from "solved."

The apparent success of modern handwriting recognition is increasingly a combination of imperfect OCR and powerful language-model reconstruction. What users see is often not what the system read, but what the system inferred.

Handwriting Recognition: Better Than It Looks, But Not For The Reason You Think

a hand written page which has been scanned


Modern handwriting recognition demos can be surprisingly impressive. Feed a handwritten page such as the one below into a state-of-the-art OCR system and the resulting transcript may appear almost perfect at first glance. However, a closer examination reveals that the raw recognition quality is often much poorer than it seems. The magic comes not from character recognition alone, but from the language model sitting behind it.

Consider several examples from this handwritten page. The handwritten phrase corresponding to"airport" may initially be recognized as something closer to"surpat". Similarly,"vehicle" can become"valuele","takes a U-turn" becomes"tales a Utum","X-ray machine" becomes"A-vay machine", and"immigration card" appears as"immerdian card". Taken literally, these outputs are poor transcriptions and would be difficult for a human reader to understand without context.

Large language models change the game. Given the surrounding sentence, the model realizes that a traveler arriving at a terminal is far more likely to write"airport" than"surpat","vehicle" than"valuele", and"X-ray machine" than"A-vay machine". The model effectively performs error correction using context, world knowledge, and statistical expectations about language. In other words, it is not necessarily reading the handwriting correctly; it is often inferring what the author probably meant.

This distinction becomes even more obvious with longer phrases. The raw transcription contains fragments such as"want continues - the grove length increase" which are difficult to interpret. Yet an LLM can confidently reconstruct the phrase as"wait continues - the queue length increases". Likewise,"dot Mupose" can be corrected to"what purpose","quese" to"queue", and"lanton" to"London". These corrections are plausible and often accurate, but they are driven at least as much by language understanding as by handwriting recognition itself.

There is an important lesson here. Traditional OCR performance metrics focus on recognition accuracy: did the system correctly identify the handwritten characters? Large language models introduce a second layer that asks a different question: given a noisy transcription, what is the most likely intended text? For users, the final output looks dramatically better, creating the impression that handwriting recognition has become nearly solved. In reality, the underlying recognizer may still be making substantial mistakes, many of which are quietly repaired by contextual reasoning.

The result is both impressive and potentially misleading. If the document contains common travel terms such as airport, boarding pass, London, or immigration card, contextual correction works remarkably well. But when the document contains uncommon names, technical terminology, passwords, identifiers, code snippets, or domain-specific jargon, the same mechanism can "correct" the text into something that is fluent but wrong. Put differently, today's systems often excel not because they perfectly read handwriting, but because they are extraordinarily good at guessing what the handwriting was intended to say.

Summary: Looking at the sample page below, the raw transcription produced words such as"surpat" for airport,"valuele" for vehicle,"A-vay machine" for X-ray machine, and "immerdian card" for immigration card. Yet the final output often looks remarkably accurate because the language model uses context to infer what was probably intended. In many cases, what appears to be excellent handwriting recognition is actually a combination of imperfect character recognition and powerful language understanding. The real breakthrough is not that the machine can read every handwritten word, but that it can often guess the right one.

 Extra Reading:

Confidence is the LLM estimate of how likely the "best guess" matches the intended handwritten word. 
Handwritten textBest guessConfidence
bleambecause / blame20%
surpatairport90%
valuelevehicle95%
canncame85%
tales a Utumtakes a U turn95%
1000per10:00 pm90%
frightflight99%
instid enquiryinitial enquiry80%
want continueswait continues95%
grove lengthqueue length90%
tweekningscreening60%
whoumhmm / whomm30%
bequinbegin95%
mere slowlymove slowly95%
A-vay machineX‑ray machine98%
boarding fan the west itemboarding pass, the next item35%
had baggagehand baggage98%
lantonLondon99%
yhyes95%
dot Muposewhat purpose90%
fan et aromfrom her arm / from a folder20%
gilt the boarding tamgives the boarding pass85%
should Day are of the ben pleasant girlsshould say one of the best pleasant girls70%
tack your bagslock your bags75%
20 1TSA lock? / combination lock?10%
immerdian cardimmigration card99%
quesequeue99%
sigafare vise pageSingapore visa page80%
Monty me in 3001cost me Rs 300/-65%
hundleyhundred85%

Places where context helps.



Thursday, May 18, 2017

Autonomous Vehicles | Will They Will They Not in India

Background
Almost every other day there is a news article explaining how Artificial Intelligence (mostly wedded to Deep Neural Networks) will take over the mundane task of driving your car. The positive articles suggest that this vehicle automation will make us all do -better- things like read newspaper while being (auto) chauffeur driven and the negative articles say that these will take away the jobs of taxi drivers (invariably taxi is equivalent to Uber!).

Depending on  whom you speak to you get an answer suggesting anything from 6 months to next couple of year. But almost all seem to be very positive about the possibility of robo-cars on the roads. It is not that they have not been tried, they have in a very very controlled scenario (equivalent to prototype) and when it has been left semi-loose we have had reports of them getting a little scenarios which we humans do. So there is still work to do.

Will it become a reality?
May be in some places, but very unlikely in a city like Mumbai, Delhi (India) where the traffic is not as bad as the picture below is quite chaotic and to expect the unexpected is the first thing a driver learns!


The concept of fine driving (from fine dining!) is absent, the concept of lane depends on the time of the day. The marking on the lanes are courtesy rains and the contractor who gets paid to do the work. The signals may not be visible, the pedestrians could suddenly cross the main road! Every "assumption" under which the self-drive works is not true. So even if self-drive cars make a debut in the west; it will most likely not see much of Indian roads!

If they made it on road in India
For the sake of argument, even if they made it and if we assume (second assumption!) that it will effect Uber drivers in India. We will certainly have the drivers make a protest. May be they do not need to anything special, they just need to get onto the road in hoards and just pretend to move on the road just in front of the auto-driving car multiple times to fool the AI driver to feel that it is not safe to move. The car would stay put! Drivers back into the Uber seat. 

May be drivers elsewhere need to worry. Not the drivers in India. Not anytime soon. Never.
 

A Lesson Hidden in Two Simple Words

Sometimes profound ideas are hidden in the most ordinary places. Recently, while looking at two common English words, I stumbled upon an int...