Showing posts with label LLM. Show all posts
Showing posts with label LLM. Show all posts

Saturday, August 22, 2026

Handwriting Recognition - A 2026 status check

Background: For years, I had a habit of writing whenever I traveled. No laptop, no apps, just a pen, a notebook, and pages of cursive observations about airports, flights, fellow travelers, and the emotions that come with being away from home. Two decades later, I decided to digitize those notebooks and make them available on Kindle, eventually publishing them as Transit Tales: Between Airports and Emotions: Travel Blog 20 Years Too Late (View on Amazon). As a first step, I assumed modern LLM-powered handwriting recognition would effortlessly convert the pages into text. Instead, the journey led to an unexpected discovery: handwriting recognition is still far from "solved."

The apparent success of modern handwriting recognition is increasingly a combination of imperfect OCR and powerful language-model reconstruction. What users see is often not what the system read, but what the system inferred.

Handwriting Recognition: Better Than It Looks, But Not For The Reason You Think

a hand written page which has been scanned


Modern handwriting recognition demos can be surprisingly impressive. Feed a handwritten page such as the one below into a state-of-the-art OCR system and the resulting transcript may appear almost perfect at first glance. However, a closer examination reveals that the raw recognition quality is often much poorer than it seems. The magic comes not from character recognition alone, but from the language model sitting behind it.

Consider several examples from this handwritten page. The handwritten phrase corresponding to"airport" may initially be recognized as something closer to"surpat". Similarly,"vehicle" can become"valuele","takes a U-turn" becomes"tales a Utum","X-ray machine" becomes"A-vay machine", and"immigration card" appears as"immerdian card". Taken literally, these outputs are poor transcriptions and would be difficult for a human reader to understand without context.

Large language models change the game. Given the surrounding sentence, the model realizes that a traveler arriving at a terminal is far more likely to write"airport" than"surpat","vehicle" than"valuele", and"X-ray machine" than"A-vay machine". The model effectively performs error correction using context, world knowledge, and statistical expectations about language. In other words, it is not necessarily reading the handwriting correctly; it is often inferring what the author probably meant.

This distinction becomes even more obvious with longer phrases. The raw transcription contains fragments such as"want continues - the grove length increase" which are difficult to interpret. Yet an LLM can confidently reconstruct the phrase as"wait continues - the queue length increases". Likewise,"dot Mupose" can be corrected to"what purpose","quese" to"queue", and"lanton" to"London". These corrections are plausible and often accurate, but they are driven at least as much by language understanding as by handwriting recognition itself.

There is an important lesson here. Traditional OCR performance metrics focus on recognition accuracy: did the system correctly identify the handwritten characters? Large language models introduce a second layer that asks a different question: given a noisy transcription, what is the most likely intended text? For users, the final output looks dramatically better, creating the impression that handwriting recognition has become nearly solved. In reality, the underlying recognizer may still be making substantial mistakes, many of which are quietly repaired by contextual reasoning.

The result is both impressive and potentially misleading. If the document contains common travel terms such as airport, boarding pass, London, or immigration card, contextual correction works remarkably well. But when the document contains uncommon names, technical terminology, passwords, identifiers, code snippets, or domain-specific jargon, the same mechanism can "correct" the text into something that is fluent but wrong. Put differently, today's systems often excel not because they perfectly read handwriting, but because they are extraordinarily good at guessing what the handwriting was intended to say.

Summary: Looking at the sample page below, the raw transcription produced words such as"surpat" for airport,"valuele" for vehicle,"A-vay machine" for X-ray machine, and "immerdian card" for immigration card. Yet the final output often looks remarkably accurate because the language model uses context to infer what was probably intended. In many cases, what appears to be excellent handwriting recognition is actually a combination of imperfect character recognition and powerful language understanding. The real breakthrough is not that the machine can read every handwritten word, but that it can often guess the right one.

 Extra Reading:

Confidence is the LLM estimate of how likely the "best guess" matches the intended handwritten word. 
Handwritten textBest guessConfidence
bleambecause / blame20%
surpatairport90%
valuelevehicle95%
canncame85%
tales a Utumtakes a U turn95%
1000per10:00 pm90%
frightflight99%
instid enquiryinitial enquiry80%
want continueswait continues95%
grove lengthqueue length90%
tweekningscreening60%
whoumhmm / whomm30%
bequinbegin95%
mere slowlymove slowly95%
A-vay machineX‑ray machine98%
boarding fan the west itemboarding pass, the next item35%
had baggagehand baggage98%
lantonLondon99%
yhyes95%
dot Muposewhat purpose90%
fan et aromfrom her arm / from a folder20%
gilt the boarding tamgives the boarding pass85%
should Day are of the ben pleasant girlsshould say one of the best pleasant girls70%
tack your bagslock your bags75%
20 1TSA lock? / combination lock?10%
immerdian cardimmigration card99%
quesequeue99%
sigafare vise pageSingapore visa page80%
Monty me in 3001cost me Rs 300/-65%
hundleyhundred85%

Places where context helps.



Saturday, May 31, 2025

The Curious Case of the Overachieving LLM

The essay explores the absurdity of overcomplicating simple communication through the use of advanced AI. It narrates a scenario where two humans use a language model to draft and summarize an email about a 3 PM meeting, highlighting the ironic inefficiency of employing sophisticated technology for mundane tasks. The tale humorously questions whether AI is simplifying processes or merely adding unnecessary complexity to human interactions.

This was entirely generated using Microsoft Copilot with little or no modifications. The prompt used was Given the scenario “Human A uses LLM to compose a mail to Human B and Human B uses LLM to summarize the received email!” Compose a humorous essay keeping in mind if using LLM was necessary in the first place.


The Curious Case of the Overachieving LLM
{Or the absurdity and ironic inefficiency of overcomplicating simple communication using advanced AI, specifically large language models (LLMs).}



In the age of technological prowess, one might imagine that communication has become simpler, faster, and perhaps even more meaningful. But then, there exists *the scenario*. A scenario so absurdly unnecessary, yet so hilariously emblematic of modern times, that one cannot help but marvel at its ingenuity—or perhaps, its inefficiency.


To set the stage: Human A has a straightforward piece of information to convey to Human B. We’re not talking about complex geopolitical strategies or the secret recipe for grandma’s lasagna. No, it’s likely something as mundane as, “The meeting starts at 3 PM.” But here’s where the plot thickens: Human A does not simply type out the sentence and send it to Human B. Oh, no. Human A decides that this task requires the intellectual gravitas of a cutting-edge language model (LLM). Why bother with simplicity when you can involve artificial intelligence?

So, Human A opens their laptop, logs into their favorite LLM interface, and begins the prompt. A symphony of keystrokes ensues: “Generate a formal yet approachable email informing Human B that the meeting starts at 3 PM. Use a tone that’s professional but friendly and maybe add a touch of humor so it doesn’t seem too stiff.” Human A leans back, satisfied, as the LLM whirs to life, producing an email that reads more like a Shakespearean sonnet than a logistical memo:


--- 
Dear Human B,

I hope this message finds you in good spirits and perhaps enjoying a cup of coffee or tea as the afternoon unfolds. I am writing to let you know that the meeting we’ve been eagerly anticipating will commence at 3 PM sharp. How thrilling, right? Please don’t be late, for the clock waits for no one. Looking forward to seeing you then!


Warm regards,
Human A

---

Human A beams with pride. The email is a masterpiece—an Olympic gold medalist in the sport of overcommunication. But the saga doesn’t end here. Enter Human B.

Human B receives this poetic missive and, instead of simply reading it, decides that *they too* must harness the mighty power of the LLM. After all, why expend mental energy extracting information when artificial intelligence can do it for you? Human B prompts the LLM: “Summarize the email I just received. Focus on key details and remove any fluff.” The LLM dutifully complies, responding:

--- 
Summary: The meeting starts at 3 PM.

---




And there it is. Two humans, two interactions, two uses of advanced technology. And yet, the final result is precisely what could have been achieved if Human A had simply texted Human B, “Meeting at 3 PM.”


But let us not be quick to judge. The LLM is, after all, the unsung hero of this tale. It played mediator, translator, and even humorist, ensuring that the essence of the message survived the gauntlet of unnecessary embellishment. Without the LLM, Human A might have agonized over word choice, and Human B might have spent a full 30 seconds reading the email instead of 3 seconds skimming a summary. Clearly, this is progress.


The scenario raises important philosophical questions about the role of technology in human communication. Are we outsourcing not just our tasks but also our reading comprehension? Are LLMs becoming middlemen in conversations that could be as direct as handing someone a sticky note? And perhaps the most pressing question: What are we doing with all the extra time saved?


The answer, of course, is obvious. We’re spending it prompting LLMs to write essays like this one.


In conclusion, the curious case of Human A, Human B, and the overachieving LLM underscores our unyielding commitment to efficiency—even if that efficiency involves circuitous routes and redundant processes. The LLM, in all its glory, is not just a tool; it is a testament to humanity’s ability to complicate simplicity. Who needs direct communication when you can have an artificial intelligence-powered extravaganza? Now, if you’ll excuse me, I need to ask an LLM to summarize this essay.

Original PDF Article

Tuesday, October 03, 2023

Technology in Bharathiya Bhasha for Schools


Before I try and put my views on what we should be doing in terms of Technology in Bharatiya Bhasha (Indian languages), I would quickly like to skim through what is the current status or the work that has happened, especially in terms of using technology in Bharatiya languages.

If you look specifically at the topics for discussion, on Day 2, Session IV session,  namely, development of Indic keyboard, handwriting recognition for Bharatiya scripts or the voice-to-text conversion for Bharatiya languages, they broadly fall into the realm of Human Machine Interaction or Human Computer Interaction.

A significant amount of work has happened and is happening, as we speak, in several academic institutes, startups, established industries, but in isolation and in pockets and in many cases the same things (same languages, same use case) is being done. Fortunately, several academic institutes have been liberally funded by the department of science and technology and the progress in terms of outputs or performance has been immense. So the good thing is some of the best minds in India have been working on these cutting-edge technologies for quite some time now. Add to this, the technological improvements that have shaped up in the recent past, namely (a) deep learning architectures especially for language processing and  (b) the recent improvements in the kind of computing, quantum computing included; there has been a significant jump in the performance of most of the technologies and working systems. 

Our own work in Bharatiya Bhasha and the recent advances in deep architecture and computing is an advantage that we can ride on.
While most of the technologies have been demonstrated for a certain type of data and for certain specific languages which are popular around the globe, this is not a disadvantage for us. These architectures which enable enhanced performance can actually be used quite effectively to address Bharatiya languages especially if we have access to the data required to train these architectures. As an example, if you want to build a voice-to-text system you need to have speech data spoken in Bharatiya language and the corresponding Bharatiya text  that you want to generate.

Technological effectiveness has been demonstrated  for English for which there is access to significant amount of readily available data.

Fortunately we are not far behind, we have agencies like the National Language Translation Mission, which have started collecting speech data for different Indian languages (called Bhasha Daan) and actually building demonstrable systems which work as of now in the offline mode and not necessarily in real time yet.  But for real time performance, the technology demonstrated is significantly matured. I wouldn't claim it to be as good as for English but definitely we are on the path of developing  high quality language systems. However these are generic systems.

However, if you look from the education angle there is a need for certain type of speech and text data. For example you might build a speech recognition engine for an Indian language, but for them to work for children of different age groups would take a lot of effort. What is probably required is to be able to collate speech data from school going children. Now this is not a very easy task because not only  are there a large number of Indian languages (+dialects +accents) but the variability in spoken speech in children is very very varied.  A child's voice changes dramatically with age. So all this has to be kept in mind if we want to make a serious effort to build something that can be used by our school and college children.  As the saying goes “data is the new oil” we need to collect data from people for whom we want to build functional systems. 

Building systems for children is a mammoth task requiring government agencies, academic institutions and industries to work together very closely and collect data.
Lets face it. Most of us have been influenced my our teachers to choose a certain specialization of career path. We have had our share of motivated teachers who make sure the children develop interest in that subject.  Specifically from the school and children angle why can not we have motivated teachers for all the children?

Say a teacher teaching in Telugu in Andhra Pradesh has the rare ability to motivate children. Why not have this teacher teach in Odiya in Orissa school? What we probably need is a speech to speech translation technology that can actually translate from Telugu to Odiya. Language translation technology exists. However most of these systems have been trained using generic data. For example they may be built by scraping the data (available in abundance) that has come from newspapers or on Wikipedia articles which have very specific kind of information. So what is really required, for enabling Telugu teacher teaching in Odiya is for us to very systematically collect data or scrape data which is very very specific to the school curriculum. By doing this you should be able to, in a way, get a good teacher in language A to be able to deliver the same technical knowledge in language B without the student actually realizing that the teacher can not speak in Odiya.

An additional possibility is to enable the teaching material to be spoken in the voice of a certain personality. How about learning mathematics from Shakuntla Devi? or history from Amitab Bhachan? Personalized text to speech as a technology exists. We need data, specific to school curricula and in a form that can be used to train speech generating systems (text to speech)

How about learning mathematics from Shakuntla Devi? or history from Amitab Bhachan? 

For varying reasons, Indian students do not ask too many questions, though they are inquisitive. Technology can help build self help systems, like conversational interfaces,  where the children can actually seek answers to their queries without any inhibitions. This is possible using technologies like large language models (LLM). To build LLMs, especially  when it comes to schools, you need to be very careful about the authenticity of the data. You cannot have children learn incorrect information. Luckily the LLMs don't play so much havoc with syntax  but they have hallucinations, which is a huge problem from an angle of stating true facts. The curation of data  is very very important if these are to be adopted for schools and colleges. 

We need a collaborative effort involving several different organizations to come together to make it happen.
To summarize, what I think, are the important things that we should be working on are

  • build systems which can listen to  children's voices and respond interactively in the language of their choice with empathy.
  • give them technology which can be used  to listen to their lessons in the language of their choice and in the voice of their choice.
  • tools that can enable them to self assess themselves again in the language of their choice by interacting with a voice of their choice.


Exams are every students nightmare every year we have our Prime Minister having a session with children to how to face exams! Self assessment conversational interfaces might help ease the exam nightmare.

Exams are every students nightmare. Technology can help.

Most of these things are something the current or the near future technology can enable; but we need to put very serious and concentrated effort in collecting data, curating it and making sure it is in a form that can be digested by machine learning architectures.

Before I end, there is one small catch. Technologies that cater to human machine interactions have cultural aspects built into them as well. Today we use design aspects that are broadly conceptualized in the western world to cater to the western cultures and as a result are not the right fit.  One way of overcoming this is to  build people's strength in higher colleges 

To wrap-up. We are a huge country. We have loads of challenges in terms of number of languages, dialects, accents. However, we have the intellectual energy in terms of ideas. We have access to usable technology. We have the ability to build working solutions. We have people who are motivated. We have the belief.

We just need a collaborative and a focused effort to move the needle.

Thank you.

A Lesson Hidden in Two Simple Words

Sometimes profound ideas are hidden in the most ordinary places. Recently, while looking at two common English words, I stumbled upon an int...