Technology in Bharathiya Bhasha for Schools

Before I try and put my views on what we should be doing in terms of Technology in Bharatiya Bhasha (Indian languages), I would quickly like to skim through what is the current status or the work that has happened, especially in terms of using technology in Bharatiya languages.

If you look specifically at the topics for discussion, on Day 2, Session IV session,  namely, development of Indic keyboard, handwriting recognition for Bharatiya scripts or the voice-to-text conversion for Bharatiya languages, they broadly fall into the realm of Human Machine Interaction or Human Computer Interaction.

A significant amount of work has happened and is happening, as we speak, in several academic institutes, startups, established industries, but in isolation and in pockets and in many cases the same things (same languages, same use case) is being done. Fortunately, several academic institutes have been liberally funded by the department of science and technology and the progress in terms of outputs or performance has been immense. So the good thing is some of the best minds in India have been working on these cutting-edge technologies for quite some time now. Add to this, the technological improvements that have shaped up in the recent past, namely (a) deep learning architectures especially for language processing and  (b) the recent improvements in the kind of computing, quantum computing included; there has been a significant jump in the performance of most of the technologies and working systems. 

Our own work in Bharatiya Bhasha and the recent advances in deep architecture and computing is an advantage that we can ride on.

While most of the technologies have been demonstrated for a certain type of data and for certain specific languages which are popular around the globe, this is not a disadvantage for us. These architectures which enable enhanced performance can actually be used quite effectively to address Bharatiya languages especially if we have access to the data required to train these architectures. As an example, if you want to build a voice-to-text system you need to have speech data spoken in Bharatiya language and the corresponding Bharatiya text  that you want to generate.

Technological effectiveness has been demonstrated  for English for which there is access to significant amount of readily available data.

Fortunately we are not far behind, we have agencies like the National Language Translation Mission, which have started collecting speech data for different Indian languages (called Bhasha Daan) and actually building demonstrable systems which work as of now in the offline mode and not necessarily in real time yet.  But for real time performance, the technology demonstrated is significantly matured. I wouldn't claim it to be as good as for English but definitely we are on the path of developing  high quality language systems. However these are generic systems.

However, if you look from the education angle there is a need for certain type of speech and text data. For example you might build a speech recognition engine for an Indian language, but for them to work for children of different age groups would take a lot of effort. What is probably required is to be able to collate speech data from school going children. Now this is not a very easy task because not only  are there a large number of Indian languages (+dialects +accents) but the variability in spoken speech in children is very very varied.  A child's voice changes dramatically with age. So all this has to be kept in mind if we want to make a serious effort to build something that can be used by our school and college children.  As the saying goes “data is the new oil” we need to collect data from people for whom we want to build functional systems. 

Building systems for children is a mammoth task requiring government agencies, academic institutions and industries to work together very closely and collect data.

Lets face it. Most of us have been influenced my our teachers to choose a certain specialization of career path. We have had our share of motivated teachers who make sure the children develop interest in that subject.  Specifically from the school and children angle why can not we have motivated teachers for all the children?

Say a teacher teaching in Telugu in Andhra Pradesh has the rare ability to motivate children. Why not have this teacher teach in Odiya in Orissa school? What we probably need is a speech to speech translation technology that can actually translate from Telugu to Odiya. Language translation technology exists. However most of these systems have been trained using generic data. For example they may be built by scraping the data (available in abundance) that has come from newspapers or on Wikipedia articles which have very specific kind of information. So what is really required, for enabling Telugu teacher teaching in Odiya is for us to very systematically collect data or scrape data which is very very specific to the school curriculum. By doing this you should be able to, in a way, get a good teacher in language A to be able to deliver the same technical knowledge in language B without the student actually realizing that the teacher can not speak in Odiya.

An additional possibility is to enable the teaching material to be spoken in the voice of a certain personality. How about learning mathematics from Shakuntla Devi? or history from Amitab Bhachan? Personalized text to speech as a technology exists. We need data, specific to school curricula and in a form that can be used to train speech generating systems (text to speech)

How about learning mathematics from Shakuntla Devi? or history from Amitab Bhachan? 

For varying reasons, Indian students do not ask too many questions, though they are inquisitive. Technology can help build self help systems, like conversational interfaces,  where the children can actually seek answers to their queries without any inhibitions. This is possible using technologies like large language models (LLM). To build LLMs, especially  when it comes to schools, you need to be very careful about the authenticity of the data. You cannot have children learn incorrect information. Luckily the LLMs don't play so much havoc with syntax  but they have hallucinations, which is a huge problem from an angle of stating true facts. The curation of data  is very very important if these are to be adopted for schools and colleges. 

We need a collaborative effort involving several different organizations to come together to make it happen.

To summarize, what I think, are the important things that we should be working on are

• build systems which can listen to  children's voices and respond interactively in the language of their choice with empathy.
• give them technology which can be used  to listen to their lessons in the language of their choice and in the voice of their choice.
• tools that can enable them to self assess themselves again in the language of their choice by interacting with a voice of their choice.

Exams are every students nightmare every year we have our Prime Minister having a session with children to how to face exams! Self assessment conversational interfaces might help ease the exam nightmare.

Exams are every students nightmare. Technology can help.

Most of these things are something the current or the near future technology can enable; but we need to put very serious and concentrated effort in collecting data, curating it and making sure it is in a form that can be digested by machine learning architectures.

Before I end, there is one small catch. Technologies that cater to human machine interactions have cultural aspects built into them as well. Today we use design aspects that are broadly conceptualized in the western world to cater to the western cultures and as a result are not the right fit.  One way of overcoming this is to  build people's strength in higher colleges 

To wrap-up. We are a huge country. We have loads of challenges in terms of number of languages, dialects, accents. However, we have the intellectual energy in terms of ideas. We have access to usable technology. We have the ability to build working solutions. We have people who are motivated. We have the belief.

We just need a collaborative and a focused effort to move the needle.

Thank you.

Publishing Journey - Questions and Answers

At what point in your career did you start publishing your works and what really motivated you? 

My journey into publishing my work began during my master's program, although I can't pinpoint the exact moment it started. It was around the time I was pursuing a master's in engineering, and towards the end of my program, I found myself involved in a research project as a research assistant. This project involved classifying ships in the ocean using emerging neural network technology. While the specifics of the project aren't crucial, what mattered was the timing – this was when neural networks were just gaining traction, and I recall the IEEE Transactions on Neural Networks publishing its initial issues around that period. Working with limited computing resources in the early '90s, I achieved some promising results, and it felt like something worth sharing with the scientific community. I was fortunate to be part of a research team that had a strong culture of publishing their findings and presenting at conferences. Seeing the potential of my work motivated me to start writing and documenting my research. The feeling of having your work published and recognized, especially in those pre-digital days when it meant seeing your words in print, was exhilarating. It was at that moment that I realized I had something valuable to contribute to the field. From then on, my publishing journey has been about communicating the work I do. It's important to note that many of us tend to shy away from documenting our work, but if we don't, even our best efforts might go unnoticed. So, motivation came from both the desire to see my work in print or, in today's terms, in digital media, and the necessity to share it with the world effectively. Another aspect that drove me to publish was my innate tendency to document my experiences from a young age. I found immense satisfaction in doing so. Looking back at the paragraphs and writings I penned years ago, it instilled in me a sense of self-confidence and accomplishment. Documenting my journey and research not only preserves my work but also provides a sense of continuity and personal growth over time. In essence, it's a way of validating my own progress and contribution to the world of knowledge.   

How can aspiring authors leverage technology and digital platforms to enhance their publishing experience? 

Today, aspiring authors have a wealth of machine and technology resources at their disposal, making the process of producing their work more accessible. This technological support is particularly significant in the Indian context, where English is often a second language. Despite having learned English in our educational curriculum or attending English-medium schools, it may not be our primary language spoken at home or in natural settings. From an Indian perspective, technology is a game-changer. In the past, we relied on grammar and spell-checking tools, which were quite useful. However, as an author looking to express original ideas that have not been previously published, it's advisable to choose platforms or software that allow you to focus on your thoughts without the burden of formatting. Instead of using "what you see is what you get" document platforms, consider typesetting platforms or software. These tools handle the formatting for you, freeing up more mental space for your writing. You can concentrate on your content without worrying about where to place figures or captions, which can be incredibly liberating. Furthermore, take advantage of the tools at your disposal, such as grammar checkers, which have gained immense popularity. Nowadays, you can even find technological aids that can rephrase sentences to make them more readable or less convoluted. In today's world, we have a plethora of tools available to us. While advanced AI like Chat GPT is an option, for now, focusing on platforms that facilitate idea generation is crucial for authors who are meant to be idea generators. In summary, modern authors have a wide array of technological resources to streamline their writing process, and selecting the right tools can greatly enhance their productivity and creativity.

As an author what challenges have you typically faced during the publishing journey, and how could you overcome them?

The challenges in the realm of publishing vary depending on the type of publication you are aiming for. For instance, when you're working on a journal paper or a conference paper, one immediate challenge is meeting the deadlines set by the publication. Knowing the precise date by which you must have your manuscript ready can be both a blessing and a challenge. Moreover, these publications typically have strict page limits, which means you must condense your content effectively. It's akin to composing a telegram for those who remember; every word must count. On the other hand, writing a book presents a different set of challenges. Writing a book is more like running a marathon. While there may not be rigid deadlines, you often set softer ones for yourself. It becomes a journey of self-motivation, as there's no external pressure. When you commit to writing a book, you embark on a self-discipline journey. Unlike journal papers, where the publication process is streamlined and deadlines are clear, writing a book requires consistent effort and determination. Fortunately, in today's publishing landscape, many publishing houses offer preset formats, making formatting concerns less of an issue. You can focus on your writing, and the rest is taken care of by the publishing platform. However, the primary challenges shift towards the content itself. Ensuring originality and avoiding plagiarism becomes crucial. While plagiarism checkers can help, the ultimate responsibility lies with the author to maintain academic integrity. In summary, the challenges in publishing differ based on the type of publication, with journals having stricter deadlines and page limits, while writing a book demands self-discipline and originality. Modern publishing platforms have streamlined many aspects of the process, but authors must remain vigilant about maintaining the integrity of their work.

What advice would you give to individuals who are passionate about sharing their knowledge but may be hesitant yet to take the leap into the world of publishing?

Here's some straightforward advice: If you have the passion and knowledge to share, there's nothing to fear about publishing your work. There are numerous avenues available for getting your ideas out there. Start by identifying what truly excites you and what you're eager to write about. Don't obsess over crafting linguistically elaborate prose. Most ideas can be effectively communicated using simple vocabulary, and you don't need an extensive vocabulary to make a meaningful impact. If your goal is to publish, and you have a substantial amount of material, then you mostly need the discipline to organize it. Be prepared for multiple rounds of editing. Writing a longer piece, such as a book, is akin to translating your thoughts onto paper and then meticulously refining them through deletions, revisions, and rearrangements. Even after numerous edits, you may still feel like it could be better – it's a bit like working on your thesis up until the last day of submission. Remember, a manuscript is never truly in its final form until you submit it. Focus on your passion for writing and don't overly concern yourself with how many people read your work or the number of likes and thumbs-ups it receives. The key is to channel your passion and take that leap. You don't need to aspire to be a wildly successful author; you simply need to express yourself. Writing can actually be more forgiving for introverts compared to extemporaneous speaking or addressing a crowd. It provides a platform for introverts to articulate their thoughts and feelings through text, making it a valuable means of self-expression. So, embrace your passion and start writing without hesitation. There's nothing holding you back.

Authorship or Acknowledgement? Order of Authors!

 {Personal views}

Being in an R&D organization means there are several instances when you have to write (Scientific or Technical Papers) about what you do in peer reviewed conference or journals.Very often, the resulting work is a team effort and as a consequence most papers, written today, have multiple authors. 

Few decades ago, as a research scholar, it was just you and your supervisor as the two sole authors of any output that came out of the PhD exploration. This was indeed true, especially if you were writing a paper based on your ongoing research towards a PhD. In the pre-google days, the trend was to email the second author (usually the supervisor) to ask for a copy of the paper so that you could read the research and hopeful build on it because you knew that the supervisor would be more static in terms of geo coordinates than the scholar.  

However the concept of multiple authors for a research article is seeping into academic research as well. These days labs write papers and not individuals. As a research scholar (pursuing your PhD) you are "contributing" to the work of another research scholar (in the same lab) and s\he is also "contributing" to your work and as a result you are an author in not only your own "own" work but also in the work being done by the other person. To such an extent that, if you are a recruiter, it gets difficult to know who contributed to what in that article. And this ambiguity grows when the lab is of some decent size (more authors per paper; more combinations).

No I am not talking about citations or the number of papers per person or what ever the measure we have to measure the outcome of research work. That is for another day, maybe.

During the course of a PhD,  the general trend was, two authors (the research scholar and the supervisor);  but today most publications have multiple authors working in similar areas belonging to the same lab under the same supervisor!

Authorship or Acknowledgement?

Having always taken pride in the belief that I have made sure that the research work that we do and publish in the lab has authorship based on the actual contribution by the member of the lab and not necessarily due to they being part of the lab. I was in for a shock, when one of my senior colleagues begged to differ saying that there were instances, within the lab, when this was not true. I was in for a pride-check or a shock. I am still trying to figure out what was the instance!

This came up when a request for authorship of research work based on a two or three days experimental effort was made to me and I thought that effort, at best, deserved an acknowledgement and not an authorship in the research work.

I have not looked at authorship or acknowledgment in any formal detail and made most of  my judgement purely based on a notion of "contribution" a person made to the research work being communicated. 

While I thought it was commonsense ... there is plenty of material on authorship, acknowledgement and order of authorship. I quote.

Determining the authorship of a work or the criteria for a person to be on the author list. 

Authorship

[A] According to ICMJE authorship recommendation be based on the following 4 criteria:

1. Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND
2. Drafting the work or revising it critically for important intellectual content; AND
3. Final approval of the version to be published; AND
4. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

note that all the authors need to meet these 4 criteria to claim ownership of the article. Clearly denying #2 and #3 should not be used to disqualify someone from authorship. All individuals who meet the criterion #1 should have the opportunity to participate in the review, drafting, and final approval of the manuscript.

Acknowledgements 

According to ICMJE  contributors who meet fewer than all the 4 criteria for authorship should not be listed as authors, but they should be acknowledged.
 

Examples of activities that alone (without other contributions) do not qualify a contributor for authorship are

1. acquisition of funding;
2. general supervision of a research group or general administrative support; and
3. writing assistance, technical editing, language editing, and proofreading.

Contributions  may be acknowledged individually or together as a group and their contributions should be specified (e.g., "served as scientific advisors," "critically reviewed the study proposal," "collected data," "participated in writing or technical editing of the manuscript").

 
[B] According to Nature 


Each author is expected to have made substantial contributions to the conception or 

1. substantial contributions to design of the work; or 
2. substantial contributions to the acquisition, analysis, or 
3. substantial contributions to interpretation of data; or
4. substantial contributions to the creation of new software used in the work; or
5. substantial contributions to drafting the work or substantively revised it

AND to have approved the submitted version (and any substantially modified version that involves the author's contribution to the study);

AND to have agreed both to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.

[C] Authorship (in natural sciences) should be limited to those who have contributed substantially to the work according to Proceedings of the National Academy of Sciences of the United States of America.

[D] Anyone listed as Author on an ACM manuscript submission must meet all the following criteria:
they have made substantial intellectual contributions to some components of the original work described in the manuscript; and
they have participated in drafting and/or revision of the manuscript and
they are aware the manuscript has been submitted for publication; and
they agree to be held accountable for any issues relating to correctness or integrity of the work.


Other contributors may be acknowledged at the end of the paper, before the bibliography.

While my own judgement was purely based on a notion of "{substantial} contribution" a person made to the research work being communicated. It is not far from what ACM or Nature of National Academy of Science put it more formally! At this point of time another crucial point crops up. What should be the order of authorship, especially if you have a team working on a problem!

Earlier there was no need to say anything specifically about contribution but today because of the increase in the number of authors per paper, we have a new trend "these authors have equal contribution" what ever that means!

This is also a point of discussion!

 Order of Authorship

According to  Editage  

1. The first author is usually the person who has made the most significant intellectual contribution to the work, in terms designing the study, acquiring and analyzing data from experiments, and writing the manuscript. 
2. The subsequent authors are usually listed as per their contribution to the research, starting with the one who contributed the most to the least.
3. The corresponding author is the one who receives all notifications from the journal including manuscript status, reviewers’ comments, and the final decision.  The corresponding author is often the group leader or a senior researcher whose contact address is not likely to change in the near future. 

In order to avoid any authorship dispute, it is a good practice to discuss authorship and the order of authors at the beginning of the project itself, and keep a record of each of the contributors involved throughout the project.

In Mathematics the authors are usually listed in alphabetical order (popularly called Hardy-Littlewood Rule).

In Computer Science in general the principal contributor is the first in the author list. However, the practice of putting the principal investigator last in the author list has increasingly become an accepted standard across most areas in science and engineering.

End Note

Clearly, while there is a reasonably good discussion about who can stake claim to an authorship, who should be acknowledged and what is the order of authorship.  But the description is so "loose" that each team has to use their commonsense judgement to decide how they interpret "contribution". 

The central problem remain in terms of what can be termed as "substantial contribution" which probably can not be quantified precisely!

Some Thoughts!

1. A junior member of the team who works on the experiments (and hopefully analyzes the results too) could be the first author. Most research in this digital era is based on the experiments and analysis of the results anyway!
2. The PI\supervisor (even, if has contributed significantly to the experimentation idea and details) becomes the second or the third author
3. Any authors of the paper must have made "substantial intellectual contributions" to some components of the original work described in the manuscript
1. How do we determine "significant" contribution?
2. Authors could be strongly encouraged to state their specific contributions before publishing a paper
3. This can determine the authorship order, else
4. We can follow the alphabetical listing of authors (as done in Mathematics!)
4. Should we encourage PI in the author list without specific contributions? Namely, discourage the practice of putting the principal investigator last in the author list has increasingly become an accepted standard across most areas in science and engineering.
   

Appraisals - The Yearly Ache

{A Personal View}

Background

Apprisals are an yearly ache that we have imposed on ourselves to make ourselves a better person than we were previously. It is stressful to both the  assessor (manager) who is on one side of the spectrum and the assessee (team member). It is at this time of the year the distance between the manager and the team members is at the maximum, even though they are part of the same team.

Irrespective of which role you play you have a solid list of greviences against the  other role. So it is nothing to do with the role, it is to do with the situation. While the tension is on during the apprisal time - the folks in human resource bring in their wisdom and introduce the dreaded Gaussian curve\band and introduce a constraint where you as a manager have to necessarily have a certain number of your team members in a certain region under the curve and then the finance folks attach a monetary value to each part of the curve. These two, the constraint linked to money, makes the situation even more tense than it actually is and widens the gap further between the assessor and assessee. Put all this together and you have a self created "आ बैल मुझे मार" situation. Year after year we all go through this annual ache, grudging it but without trying to ease the ache and it looks like we take it for granted that this is something we can not avoid because this is the process that we have been following!

Nobody seems to own this situation. While in private all agree that this is an ache that should be avoided and push the blame on someone up the decision making chain and sincerely hoping that something has to be done. But in public, most of them claim that this is a necessary evil to reward the people who do their job well and penalize the people who were unable to do their job as well so that people work to make them selves better.

Does this reward-penalize actually work?  How objective can the assessor be? Does this actually help the person being penalized to do better or for that matter the person rewarded to do better. The statistics are sketchy and probably derived from a set of data points that might be not free of errors scientifically.

Elementary understanding says that the Gaussian curve probably is alright if you had a very huge number of people in a team but often teams are very small and the concept of the curve fails miserably. Somehow we miss this point and try to map a small team to fit into this curve. This probably is the reason for the ache. Even if you had a great talented team, some one among them has to occupy the dreaded part of the curve. Seems unfair! Na. "The 12th man in the Australian team was good enough to be the captain of the (newly formed) Zimbawae team" in a close analogy for those of us who follow the game of cricket. 

Force Gaussian

If you come to look at it, as a manager you select the best talent\people to form your team, you spend a lot of time questioning them, evaluating them before you actually recruit them. But during annual apprisal time you find the same very person who you diligently selected, surprisingly not upto the mark.

A case of "we recruit the best" and then at some point of time saying they are "not as good as" we thought they were!

This seems like a paradox. Was it that the "best talent" that you selected using your wisdom suddenly turned out to be not so good under your supervision? Or did you as the manager fail to actually spot the right talent? Either way the failure seems to be of the manager during the recruitment process.

But, I guess, the truth is the "act of trying to fit the curve on a population for which it is not suitable"! The imposed Gaussian curve fitment. The sum effect is the widended gap between the manager and the team member which results in only one thing "dip in the morale of the team".

A (fresh?) Look

While people in management and sociology might not agree and bring in theories to support the reward-penalize approach. Isn't empathy missing? Are we being robots (loosely defined a "human minus emotion"). Is a low morale team better for the organization? Answers depends on the person with whom you have this dialogue.

For an organization which recruits the best people from the very top Engineering colleges in India and then for their R&D unit take the top 1% of the recruited students 

The annual appriasal for the R&D unit should be into three bands A+, A, A-. Exceptionally poor performers in band A-, exceptionally excellent performers in A+ and the rest (99.7%) in A. 

Details

If we consider all the students in all the engineering colleges in India, we can safely assume that they fit a Gaussian curve (see Figure 1; the 0 indicates a dumb student and 10 indicates a smart student with an average student being represented by 5.) with mean 5 and variance 1.

Assuming that the organization recruits only the best among the best students from, which probably means that they do not recruit anyone who is labeled between 0 and 4, implying that  the student that they recruit are atleast better that a shade below average (this is pessimistic, because most R&D organizations demand a minimum of 60% marks during their academic stint).

Under this assumption (recruits students above 4) we see that the associates
in the organization form a skewed Gaussian with a mean 5.3 and a variance of 0.63. And that in the R&D unit (top 1% of the recruits) has a mean of 7.8 and variance of 0.09. Observe that there is so small variation in the talent of the students recruited, just 0.09.

Just to give a better view - Distribution of Engineering Students (top; G(5.00488, 0.99750)) recruited by the organization (middle; G(5.28819, 0.62633)) into R & D unit (bottom; G(6.831203, 0.16680)).

 

A very rough interpretation is that the recruits in R & D (selecting top 1% from recruits in the organization) would be hovering at 7.8 (on a scale of 1 to 10) with almost all (99.7% [*]) of them being between 6.87 and 8.732. The super small range implies that a 3 band distribution might be best suited. Further, the thin spread suggests that exceptional over performers (A+) are too few and so are exeptional under performers (A-) meaning that a majority (≡ almost all) in R & D are A.

[*] Using the well known P(μ−2σ < X ≤ μ+3σ) ≈ 99.7%