International Week of the Deaf: Innovative sign language interpreter offers hope for hard of hearing

The ingenious interpreter analyses an individual’s hand gestures and seamlessly translates them into the sign language understood by the recipient.

LUCKNOW Imagine being in a foreign land, surrounded by people speaking languages you can’t understand. You might resort to online translation tools or hand gestures to bridge the gap. But what if you couldn’t hear or speak at all? For the hearing-impaired, this is a daunting reality that often isolates them not only from the hearing world but even from their fellow hearing-impaired individuals in other countries.

Breaking the Silence: Dev Walia and his team working with hearing impaired kids in a village near Greater Noida. (Sourced)

Sign language, it turns out, is not a universal code but a diverse spectrum, with regional variations as distinct as spoken languages. Britain has its sign language, as do the Americans, the Chinese, and the French. Meanwhile, in South Asia, the Indo-Pak sign language prevails among the deaf community. This language divide has long hindered cross-cultural communication between deaf individuals.

Further compounding the communication barriers faced by the deaf is the fact that India has a mere 250 sign language interpreters. To bridge this gap, Dev Walia, a young student from Meerut currently pursuing higher studies in Ireland, along with two of his friends, has developed a hybrid sign language interpreter—a machine learning-powered framework that transcends linguistic boundaries.

“Sign language is unique to each region, and even within regions, there are differences. For example, in India, the sign language often uses both hands, while in some western countries it doesn’t. This diversity has made it incredibly challenging for the deaf from different nationalities to communicate,” said Walia.

The ingenious interpreter, fueled by TensorFlow (an open-source software library for machine learning), analyses an individual’s hand gestures and seamlessly translates them into the sign language understood by the recipient. To its credit, the system can translate all kinds of sign languages, said Himanshu Jakhmola, another member of the team behind the interpreter.

Speaking on the hybrid sign language interpreter, Sanjay Kumar Dubey, a professor from Noida who mentored Walia and his team, said, “This TensorFlow-powered framework opens doors for communication not just between the deaf but also with the hearing world. It fosters social integration, offers deaf children access to education, boosts their self-confidence, and enables them to participate fully in society. Our ultimate goal is to create an easily accessible app or webpage that can interpret multiple sign languages.”

The inspiration for this innovative platform struck Dev and his friends when they were working with deaf children as part of a Pehchaan NGO initiative. Now, the team is taking the next step to make this communication lifeline available to a wider audience by launching it on the internet. However, the team, at present, lacks financial and technical support to design and run an app or a webpage.

Nevertheless, their research, exploring the development and benefits of this framework, has been published in the renowned international journal ‘Springer’. “We believe that our platform, the first of its kind, holds immense potential to bridge the communication gap within the deaf community, provided it can become a commercially viable entity,” said Akull Nainwal, the third member of the team.

Sharing his experience of using the interpreter, Aditya Kishan, a member of the hearing-impaired community, stated, “I got the opportunity to use the interpreter during its trials and found it to be extremely useful, particularly for educated individuals within the community. While this technology may not be as helpful for physical interactions between deaf individuals from different nationalities, it is a valuable tool for facilitating communication over video calls.”

Speaking on the innovation, Akhilendra Kumar, former deputy commissioner, Divyangjan U.P, who has also served as the registrar of Lucknow-based Dr Shakuntala Misra National Rehabilitation University, said, “It is a unique attempt to address a niche challenge faced by the hearing-impaired community in today’s connected world. I hope to see the interpreter available on app stores so that there is better access. Having said this, I do acknowledge that funding an app that targets the hearing impaired and that too, only the ones who wish to communicate with other deaf individuals from different nations might not be a financially lucrative prospect. So, it must be taken up as CSR project or may be, the government authorities can step in to help.”

In the wake of International Week of the Deaf (September 19 to 25), the interpreter a modest yet powerful step towards inclusivity in a world that often overlooks the unique challenges faced by the deaf community.