Google rolls out AI-powered screen reader with interaction capabilities- Details

Google brings new updates to accessibility features such as TalkBack and Expressive Captions on Android.

After Apple introduced a new set of accessibility features across products, Google on Thursday announced new updates to Android and Chrome’s accessibility features to honour Global Accessibility Awareness Day. The company revealed that it is bringing new AI and Gemini capabilities to Android for users with visual and hearing impairments. Google has upgraded the TalkBack feature, Android’s screen reader, with Gemini. Now, users can also ask follow-up questions about images opened on the screen. This will provide users with seamless AI interaction and Android device usage. Know more about Google’s new accessibility features for Android and Chrome. 

Google upgrades TalkBack feature with Gemini AI capabilities. Know how it works.(Google )

Google’s TalkBack feature with Gemini

Last year, Google integrated Gemini AI into Android’s TalkBack feature. This enables users to get image descriptions when alt text is not available. Now, with a new update, users can now ask questions about the image or content displayed on the screen. This gives Gemini on-screen awareness as it can provide responses to users based on the question and image. Google said, “That means the next time a friend texts you a photo of their new guitar, you can get a description and ask follow-up questions about the make and colour, or even what else is in the image.”

Alongside upgrades to the TalkBack feature, Google also introduced “Expressive Captions”, which works similarly to Live captions. It captures audio or words on your screen and transcribes them into text in real-time.  The feature leverages artificial intelligence (AI) to analyse and understand emotions or expressions as well. Apart from words, this feature also recognises other sounds such as “Whistling” or “Throat clearing” and provides live captions. Google reported that the feature is rolling out in English in the United States, the United Kingdom, Canada, and Australia. 

Apart from these features, Chromebooks also offer Face Control, which manages actions via facial gestures. Google has also made Interaction with PDFs seamless with Optical Character Recognition (OCR), offering screen reader support, search, text selection, and more. These features come soon after Apple rolls out advanced accessibility features like Nutrition Labels for App Store, Magnifier for Mac, Braille Experience, Live Captions on Apple Watch, and more. 

Mobile Finder: Samsung Galaxy S25 Edge LATEST specs, features, and price