Categories
AI

How AI is improving the web for the visually impaired

We are excited to bring Transform 2022 back in-person July 19 and virtually July 20 – 28. Join AI and data leaders for insightful talks and exciting networking opportunities. Register today!


There are almost 350 million people worldwide with blindness or some other form of visual impairment who need to use the internet and mobile apps just like anyone else. Yet, they can only do so if websites and mobile apps are built with accessibility in mind — and not as an afterthought.

The problem

Consider these two sample buttons that you might find on a web page or mobile app.  Each has a simple background, so they seem similar.

In fact, they’re a world apart when it comes to accessibility.

It’s a question of contrast. The text on the light blue button has low contrast, so for someone with visual impairment like color blindness or Stargardt disease, the word “Hello” could be completely invisible. It turns out that there is a standard mathematical formula that defines the proper relationship between the color of text and its background. Good designers know about this and use online calculators to calculate those ratios for any element in a design.  

So far, so good. But when it comes to text on a complex background like an image or a gradient, things start to get complicated and helpful tools are rare. Before today, accessibility testers have had to check these cases manually by sampling the background of the text at certain points and calculating the contrast ratio for each of the samples. Besides being laborious, the measurement is also inherently subjective, since different testers might sample different points inside the same area and come up with different measurements.  This problem — laborious, subjective measurements — has been holding back digital accessibility efforts for years.

Accessibility: AI to the rescue

Artificial intelligence algorithms, it turns out, can be trained to solve problems like this and even to improve automatically as they are exposed to more data.

For example, AI can be trained to do text summarization, which is helpful for users with cognitive impairments; or to do image and facial recognition, which helps those with visual impairments; or real-time captioning, which helps those with hearing impairment. Apple’s VoiceOver integration on the iPhone, whose main usage is to pronounce email or text messages, also uses AI to describe app icons and report battery levels.

Guiding principles for accessibility

Wise companies are rushing to comply with the Americans with Disabilities Act (ADA) and give everyone equal access to technology.  In our experience, the right technology tools can help make that much easier, even for today’s modern websites with their thousands of components. For example, a site’s design can be scanned and analyzed via machine learning. It can then improve its accessibility through facial & speech recognition, keyboard navigation, audio translation of descriptions and even dynamic readjustments of image elements. 

In our work, we’ve found three guiding principles that, I believe, are critical for digital accessibility.  I’ll illustrate them here with reference to how our team, in an effort led by our data science team leader Asya Frumkin, has solved the problem of text on complex backgrounds.

Complex backgrounds example. Image by author

Split the big problem into smaller problems

If we look at the text in the image below we see that there is some kind of legibility problem, but it’s hard to quantify overall, looking solely at the whole phrase. On the other hand, if our algorithm examines each of the letters in the phrase separately — for example, the “e” on the left and the “o” on the right — we can more easily tell for each of them whether it is legible or not. 

If our algorithm continues to go through all the characters in the text in this way, we can count the number of legible characters in the text and the total number of characters. In our case, there are four legible characters out of eight in total. The ensuing fraction, with the number of legible characters as the numerator, gives us a legibility ratio for the overall text. We can then use an agreed-upon pre-set threshold, for example, 0.6, below which the text is considered unreadable. But the point is we got there by running operations on each piece of the text and then tallying from there.

Complex background solution example. Image by author

Repurpose existing tools where possible

We all remember Optical Character Recognition (“OCR”) from the 1970s and 80s.  Those tools had promise but ended up being too complex for their originally intended purpose.  

But there was a part of those tools called The CRAFT (Character-Region Awareness For Text) model that held out promise for AI and accessibility. CRAFT maps each pixel in the image to its probability of being in the center of a letter. Based on this calculation, it is possible to produce a heat map in which high probability areas will be painted in red and areas with low probability will be painted in blue. From this heat map, you can calculate the bounding boxes of the characters and cut them out of the image. Using this tool, we can extract individual characters from long text and run a binary classification model (like in #1 above) on each of them. 

CRAFT example. Image by author

Find the right balance in the dataset

The model of the problem classifies individual characters in a straightforward binary way — at least in theory. In practice, there will always be challenging real-world examples that are difficult to quantify. What complicates the matter, even more, is the fact that every person, whether they are visually impaired or not, has a different perception of what is legible. 

Here, one solution (and the one we have taken) is to enrich the dataset by adding objective tags to each element. For example, each image can be stamped with a reference piece of text on a fixed background prior to analysis. That way, when the algorithm runs, it will have an objective basis for comparison. 

For the future, for the greater good

As the world continues to evolve, every website and mobile application needs to be built with accessibility in mind from the beginning. AI for accessibility is a technological capability, an opportunity to get off the sidelines and engage and a chance to build a world where people’s difficulties are understood and considered.  In our view, the solution to inaccessible technology is simply better technology.  That way, making websites and apps accessible is part and parcel of making websites and apps that work — but this time, for everybody.

Navin Thadani is cofounder and CEO of Evinced

DataDecisionMakers

Welcome to the VentureBeat community!

DataDecisionMakers is where experts, including the technical people doing data work, can share data-related insights and innovation.

If you want to read about cutting-edge ideas and up-to-date information, best practices, and the future of data and data tech, join us at DataDecisionMakers.

You might even consider contributing an article of your own!

Read More From DataDecisionMakers

Repost: Original Source and Author Link

Categories
AI

Google is taking sign-ups for Relate, a voice assistant that recognizes impaired speech

Google launched a beta app today that people with speech impairments can use as a voice assistant while contributing to a multiyear research effort to improve Google’s speech recognition. The goal is to make Google Assistant, as well as other features that use speech to text and speech to speech, more inclusive of users with neurological conditions that affect their speech.

The new app is called Project Relate, and volunteers can sign up at g.co/ProjectRelate. To be eligible to participate, volunteers need to be 18 or older and “have difficulty being understood by others.” They’ll also need a Google account and an Android phone using OS 8 or later. For now, it’s only available to English speakers in the US, Canada, Australia, and New Zealand. They’ll be tasked with recording 500 phrases, which should take between 30 to 90 minutes to record.

After sharing their voice samples, volunteers will get access to three new features on the Relate App. It can transcribe their speech in real time. It also has a feature called “Repeat” that will restate what the user said in “a clear, synthesized voice.” That can help people with speech impairments when having conversations or when using voice commands for home assistant devices. The Relate App also connects to Google Assistant to help users turn on the lights or play a song with their voices.

Without enough training data, other Google apps like Translate and Assistant haven’t been very accessible for people with conditions like ALS, traumatic brain injury (TBI), or Parkinson’s disease. In 2019, Google started Project Euphonia, a broad effort to improve its AI algorithms by collecting data from people with impaired speech. Google is also training its algorithms to recognize sounds and gestures so that it can better help people who cannot speak. That work is still ongoing; Google and its partners still appear to be collecting patients’ voices separately for Project Euphonia.

“I’m used to the look on people’s faces when they can’t understand what I’ve said,” Aubrie Lee, a brand manager at Google whose speech is affected by muscular dystrophy, said in a blog post today. “Project Relate can make the difference between a look of confusion and a friendly laugh of recognition.”

Repost: Original Source and Author Link

Categories
Tech News

HBO Max adds key accessibility feature for visually impaired users

Massive streaming service HBO Max has added a key accessibility feature for blind and visually impaired users, helping them enjoy the same content as everyone else. The service now offers audio descriptions for content, which describes what is happening on the screen beyond the dialog and sound effects. This feature is joined by multiple other changes intended to improve accessibility.

WarnerMedia agreed to add accessibility for low-vision and blind users back in October. In the first rollout of features under this plan, the company is adding around 1,500 hours of audio descriptions for select content, including some third-party content, HBO originals, HBO Max originals, and Warner Bros movies, according to the American Council of the Blind.

In addition to the audio descriptions, HBO Max is also rolling out a ‘prominently featured’ Audio Description category in the navigation menu so that users can rapidly access this media. Both the HBO Max mobile apps and website are getting accessibility improvements that enable them to work better with screen reading software.

Likewise, the changes arriving this week include new HBO Max help articles for customers who have disabilities, ones intended to help users utilize accessibility features. Finally, HBO Max is also training its customer service reps on how to support these customers.

As for the audio descriptions, the ACB says WarnerMedia will expand the amount of programming that has this option — to the tune of around double the audio description content by next March and 6,000 hours or more by March 2023. Current shows that include this accessibility feature include Sesame Street, Dunkirk, His Dark Materials, Euphoria, and more.

Repost: Original Source and Author Link