Smart watch alerts users with hearing loss!

Smartwatches offer people a private method of receiving notifications about their surroundings, such as a phone call, health alerts, or an upcoming package delivery. Now researchers at the University of Washington have developed SoundWatch, a smartwatch app for people who are deaf and hard of hearing who want to be aware of nearby sounds. When the smartwatch picks up a sound the user is interested in, for example a siren, microwave beep or bird chirp, SoundWatch identifies it and sends the user a friendly buzz along with information about the sound.

This technology provides a way for people to experience sounds that require action, such as removing food from the microwave when it’s dry, but these devices can also enhance people’s experiences and help them feel more connected to the world. The team began this project by designing a system for the deaf and hard of hearing who wanted to know what was going on in their homes.

The first system developed, called HomeSound, uses Microsoft Surface tablets distributed throughout the home that act as a network of interconnected displays, each display providing a basic floor plan of the home and alerting the user to a sound and its source. The screens also show the waveform of the sound, to help users identify it, and store a history of all the sounds that a user might have missed when not at home. Researchers tested HomeSound in the homes of six deaf or hard-of-hearing participants for three weeks, and they were instructed to go about their lives as normal and complete weekly surveys.

Based on their reports, a second prototype used machine learning to classify sounds in real time, the researchers created a dataset of more than 31 hours of 19 common household-related sounds, such as a dog barking or the meowing of a a cat, a crying baby and knocks on the door. The researchers then zeroed in on a smart surveillance system, which allows users to receive audible alerts wherever they are, even in places where they don’t have their phones, like at the gym.

Because smartwatches have limited storage and processing capabilities, the team needed a system that wouldn’t eat up the watch’s battery and was also fast and accurate. First the researchers compared a compressed version of the HomeSound classifier against three other available sound classifiers, the HomeSound variant being the most accurate, but also the slowest.

To speed up the system, the team has the watch send the sound to a device with more processing power, the user’s phone, for classification. Having a phone sort sounds and send the results to the watch not only saves time but also maintains user privacy because sounds are only transferred between the user’s own devices. Incredible how technological advances continue to be applied to provide a better quality of life! Don’t you think?