A combined research team from Apple and Carnegie Mellon University have shown dumb devices made smart. They created a demonstration video (below) to accompany an academic paper.
Rather than every device you own having to be smart, the approach proposes that something like a HomePod be trained to work out what is happening in the home …
The paper was co-authored by two researchers from each organization. TechCrunch explains the idea behind it.
The system, which they’ve called Listen Learner, relies on acoustic activity recognition to enable a smart device, such as a microphone-equipped speaker, to interpret events taking place in its environment via a process of self-supervised learning with manual labelling done by one-shot user interactions — such as by the speaker asking a person ‘what was that sound?’, after it’s heard the noise enough time to classify in into a cluster.
A general pre-trained model can also be looped in to enable the system to make an initial guess on what an acoustic cluster might signify. So the user interaction could be less open-ended, with the system able to pose a question such as ‘was that a faucet?’ — requiring only a yes/no response from the human in the room.
What does this get you? A concept video shows the example of the smart speaker recognizing the ping of an ordinary dumb microwave and alerting the user that their food is ready. In this way you don’t need to replace your existing appliances with expensive smart ones, you rather let your HomePod or similar device work out what’s happening and then alert you.
Another example demonstrated is recognizing a knock at the door.
It’s fun to think of other applications for this, and some quickly occurred to me.
One example is running a bath. If HomePod can recognize the sound of the bath starting to run, and then the faucet being turned off, it can learn how long the bath should run and alert us if we forget to turn off the faucet.
Another would be to recognize individual household members by their footsteps, triggering customized actions when they return home. No need for a smart camera system with face-recognition, just a HomePod listening out for their distinctive steps.
Check out the video below, and please share your own ideas for dumb devices made smart in the comments.
FTC: We use income earning auto affiliate links. More.
Comments