27 September 2023

Awesome & terrifying: Facebook’s plan to give virtual assistants bodies

Start the conversation

Tristan Greene* says Facebook recently showed off the progress its AI research team has made in the realm of household robotics, potentially making that virtual assistant a little less virtual.


The dream is to take the virtual assistant out of the speaker and put it into an autonomous body capable of traversing your house.

If you’re thinking “Rosie the Robot” from The Jetsons, you’re not far off. Just think smaller.

Per a Facebook AI blog post:

“To accomplish a task like checking to see whether you locked the front door or retrieving a cell phone that’s ringing in an upstairs bedroom, AI assistants of the future must learn to plan their route, navigate effectively, look around their physical environment, listen to what’s happening around them, and build memories of the 3D space.”

Facebook’s created a new system called SoundSpaces that gives robots the ability to interpret sounds.

Current virtual assistants merely listen for wake words and then use natural language processing to interpret verbal commands as triggers.

While the microphone array technology behind some of these assistants is impressive, they’re merely designed to pick up voices in noisy environments.

In order for a robot to retrieve a phone ringing in another room it has to be able to “hear” the sound and determine where, relative to its own current position, it’s coming from.

According to the blog post, SoundSpaces allows developers to “train AI agents in 3D environments with highly realistic acoustics. This opens up an array of new embodied AI tasks, such as navigating to a sound-emitting target, learning from echolocation, or exploring with multimodal sensors.”

Facebook‘s other big reveal was its Semantic MapNet.

This one’s a bit more complex, but in essence it gives robots the ability to navigate their environment using both computer vision and the aforementioned SoundStage system while also recognising “feature-based” semantic correlations. In other words it can remember where the couch is with greater precision than it could by simply examining pixels of images generated by its other sensors.

The robotics market has gone through a bit of upheaval as of late. A few years back it seemed like everyone was working on a gadget you could squeeze a smart assistant into that would follow you around.

These novelties mostly ended up being convoluted amalgams of random virtual assistant abilities crammed into toy robot bodies.

While they can be fun for a moments, they’re ultimately not useful.

Many of those startups are no longer in business.

So far the idea of a robot maid, like “Rosie,” is science fiction. That’s why we have to think smaller.

It sounds like what Facebook‘s trying to create is a useful robot that won’t run the risk of accidentally killing you if it bumps into you on the stairs.

And that means it’ll be tiny.

There are only so many ways a robot can navigate a home environment.

And unless these things are only going to be sold to rich people who live esoterically in modified aircraft hangars, the smart money’s on a small gadget that’ll be capable of carrying a kilogram or two, walking up stairs, and potentially climbing the walls like a spider right before dropping on your face to record your dreams while you sleep – that last part is pure conjecture.

If the idea of Facebook putting a robot in your house capable of exploring, mapping, and recording everything that happens sounds scary to you… well, it should be.

Facebook is an AI company that makes its money selling advertising.

Anything it can get in your home is going to logically work toward that end.

But, it’s also awesome.

If we could put aside the privacy concerns for a moment (which is a lot like saying “If we can ignore the impending nuclear explosion for a second…”) the idea of smart assistants that can actually assist is huge.

It might be cool to imagine having a giant robot bodyguard that can cook, clean, kick in doors, and subdue a burglar.

But the reality is that level of safe autonomy could be decades away.

Instead, think of it like Alexa or Google Assistant, only you could give it commands such as “grab me a beer,” or “go give the baby their pacifier.”

Instead of pulling out your phone to start a Zoom call, you could simply whistle for your robot and it’d mount itself on a tripod and connect you.

If you got up to pace while you’re talking it’d rotate to follow you.

They might even be capable of passing the butter.

These, of course, are just a few potential abilities a small robot-embodied virtual assistant could provide.

In reality there’s no end to the possible utility a robust, well-engineered robot with Facebook‘s new AI features could bring.

Of course, any company attempting to tie us at the hip to what, essentially, is the equivalent of an iPhone that can walk and carry things is going to have to gain the public trust in a major way.

It’s one thing to know that Alexa and Siri can hear everything going on in your house.

But your Echo Dot or iPhone can’t wander into the bathroom while you’re doing your business and then report back to advertising HQ.

Who knows what kind of data these things will gather and how the companies making them will use it.

Still, how cool would it be to have a little robot sidekick?

*Tristan Greene writes about human-centric artificial intelligence advances, political policy concerning tech, and the horrors of living on dry land.

This article first appeared at thenextweb.com

Start the conversation

Be among the first to get all the Public Sector and Defence news and views that matter.

Subscribe now and receive the latest news, delivered free to your inbox.

By submitting your email address you are agreeing to Region Group's terms and conditions and privacy policy.