Meta's Muse AI Assistant Vulnerable to Hijacking Via Undocumented Setting

macOS security researcher Patrick Wardle discovered a flaw in Meta's new Muse AI assistant that an attacker can use to turn it into "the ultimate backdoor."

The flaw is to do with an undocumented setting, endo_voyager_dictation_endpoint, that can be "redirected locally with no special privileges." When you click the microphone to dictate a prompt, Muse sends it to the attacker's endpoint instead.

This flaw gives an attacker access to everything Muse has access to, which can be a lot of your data since it's an agentic AI assistant that you're meant to give access to your local apps like messaging and emails and even online accounts.

Patrick posted a Proof of Concept on his GitHub that you can download and try out for yourself if you're so inclined.

The attack is local, meaning the attacker needs to already be on your machine in order to make use of it, which mitigates the threat somewhat. However, another vector can allow for remote execution: a ClickFix attack like what has been making the rounds in the news lately.

ClickFix is a type of social engineering attack that presents a fake CAPTCHA and asks you to do something that would compromise your computer like copy and run a command in the command line, giving the attacker access to your machine in one fell swoop.

CAPTCHAS, by nature of being so ubiquitous and annoying that most people turn their brain off and mindlessly complete whatever task is shown to them, have turned out to be the ultimate training ground for social engineering attacks.

macOS implemented a warning before you paste a command into the terminal, but it's still easily bypassable and this warning is absent on other operating systems.

Meta described how secure Muse was in multiple blog posts. The AI itself runs in what they call a Secure VM on Meta's servers, a separate, dedicated computer that no one else's agent can access. A separate "Sentinel" runs on your machine and controls what goes to the VM in the cloud, keeping separation between your machine and the agent.

Meta calls this "first-of-its-kind privacy, safety, and security protections," although they say they haven't built it on confidential computing yet and that will be coming later.

Clearly, there were still some security holes left though. The unfortunate reality is that you can try to secure AI agents as much as you want, but they need access to your apps and data in order to work. And any AI suffers from prompt injection attacks because they inherently can't tell the difference between their instructions and untrusted input. There will always be increased risk when using AI agents on your computer.