Muse Looks Cute, but Looks Are Deceiving

Jason Aten, in a good follow-up to his previous column at Inc. (the one where he described how Muse, running on his Mac, read his Messages database):

The entire reason Muse is interesting is that it isn’t just a chatbot, but can actually do things for you. But that promise also comes with a pretty high bar of responsibility. If your goal is to put a powerful AI agent on millions of devices, you have to be incredibly clear about what’s happening, and what people are agreeing to. If your primary audience does not understand what Full Disk Access means, you should not surprise them with “I’m reading your text messages.” It’s unreasonable to expect them to learn about things like virtual machines, permission architectures, macOS TCC, Sentinel agents, or database synchronization in order to understand what the thing is doing or what is happening with their personal information. Having an agent installed on people’s computers that appears overeager to access their information or take action on their behalf isn’t helpful. It’s terrifying.

Aten makes some good points about the inherent tension between Muse — especially in the context of running on your Mac desktop — being powerful cutting-edge technology yet being presented by Meta in an extremely consumer-friendly package. Muse is getting a lot of attention — including mine — because it’s both groundbreaking technically (each user gets their own entire persistent Linux VM running in Meta’s cloud) and because it’s packaged in an easy-to-install easy-to-use way. It’s literally presented as a cute mascot. It’s the first consumer-accessible agentic AI system, and Meta has truly done an amazing job with that. But it’s a genuinely open question whether consumers have any understanding what this means. If you buy a power saw that can cut your fingers off, you are almost certainly aware that you are buying a power saw that can sever your fingers. You may or may not choose to use it with appropriate carefulness. But you know just by looking at it that a power saw (or a nail gun, or a chef’s knife, or a hammer, or any other powerful tool) is dangerous. I don’t think people realize how powerful — and thus dangerous — Muse is, especially if it’s running on your Mac.

★