Moltbook is scary—but not for the reasons so many headlines said

Hello and welcome to Eye on AI. In this edition…why you really should be worried about Moltbook…OpenAI eyes on IPO…Elon Musk merges SpaceX and xAI…Novices don’t benefit as much from AI as people think…and why we need AI regulation now.

This week, everyone in AI—and a lot of people outside of it—was talking about Moltbook. The social media platform created for AI agents was a viral sensation. The phenomenon had a lot of people, even a fair number of normally sober and grounded AI researchers, wondering aloud about how far we were from sci-fi “takeoff” scenarios where AI bots self-organize, self-improve, and escape human control.

Now, it appears that a lot of the alarmism about Moltbook was misplaced. First of all, it isn’t clear how many of the most sci-fi-like posts on Moltbook were spontaneously generated by the bots and how many only came about because human users prompted their OpenClaw agents to output them. (The bots on Moltbook were all created using the hit OpenClaw, which is essentially an open-source agentic “harness”—software that enables AI agents to use a lot of other software tools—that can be yoked to any underlying AI model.) It’s even possible that some of the posts were actually from humans posing as bots.

Second, there’s no evidence the bots were actually plotting together to do anything nefarious, rather than simply mimicking language about plotting that they might have picked up in their training, which includes lots of sci-fi literature as well as the historical record of a lot of sketchy human activity on social media.

As I pointed out in a story for Fortune earlier today, many of the fear-mongering headlines around Moltbook echoed those that attended a 2017 Facebook experiment in which two chatbots developed a “secret language” to communicate with one another. Then, as now, a lot of my fellow journalists didn’t let the facts get in the way of a good story. Neither that older Facebook research nor Moltbook presents the kind of Skynet-like dangers that some of the coverage suggests.

Now for the bad news

But that’s kind of where the good news ends. Moltbook shows that when it comes to AI agents, we are in the Wild Wild West. As my colleague Bea Nolan points out in this excellently reported piece, Moltbook is a cybersecurity nightmare, chock full of malware, cryptocurrency pump and dump scams, and hidden prompt injection attacks—i.e. machine readable instructions, sometimes not easily detected by people, that try to hijack an AI agent into doing something it’s not supposed to do. According to security researchers, it seems that some OpenClaw users suffered significant data breaches after allowing their AI agents on to Moltbook.

Prompt injection is an unsolved cybersecurity challenge for all AI agents that can access the internet right now. And it’s why many AI experts said they are extremely careful about what software, tools, and data they allow AI agents to access. Some only let agents access the internet if they are in a virtual machine where they can’t gain access to important information, like passwords, work files, email, or banking information. But on the other hand, these security precautions make AI agents a lot less useful. The whole reason OpenClaw took off is that people wanted an easy way to spin up agents to do stuff for them.

Then there are the big AI safety implications. Just because there’s no evidence that OpenClaw agents have any independent volition, doesn’t mean that putting them in an uncontrolled conversation with other AI agents is a great idea. Once these agents have access to tools and the internet, it doesn’t really matter in some ways if they have any understanding of their own actions or are conscious. Merely by mimicking sci-fi scenarios they’ve ingested during training, it is possible that the AI agents could engage in activity that could cause real harm to a lot of people—engaging in cyberattacks, for instance. (In essence, these AI agents could function in ways that are not that different from super-potent “worm” computer viruses. No one thinks the ransomware WannaCry was conscious. It did massive worldwide damage nonetheless.)

Why Yann LeCun was wrong…about people, not AI

A few years ago, I attended an event at the Facebook AI Research Lab in Paris at which Yann LeCun, who was Meta’s chief AI scientist at the time, spoke. LeCun, who recently left Meta to launch his own AI startup, has always been skeptical of “takeoff” scenarios in which AI escapes human control. And at the event, he scoffed at the idea that AI would ever present existential risks.

For one thing, LeCun thinks today’s AI is far too dumb and unreliable to ever do anything world-jeopardizing. But secondly, LeCun found these AI “takeoff” scenarios insulting to AI researchers and engineers as a professional class. We aren’t dumb, LeCun argued. If we ever build anything where there was the remotest chance of AI escaping human control, we’d always build it in an “airlocked” sandbox, without access to the internet, and with a kill switch that AI couldn’t disable. In LeCun’s telling, the engineers would always be able to take an ax to the computer’s power cord before the AI could figure out how to break out of its digital cage.

Well, that may be true of the AI researchers and engineers who work for big companies, like Meta or Google DeepMind, or OpenAI or Anthropic for that matter. But now AI—thanks to the rise of coding agents and assistants—has democratized the creation of AI itself. Now a world full of independent developers can spin up AI agents. Peter Steinberger who created OpenClaw is an independent developer. Matt Schlicht, who created Moltbook, is an independent entrepreneur who vibe coded the social platform. And, contra LeCun, independent developers have consistently demonstrated a willingness to chuck AI systems out of the sandbox and into the wild, if only to see what happens…just for the LOLs.

With that, here’s more AI news.

Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn

This story was originally featured on Fortune.com