OpenAI shares details from thwarted romance scams, fake law firms, and an effort to smear Japan's prime minister

• OpenAI released its latest report on scammers and state actors are maliciously using ChatGPT.
• The report shows evidence of everything from fake dating services to intelligence operations.
• In one instance, OpenAI said a user identified themselves as a scammer when asking ChatGPT for tax advice.

OpenAI is peeling back the veil on how scammers are trying to use ChatGPT to do everything from modern twists on romance scams to smear the Prime Minister of Japan.

On Wednesday morning, OpenAI released the latest edition of its intelligence threat report.

Screenshots included in the report show a purported romance scam that OpenAI said likely originated in Cambodia. The report said users asked ChatGPT to create a logo for a fake high-end dating service, generate images of fake women, and provide tax advice. Incredibly, according to OpenAI, when asking for financial advice the user(s) stated their occupation as "scammer."

OpenAI estimated that the scam, which it said targeted Indonesian men interested in luxury lifestyle content, was "likely defrauding hundreds of victims a month."

The company said the operation worked by getting users to choose from a list of fictitious women and relationship types. After building trust, an AI chatbot posing a flirty receptionist directed the conversation over to Telegram. OpenAI said that on Telegram, a mixture of humans using ChatGPT and API, would use "romantic and sexually-explicit language" to direct users to fake dating services and eventually entice them to do a series of "tasks" or "missions" that "required increasingly large payments via bank transfers or digital payment wallets."

It isn't just faux romance that OpenAI said it thwarted. The AI company also said it banned "a cluster of ChatGPT" accounts that posed as law firms, individual attorneys, and US law enforcement.

OpenAI said the scammers asked ChatGPT to generate a fake New York State Bar Association membership card and create social media content to further the scam.

Of the operations OpenAI highlighted, the most brazen may be one attributed to an "individual associated with Chinese law enforcement." According to OpenAI, the individual tried to use ChatGPT to plan "a covert IO" or intelligence operation targeting Japanese Prime Minister Sanae Takaichi. The query came after Takaichi publicly criticized human rights issues in Mongolia.

OpenAI said it gained significant insight into similar "cyber special operations" that were used to "suppress dissent and silence critics both online and offline, at home and abroad," in part because the individual asked ChatGPT to "edit and polish periodic status reports."

"This effort appears to be large-scale, resource-intensive and sustained, counting at least hundreds of staff, thousands of fake accounts across scores of platforms, the use of locally deployed AI models, and a playbook of dozens of tactics," OpenAI wrote in its report. "These range from abusive reporting of dissidents' social media accounts, through mass online posting, to forging documents and impersonating US officials."

Representatives of the Chinese and Japanese embassies did not immediately respond to Business Insider's request for comment.

ChatGPT refused to assist in the planning of the operation targeting Takaichi, OpenAI said. Seemingly undeterred, the user later asked ChatGPT "to polish a status report on what was clearly the same campaign," the implication being that the operation continued anyway.

OpenAI said the user's activity included the use of Chinese AI models, including DeepSeek and Qwen. Based on available data, OpenAI said it could also map the extent of the influence operations.

"This is what Chinese, modern trans-national repression looks like," Ben Nimmo, the principal investigator on OpenAI's intelligence and investigations team, told reporters ahead of the release. "It's not just digital. It's not just about trolling. It's industrialized. It's about trying to hit critics of the CCP with everything, everywhere, all at once."