OpenAI helps spammers plaster 80,000 sites with messages that bypassed filters

Company didn't notice its chatbot was being abused for (at least) 4 months.

Spammers used OpenAI to generate messages that were unique to each recipient, allowing them to bypass spam-detection filters and blast unwanted messages to more than 80,000 websites in four months, researchers said Wednesday.

The finding, documented in a post published by security firm SentinelOne’s SentinelLabs, underscores the double-edged sword wielded by large language models. The same thing that makes them useful for benign tasks—the breadth of data available to them and their ability to use it to generate content at scale—can often be used in malicious activities just as easily. OpenAI revoked the spammers’ account after receiving SentinelLabs’ disclosure, but the four months the activity went unnoticed shows how enforcement is often reactive rather than proactive.

“You are a helpful assistant”

The spam blast is the work of AkiraBot—a framework that automates the sending of messages in large quantities to promote shady search optimization services to small- and medium-size websites. AkiraBot used python-based scripts to rotate the domain names advertised in the messages. It also used OpenAI’s chat API tied to the model gpt-4o-mini to generate unique messages customized to each site it spammed, a technique that likely helped it bypass filters that look for and block identical content sent to large numbers of sites. The messages are delivered through contact forms and live chat widgets embedded into the targeted websites.

“AkiraBot’s use of LLM-generated spam message content demonstrates the emerging challenges that AI poses to defending websites against spam attacks,” SentinelLabs researchers Alex Delamotte and Jim Walter wrote. “The easiest indicators to block are the rotating set of domains used to sell the Akira and ServiceWrap SEO offerings, as there is no longer a consistent approach in the spam message contents as there were with previous campaigns selling the services of these firms.”

AkiraBot worked by assigning the following role to OpenAI’s chat API using the model gpt-4o-mini: “You are a helpful assistant that generates marketing messages.” A prompt instructed the LLM to replace the variables with the site name provided at runtime. As a result, the body of each message named the recipient website by name and included a brief description of the service provided by it.

“The resulting message includes a brief description of the targeted website, making the message seem curated,” the researchers wrote. “The benefit of generating each message using an LLM is that the message content is unique and filtering against spam becomes more difficult compared to using a consistent message template which can trivially be filtered.”

SentinelLabs obtained log files AkiraBot left on a server to measure success and failure rates. One file showed that unique messages had been successfully delivered to more than 80,000 websites from September 2024 to January of this year. By comparison, messages targeting roughly 11,000 domains failed. OpenAI thanked the researchers and reiterated that such use of its chatbots runs afoul of its terms of service.

Story updated to modify headline.

Source

Hope you enjoyed this news post.

Thank you for appreciating my time and effort posting news every day for many years.

News posts... 2023: 5,800+ | 2024: 5,700+ | 2025 (till end of March): 1,357

RIP Matrix | Farewell my friend