GPT Chatbot Marketplace: It’s Full of Adult Content Generators

“The best GPT models will be community-driven,” OpenAI announced in November 2023. What happened nine months later? Gizmodo found that many of the GPT models (Generative Pre-trained Transformer) contradicted the OpenAI digital store regulations – a special section where you can download community-created GPTs.

Among other things, they were there chatbots to generate AI pornography, or to avoid tools that detect AI-written text or those that offered legal and medical advice, or even gambling-related solutions.

OpenAI has trouble moderating and enforcing its own rules

The existence of these types of GPT models in the OpenAI marketplace is a direct contradiction of the principles the company itself has established. The startup banned the use of technology for the purposes of creating sexual content, legal and medical advice, promoting educational fraud, assisting in gambling or impersonating other people (deepfake), as well as interfering in electoral systems.

These issues have been raised in the media before, but OpenAI clearly has a hard time moderating. This could be particularly damaging when it comes to legal advice, as research from Stanford University has shown that in over half of cases of such queries, chatbots were “hallucinating,” or providing fictitious information. In total, Gizmodo has found over 100 GPTs that violated OpenAI’s rules, and in many cases, they were used tens of thousands of times (each one separately). After the list was revealed by the portal, the company removed the GPTs from the market, but the damage had already been done.

At this point, developers of chatbots/GPTs made available on the OpenAI marketplace do not receive any profits, but the startup wanted to enable their monetization in the future. Milton Mueller, head of the Internet Governance project at the Georgia Institute of Technology, believes that it’s perverse that OpenAI has “an almost apocalyptic vision that AI will save the world, but it can’t handle something as simple as eradicating porn bots from the store.”

Mueller believes that if OpenAI really wants to create a vibrant ecosystem, it will take more than just adding a few lines of code to block certain keywords, but a deeper approach, with much greater commitment.