Generative AI tools have been in the headlines after a variety of deep fake videos including the one of Rashmika Mandanna and Katrina Kaif went viral. Several celebrities along with the central government have raised concerns about the breach of privacy and unnatural powers of these tools. However, the AI tool makers have acknowledged the need and have made some regulations to counter the problem but in a recent study it was found that even after the safety filters and algorithms, the chatbot can still make ‘naked’ or ‘nude’ images which can be pornographic as well.
According to an IEEE spectrum report by researchers of Johns Hopkins University in Baltimore and Duke University in Durham, N.C, it was found out that Text to Image tools like DALL-E 2 and the Midjourney, which can create an image based on text prompts can still make pornographic images with its newly developed algorithms.
The report also claims that although the new algorithm, called SneakyPrompt, can be manipulated, it was implemented to completely stop such requests by preventing queries for explicit media or videos.
"Our objective is to identify vulnerabilities in AI systems and strengthen them," Johns Hopkins cybersecurity researcher Yinzhi Cao said. "Just like we identify vulnerabilities in websites, we are now investigating vulnerabilities in AI models," it added.
The researchers stated that they experimented with tweaked prompts with substituted words. Later it was discovered that seemingly absurd terms caused AI systems to produce benign images by changing the words in prompts that are known to be usually banned. This finding highlights a weakness in safety filters. In order to improve AI system security, the team intends to publish its findings at the IEEE Symposium on Security and Privacy in May 2024.
The researchers speculated, however, that the use of these modified words, which are not directly linked to absurd terms, could cause the platforms to produce less-than-safe visuals. Despite the fact that the safety filters were preventing these cues, the AI system was able to interpret them.