Meta says Israeli firm posted 'deceptive' AI-generated content on Gaza war: 'Posed as Jews, African Americans...'

Meta identifies an Israeli political marketing firm likely using AI to spread misinformation on Instagram and Facebook regarding the Israel-Gaza war.

Meta identified Israeli political marketing firm, STOIC using deceptive and "likely AI-generated" content on its Facebook and Instagram platforms.

Israeli armoured personnel carriers (APC) operate, amid the ongoing conflict between Israel and the Palestinian Islamist group Hamas, near Israel's border with Gaza in southern Israel(REUTERS)

This included comments praising Israel's handling of the war in Gaza, published below posts from global news organizations and U.S. lawmakers, according to a Reuters report.

The accounts posed as Jewish students, African Americans and other concerned citizens, targeting audiences in the United States and Canada, the social media giant said in its quarterly security report.

The report is the first to disclose the use of text-based generative AI technology since it emerged in late 2022, according to Reuters. The report highlighted six covert influence operations that Meta disrupted in the first quarter.

"There are several examples across these networks of how they use likely generative AI tooling to create content. Perhaps it gives them the ability to do that quicker or to do that with more volume. But it hasn't really impacted our ability to detect them," said Meta’s head of threat investigations, Mike Dvilyanski to Reuters.

Researchers have been concerned that generative AI, which can quickly and cheaply produce human-like text, imagery and audio, could lead to more effective disinformation campaigns and sway elections, the article read.

Meta and other tech giants have grappled with how to address potential misuse of new AI technologies, especially in elections, with researchers finding examples of image generators from companies including OpenAI and Microsoft producing photos with voting-related disinformation, despite those companies having policies against such content.

Researchers also doubt the effectiveness of digital labeling systems which mark AI-generated content at the time of its creation, as they work only on images and not on text, the report read