Wikipedia warns AI companies to stop scraping and pay for data use or lose access

Wikipedia is asking AI companies to stop scraping its content and instead use its paid API to ensure proper credit and support for contributors.

Wikipedia has sent a clear message to AI developer companies: stop scraping its pages and start using its paid API service. The Wikimedia Foundation, which operates the free online encyclopedia, has urged AI developers to access its content responsibly through its paid Wikimedia Enterprise platform to ensure continued support for the website in the evolving digital age.

Wikipedia Sets Guidelines for AI Use

Wikipedia has asked AI companies to stop scraping content and use its paid API for proper attribution. (Pexels)

By MD Ijaj Khan

Ijaj Khan is a technology journalist and Senior Content Producer at Hindustan Times, with over three years of experience covering the consumer technology industry. His work spans smartphones, laptops, wearables, gaming, appliances and AI - from hands-on reviews, comparison and buying guides to breaking news and in-depth features that help readers cut through the noise and make informed decisions. Before joining HT Tech, he worked with Jagran New Media, where he sharpened his instincts for fast-paced digital reporting. He holds a Post Graduate Diploma in English Journalism and Mass Communication from the Indian Institute of Mass Communication (IIMC), Delhi. Whether he's testing the latest flagship smartphone, tracking a major AI announcement, or putting a gaming laptop through its paces, Ijaj approaches every story with the same goal - making technology feel relevant and easy to understand for everyday users, not just enthusiasts. When he's not in front of a screen for work, he's usually travelling to a new city, hunting for great food, or keeping tabs on what's next in tech before everyone else catches on.

Read moreRead less

In a blog post released on Monday, the foundation said AI developers should properly credit Wikipedia’s contributors and obtain data through official channels rather than automated scraping. The Wikimedia Enterprise product allows large-scale use of Wikipedia’s content without overloading its servers. The foundation added that the revenue from the paid product supports its nonprofit operations, which help sustain the open knowledge platform that relies on volunteers and donations.

Wikipedia’s move comes after it discovered that AI bots had been scraping its website while disguising themselves as human users. After strengthening its bot detection systems, the organisation found that much of its increased traffic in May and June came from such bots attempting to bypass detection. During the same period, the site saw an 8% decline in visits from human readers, which raises concerns about reduced engagement from actual users.

Concerns Over Web Scraping and AI Bots

Wikipedia warned that fewer visits could lead to fewer volunteers improving its content and fewer donors supporting its mission. Earlier this year, it also introduced an AI strategy aimed at assisting its editors with repetitive tasks, language translation, and workflow improvements, tools meant to support human editors rather than replace them.

Through these measures, the Wikimedia Foundation aims to maintain a fair balance between open access to knowledge and responsible use of its content in the fast-growing AI ecosystem.