From skimming that linked page, I think that this download perhaps doesn’t include recent pages? Because in the section talking about enterprise stuff, it mentions the paid API for recent articles
Comment on Wikipedia urges AI companies to use its paid API, and stop scraping
usernameusername@sh.itjust.works 3 weeks ago
I don’t get it though… Why would any company use this when Wikimedia also offers a download of the entirety of Wikipedia, for free?
Maybe it’s just that if the AI companies don’t know, they can hopefully get a little money from them?
AnarchistArtificer@slrpnk.net 3 weeks ago
usernameusername@sh.itjust.works 3 weeks ago
It seems you’re right, I’m just dumb and didn’t read ths article I linked
Crashumbc@lemmy.world 3 weeks ago
You think AI companies care what they scrape. Their system is set up to scrape anything it can get.
usernameusername@sh.itjust.works 2 weeks ago
Oh I know, I was just thinking that if the AI companies will make an exception for Wikipedia (by paying) like the Wikimedia people think, they could also download the complete thing for free. But yeah they probably won’t do any of that so this was kinda useless I think
BanMe@lemmy.world 2 weeks ago
They can scrape an ongoing log of interactions between editors about the articles themselves, which is probably fairly worthwhile content honestly. More content there than in articles probably as well.