Bots are currently scraping the internet for LLM training data at unprecedented rates[1][2][3], driving up costs and destabilizing public-facing websites. I want to talk about how this has been particularly difficult for wikis, and has gotten much worse in the last few months.
  • poVoq@slrpnk.netM
    15·
    4 months ago

    Not only wikis sadly. Anything that has public facing deep links that trigger extensive database operations are being hammered by these bots and few servers can take the load.