• FauxLiving@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        6 months ago

        LLM-driven web scraping is intense for some sites, so their bot detection software is tuned in a way that creates a lot of false positives.

        Obscuring your browser fingerprint, or blocking javascript, or using an unusual user-agent string can trigger a captcha challenge.

        If you’re not doing that and seeing a site suddenly start giving your captchas then they may be being DDoS’d by scrapers and are challenging all clients.

        A site that archives content is especially vulnerable because they have a lot of the data that is useful for AI training.

        It is incredibly annoying, but until we have a robust way of proving identity that can’t be gamed by bad actors we’re stuck with individual user challenges.

      • mjr@infosec.pub
        link
        fedilink
        English
        arrow-up
        2
        ·
        6 months ago

        Not every time, but far too often. They don’t seem to care that they’re discriminating against people with AV impairment, plus locking out some secure browsers.

      • Vik@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        6 months ago

        No but I do get about three or four challenges. I can paste the article for you if it helps?

      • Axolotl@feddit.it
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        2
        ·
        edit-2
        6 months ago

        I don’t have this problem; You probably are using TOR or a VPN and it triggered the captcha, if it’s not then it’s def strange, never seen this happen to me