• SirEDCaLot@lemmy.today
    link
    fedilink
    English
    arrow-up
    21
    ·
    2 days ago

    Ever run an AI model locally? If you want the most capability you need a fast GPU with 32-48gb RAM. And that’s all for you, ONE user.

    Copilot has millions of users, with tens or hundreds of thousands of them hitting the AI all at once. Each one needs $thousands worth of GPU and RAM dedicated to them for the length of their query processing.

    Where do you think the money to buy all that hardware comes from? You see OpenAI buying a double digit percentage of the world’s RAM production, you think they got it on clearance sale?

    No, there are investors. Investors who are pouring hundreds of billions into this AI stuff. And they don’t do this because it’s fun, they do it because they expect a BIG return.

    So what’s going on is just like your neighborhood drug pusher, only the drug pusher is more honest. He says ‘first hit’s free, man’. AI company says ‘AI models are an easy and cost effective way to modernize your workflow!’; they don’t tell you that once you’ve integrated them and fired all the humans who know how to do the work, the price is gonna go way up.

    Because the fact is, there IS a real cost of AI compute. GPU time, or at the large scale, datacenter space, power, cooling, etc.

    In another few months to few years, the C-suites will stop huffing the koolaid and will start doing cost-benefit analysis on where AI is and isn’t cost-effective vs. humans. With any luck (for the AI people) by that time the AIs will be good enough that it’s a clear benefit. If not this bubble’s gonna pop.

    • BlackPenguins@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      2 days ago

      That’s why I would rather run locally. I control my data. Better for the environment. And if I ask a programming question, sure ChatGPT will come back in seconds, but I’m fine with waiting 30-60 seconds for my own AI. How impatient are we.

    • heartSagan5@lemmy.zip
      link
      fedilink
      English
      arrow-up
      6
      ·
      2 days ago

      So, what you’re saying…is the AI Bubble is going to pop once the pencil pushers do the math? But they’re asking their local LLM for that… so it isn’t happening?

      • SirEDCaLot@lemmy.today
        link
        fedilink
        English
        arrow-up
        3
        ·
        2 days ago

        Not pop. Correct.

        A lot of the managers aggressively pushing AI have little or no understanding of it themselves. They just hear of a technology that can make a human more productive by doing most of the work for them. So absolutely that’s worth a ton of money. It’s why many companies are encouraging if not demanding employees to start using AI- because in their mind, one employee fully utilizing AI can do the work of two standard employees. Of course they believe this because they’ve never actually had to use the damn thing themselves and thus don’t realize it doesn’t do all the work for you. Or worse they think it does and your wonderful code base turns into spaghetti.

        Side note- A few companies even had leaderboards for who was using the most AI tokens. This led to ‘tokenmaxxing’, trying to consume as many tokens as possible to prove you are adopting AI. Things like 'Write unit tests for our company code base, then refactor the code base. Spin up an instance of Claude and another of ChatGPT to each generate unit tests of the old code and run them against the new code, then run the tests against each other to check each other’s work, submit full debug output to another instance of gpt 5.5 that will check for hallucinations… Keep that query going for a few paragraphs and you’ll have an army of AI workers all checking each other’s work while producing zero productive output but costing a fortune to run.

    • T156@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      1
      ·
      2 days ago

      Ever run an AI model locally? If you want the most capability you need a fast GPU with 32-48gb RAM. And that’s all for you, ONE user.

      Even then, that’s quite small. Top of the line frontier models would be looking at hundreds of gigabytes of video memory, and just as much RAM.

      A terabyte of VRAM/RAM needed for something like CoPilot is probably a fairly sensible estimate.

      • phx@lemmy.world
        link
        fedilink
        English
        arrow-up
        6
        ·
        2 days ago

        Depends on what you want to do, the model, and optimization or quantization.

        A lot of LLM stuff that seemed pretty amazing a few years ago - chatbots and the like that respond to questions in plain language - can run in comparatively light hardware. Coding agents can take more, but could also be optimized against a particular language and spit out useful snippets.

        Image stuff can be pretty complex especially at higher resolutions and detail, and creating seamless video segments gets expensive on hardware, fast.

        • SirEDCaLot@lemmy.today
          link
          fedilink
          English
          arrow-up
          2
          ·
          2 days ago

          Quite true. The thing is, there aren’t billions and billions of dollars in chatbots. The billions are for the creative stuff and the code.

          And that is where the reckoning / correction will come from, the bill has to come due eventually. When top end generative AI starts to have a real cost associated with it, then it’s no longer a blanket ‘everyone start using this immediately’ mandate, it prompts some consideration of cost versus output quality.

  • gdtf@lemmy.ml
    link
    fedilink
    English
    arrow-up
    3
    ·
    2 days ago

    Nice. Maybe now my company stops trying to shove it down our throats.

    • heartSagan5@lemmy.zip
      link
      fedilink
      English
      arrow-up
      4
      ·
      2 days ago

      For real. This whole “AI/LLM” nonsense could’ve been a nothing burger if they correctly charged it at the get go… Ah, well. Venture capital is suckers, I guess?

      • Auli@lemmy.ca
        link
        fedilink
        English
        arrow-up
        2
        ·
        2 days ago

        The problem is though they made it basically free. And that was a mistake cause people don’t like paying for stuff once it was free.

      • Sinthesis@lemmy.today
        link
        fedilink
        English
        arrow-up
        3
        ·
        2 days ago

        From what I’ve seen, the AI companies don’t even know what to charge. For example, claude.ai finally made an API for user costs and we get this fun fact:

        Values for a given date may be revised for up to 30 days as late events arrive and reconciliation runs. For invoicing-grade totals, query dates at least 30 days in the past.

  • sudoer777@lemmy.ml
    link
    fedilink
    English
    arrow-up
    6
    ·
    2 days ago

    Deepseek V4 Flash exists as a small open-weight model and is crazy cheap for what it is capable of. ClosedAI/Anthropic and other non-free models are sketchy AF with their pricing and basically everything else about them.

    • paraphrand@lemmy.world
      link
      fedilink
      English
      arrow-up
      43
      ·
      4 days ago

      Microsoft was deceptive here and never made it clear exactly what sort of deal you were getting with the flat rate. There was no indication of the actual magnitude.

  • teft@piefed.social
    link
    fedilink
    English
    arrow-up
    74
    arrow-down
    3
    ·
    4 days ago

    Man, enshittification is happening so fast for ai. Imagine the next big thing. It’ll be enshittified prior to release.

    • porous_grey_matter@lemmy.ml
      link
      fedilink
      English
      arrow-up
      38
      arrow-down
      2
      ·
      4 days ago

      This isn’t enshittification in the traditional sense, they haven’t captured the market enough for that. They’re just panicking because they’re burning cash way too fast.

        • Windex007@lemmy.world
          link
          fedilink
          English
          arrow-up
          9
          ·
          3 days ago

          Don’t wanna be “that guy”, but if if 95% of the time people use it to generally mean “greedy companies making things shitty to try and wring an extra buck”, then that just is what that word means now.

          It leaves a vacuum for the concept it was originally intended to describe, but that’s how she goes.

          • paraphrand@lemmy.world
            link
            fedilink
            English
            arrow-up
            6
            arrow-down
            1
            ·
            edit-2
            3 days ago

            I guess so. But the original definition was useful and encapsulated a specific thing. And then everyone discarding that definition is a bummer. But it’s not like there is anything to be done about it. And Doctoro has said he’s not too annoyed by it himself.

            • Windex007@lemmy.world
              link
              fedilink
              English
              arrow-up
              3
              arrow-down
              1
              ·
              3 days ago

              I wholeheartedly agree. I love when I hear a new word, and a mushy concept just snaps into focus. Like, yea, I understood that concept in my “gut” but had not special word for it? So gratifying.

              Feels like theft when it gets appropriated to something else.

              Glad to hear Cory is cool w/ it. Putting a word (even by accident) to a poorly-expresable part of the human experience that let’s people communicate more effectively? That is a contribution to society that most of us only dream of. Fucker did it by accident. Thrilled for him.

              • SpaceCowboy@lemmy.ca
                link
                fedilink
                English
                arrow-up
                3
                ·
                2 days ago

                I don’t think saying AI is being enshittified is too far from the orignal essay

                This is enshittification: surpluses are first directed to users; then, once they’re locked in, surpluses go to suppliers; then once they’re locked in, the surplus is handed to shareholders and the platform becomes a useless pile of shit. From mobile app stores to Steam, from Facebook to Twitter, this is the enshittification lifecycle.

                The AI companies were definitely subsidizing usage of the tech in the hopes of getting people locked in. Don’t think people are all that locked in. Probably some people are, suppliers like NVIDIA seem pretty locked in. But I don’t think there’s all that much to direct to shareholders relative to what they put into it to try to get lock in.

                So it’s kinda failed enshittification? Premature enshittification… preshittification?

    • kieron115@startrek.website
      link
      fedilink
      English
      arrow-up
      4
      ·
      2 days ago

      This is the same business model that tech “startups” use, just at a vastly accelerated time table. Upend some existing market (in this case, several markets) by burning through cash at an unsustainable rate. Once your customers have been hooked, and/or the alternatives eliminated as competitors, you crank up the prices to try and make the business cashflow positive.

    • zurohki@aussie.zone
      link
      fedilink
      English
      arrow-up
      14
      ·
      3 days ago

      Meta kind of did that with their VR stuff. They skipped the appealing to users part and build a bland, brand-safe, microtransaction-laden experience to sell to businesses assuming they could just use their size to force users to buy it.

    • Lost_My_Mind@lemmy.world
      link
      fedilink
      English
      arrow-up
      15
      arrow-down
      2
      ·
      4 days ago

      As a mod for the Enshitification you hate to see Enshitification in any form.

      As a mod of Fuck_AI, I’ll make an exception in this one case.

      COMMENCE THE ENSHITIFICATION!!!

      Which completely goes against my Enshitification mod mindset, but here we are.

      So…nobody told me life was going to be this way…

      clap-clap-clap-clap

      • spankysalmon@fedinsfw.app
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        1
        ·
        3 days ago

        They did, if by “everyone” you mean rich detached executives that constantly make decisions they have not the intelligence to understand.

  • replicat@lemmy.world
    link
    fedilink
    English
    arrow-up
    24
    arrow-down
    1
    ·
    3 days ago

    I’ve been using it for well over a year now.

    Hit my limit in 1 day and canceled.

  • Onno (VK6FLAB)@lemmy.radio
    link
    fedilink
    English
    arrow-up
    51
    arrow-down
    1
    ·
    4 days ago

    GitHub Microsoft just switched Copilot to metered billing, and developers are watching months of credits vanish in a single day

    • dan@upvote.au
      link
      fedilink
      English
      arrow-up
      21
      ·
      4 days ago

      They switched from heavily subsidizing it, to subsidizing it less. That’s going to happen with the other providers, too.

  • Pommes_für_dein_Balg@feddit.org
    link
    fedilink
    English
    arrow-up
    26
    ·
    4 days ago

    I have pro+ and I filed an FTC complaint when the billing change was announced and encouraged others to do the same, with the hope that it may lead to a class action settlement in the future.

    Good luck, buddy!

  • dan@upvote.au
    link
    fedilink
    English
    arrow-up
    25
    ·
    4 days ago

    I’m not sure why anyone is surprised. The new pricing is closer to what it actually costs to provide the service.

    They can’t keep subsidizing AI forever. The same thing is going to happen to other providers too.

    • Chulk@lemmy.ml
      link
      fedilink
      English
      arrow-up
      5
      ·
      4 days ago

      I think a lot of people thought that they were going to pay a flat monthly fee forever. This is just the beginning.

      I’ve been warning anyone who will listen that the metered token-based rates are the future for this industry. I’ve had coworkers dismiss my warnings, and now here we are. It’s gonna be a shit show once anthropic eventually switches over to token-based billing for services like Claude Code.

      • dan@upvote.au
        link
        fedilink
        English
        arrow-up
        3
        ·
        3 days ago

        There might continue to be flat monthly fees with some providers, but I can imagine the prices growing to at least 10x the current prices. At the moment, a $200/month plan with Anthropic or OpenAI lets you use thousands of dollars worth of tokens per month (and even that price might currently be subsidized) which isn’t sustainable.