Except the people I hear running “local models” report a lot of hardware, like a half dozen graphics cards. The common person is screwed, they have a smartphone.
Two or three years ago sure. Now…not even close. Tons of models run on basically shit now. Especially with MOEs. I run new lightweight models on an i9 and 64gb of ddr4 ram with a 4gb quadro card…in a laptop from 19’.
I think this is why a lot of these tech bros are saying silly shit about suggesting people need to rent from them. That’s their entire shtick. Otherwise they got nothing else.
If you’ve got at least 8 gigs of VRAM and at least 32 gigs of RAM you can run some pretty beefy Qwen models using Strata, far more than you normally could. Upgrade that to 12 or 16 and 64 gigs respectively and you can run a pretty decent quant of Qwen 3.8 flash next. It’s a LOT bigger than the usual 27 and 35 billion models people run locally that themselves require pretty beefy hardware. Strata is doing some magic by deciding which experts stay in VRAM vs which ones are in RAM vs which ones are so rarely used that they can just be read from the SSD as needed (though it’s best if you don’t have to result to the SSD as a backup option at all)
The federal debt and the US treasuries part of the fed balance sheet is how much taxes should increase. They’re different things, the fed can just buy bonds and put them on its balance sheet.
Government bales out AI companies and the bill they write to do it will make local LLMs illegal.
Piracy is illegal and they are of course doing a great job at stopping that…
Reverse engineering isn’t piracy
Unless you’re training your models…
I think you are right, but that seems VERY unenforceable.
Successful enforcement doesn’t look like catching every pivate LLM owner.
Except the people I hear running “local models” report a lot of hardware, like a half dozen graphics cards. The common person is screwed, they have a smartphone.
Two or three years ago sure. Now…not even close. Tons of models run on basically shit now. Especially with MOEs. I run new lightweight models on an i9 and 64gb of ddr4 ram with a 4gb quadro card…in a laptop from 19’.
I think this is why a lot of these tech bros are saying silly shit about suggesting people need to rent from them. That’s their entire shtick. Otherwise they got nothing else.
Got some references and keywords for research?
If you’ve got at least 8 gigs of VRAM and at least 32 gigs of RAM you can run some pretty beefy Qwen models using Strata, far more than you normally could. Upgrade that to 12 or 16 and 64 gigs respectively and you can run a pretty decent quant of Qwen 3.8 flash next. It’s a LOT bigger than the usual 27 and 35 billion models people run locally that themselves require pretty beefy hardware. Strata is doing some magic by deciding which experts stay in VRAM vs which ones are in RAM vs which ones are so rarely used that they can just be read from the SSD as needed (though it’s best if you don’t have to result to the SSD as a backup option at all)
I saw the video, you still need a fast NVMe, I’m still in data SSD…
https://youtu.be/8F_5pdcD3HY
This guy does a great brain dump on how little is needed to run LLMs on basically old junk.
Kudos
Local LLMs will never be illegal, because that genie is out of the bag
Bails
With what money? No one is buying bonds. The US would have to print so hard it’d cause hyperinflation, which still means the bubble pops
The fed balance sheet gets bigger like it did with AIG.
Again, how? No one is buying American bonds right now. You can’t raise the debt if no one will loan you money
The federal debt and the US treasuries part of the fed balance sheet is how much taxes should increase. They’re different things, the fed can just buy bonds and put them on its balance sheet.
How much taxes should increase?
Wait, is this just printing money with extra steps? Cause it sounds a lot like printing money with extra steps