Taalas HC1: 17,000 tokens/sec on Llama 3.1 8B vs Nvidia H200’s 233 tokens/sec. 73x faster at one-tenth the power. Each chip runs ONE model, hardwired into the transistors.
The HC1 chip doesn’t load model weights from memory. It etches them directly into the transistors. Every weight becomes a physical circuit.
That’s one way to avoid memory bandwidth constraints!
This would be great if you could have a machine that would allow you to swap chips… and then they only charge < 50 USD for each chip.
Go landfills!
That’s all technology though, sadly.
This one feels shorter-lived than the average chip, tho.
With the hardwiring and all.
The thing that differentiates ChatGPT and Claude is likely more the RAG pipeline that backs them and feeds them context. The models really aren’t getting better, we’re just getting better at using them to break tasks down into units so small AI can figure it out. I’d bet a GPT 5 model or a Claude Opus 4.6 model would last 5, maybe 10 years before you really start to notice its capabilities are falling behind. I’ll bet you could use GPT 4o for 5-10 years and it would be fine.
But if they could make it so the chip is the only thing that is obsolete, That could be recycled pretty easily, or resold.
Then it would stop being 73 times faster than NVIDIA.
That doesn’t make sense.
If you add levels of indirection, extra transistors and such, it would be surprising to manage to maintain the same level of performance, especially since this design seems to rely on hardwiring to achieve its speed…
This sounds great to me. Anything that would increase supply of AI processing could lower demand on the GPU supply. I want to be able to upgrade my gaming computer again someday!
Every chip that is produced, takes away capacity that could have been used for consumer products.
So yeah…not great.
