DeepSeek makes me optimistic

Most AI news makes me a bit nervous. A handful of huge companies with huge data centers, and the rest of us renting intelligence from them by the token.

DeepSeek is the exception. Every time they publish something I get a bit more optimistic.

They do more with less

DeepSeek works under export controls, without unlimited access to the best Nvidia chips. They can’t win by throwing more hardware at the problem, so they have to be smarter about it.

According to their Nature paper, the final training run of R1 cost about $294,000 on 512 H800 chips. That leaves out the base model and everything before it, and nobody has independently checked it. But even ten times that would be a rounding error for the big labs.

Their latest model, DeepSeek-V4.1-Flash, shows the same mindset. It has 552B parameters but only uses 8B per token while reading your input and 16B while writing. It handles a million tokens of context, and they squeezed the KV cache (the memory each token of context takes up) down to 890 bytes. That’s about a quarter of the previous version, and 437 times less than their first model. A small KV cache is what makes long contexts cheap, and it’s the kind of number that decides whether you need a data center or a desk.

The research actually ships

What I love most is how fast their research ends up in a model. Around New Year they published two papers: mHC, a way to keep training stable as models get wider, and Engram, a kind of memory the model can look things up in instead of computing everything. Both are inside V4.1-Flash now. Paper in January, in a model you can download the same year.

They publish the boring infrastructure too. During their open source week they released their kernels, communication libraries, pipeline tricks and 3FS, the filesystem they use to feed their GPUs. I wrote about 3FS’s FUSE hack earlier this year. It’s a team showing you how they actually build things, so you can do it too.

They even open-sourced the engine

Last month they went a step further and released DeepInfer, the inference engine behind their own API, under Apache 2.0. It runs on Nvidia’s newest GPUs and on Huawei’s Ascend chips, so it doesn’t depend on one chip vendor either.

They kept some things. The KV cache system and the load balancing you need for a full production service are still theirs. As one write-up put it, the engine is open, the cache is the business. Fair enough, it’s still way more than anyone else shows you.

Cheap AI can be profitable AI

DeepInfer came with my favourite number of the year. According to their Q&A on GitHub, one cluster serves 155 trillion tokens a day and makes about $38 per GPU per hour, at their cheapest off-peak price.

That’s revenue, mind you, and DeepSeek didn’t publish its costs. But analysts who plugged in typical GPU prices ended up at margins of roughly 80% or more.

That’s a big part of why I’m optimistic. Cheap AI might not be a loss-making trick to grab the market that disappears once the investors want their money back. With enough engineering, low prices and healthy margins can go together, so cheap might actually stay cheap.

Running it at home

Then there’s DwarfStar by antirez, the guy who made Redis. It’s a small inference engine written in C that runs a few excellent open models, DeepSeek V4 Flash first, on hardware people can actually own: a Mac with 96 GB of memory, a DGX Spark, a Framework Desktop. If you don’t have enough memory it streams the model from your SSD.

His reasoning, from the README: capable open models now fit on high-end personal machines, and compressed KV caches plus fast SSDs make long contexts practical. So DeepSeek’s KV cache work ends up paying off on your desk.

Fun detail: he says he built it with a lot of help from AI coding agents, with humans leading the ideas, testing and debugging. One of the best systems programmers around, building a local AI engine with AI. I find that a pretty good sign.

Open weights, under MIT

Every V4 model is on Hugging Face under the MIT license. You can download it, run it on your own hardware, fine-tune it and build a product on it. You don’t need an API key, the terms won’t change next month, and your data stays in the building.

For the companies I work with that’s a big deal. Plenty of them don’t want their documents going to an American cloud. An open model they can run themselves, or have someone run for them in Europe, makes that conversation a lot easier.

Good enough is enough

DeepSeek’s models aren’t always the best. The frontier models from Anthropic and OpenAI are still ahead on the hardest stuff, and I use them every day.

But for a lot of real work you don’t need the best model in the world. You need a good enough one that’s cheap, fast and under your own control. DeepSeek keeps showing that good enough is getting very good, and that it doesn’t have to come from the company with the most money.

So, optimistic

We’re probably not all going to end up depending on two or three providers. A smart team with limited hardware can still push the field forward, share how they did it and show that it pays. And then someone like antirez takes that work and puts it on your desk.

Open source did that for software. Nice to see it happening for AI too.