Ben Thompson, on the hype regarding Kimi K3, at Stratechery:

This is a point that bears repeating: because U.S. open weight model makers must follow the frontier labs’ terms of service, they (1) are worse than Chinese alternatives and (2) end up distilling the distillation, just with a detour through Chinese labs. Wouldn’t it be better if western open weight model makers could go to the source?

To that end, here’s an even more interesting question around distillation: why exactly is it bad? After all, what are large language models but the distillation of all of the knowledge on the open Internet, scraped by the frontier labs and distilled into the models that are themselves being distilled? Who is exactly being wronged here?

In fact, this paradox is the solution. I believe that open weight models are good for innovation (and, per the above, I think that labs on the frontier will be fine), but it’s a problem to be dependent on China. The U.S. should pass a law that (1) makes explicit that collecting data for training models is fair use, and (2) bars terms of service that forbid distillation, for U.S. companies at a minimum. Stopping distillation — which is literally just querying the API — is nearly impossible; the U.S. should go the other way and lean into a new copyright policy that both indemnifies the labs and also guarantees that what they learned fuels further innovation for everyone else.

To be clear — and Thompson emphasizes this point too — the leading Chinese models like K3 aren’t good only because of distillation. They’re not entirely rip-offs. But distillation is clearly an essential part of the formula they’re using to keep releasing models that are 6–9 months behind the U.S. frontier state of the art. So the Chinese treat all models as “open”, regardless of the terms of service. But anyone in the U.S. or other western countries that respect copyright — or contracted terms of service — who wants to distill a good model has to wait for the Chinese to release their models, like K3.

And that second paragraph I quote above distills (sorry) exactly why I want to bring out the world’s smallest violin to play a sad song for OpenAI and Anthropic regarding their objections to their frontier models being distilled against their terms of service.

Ben Thompson, in his weekly free column at Stratechery:

On one hand, I actually don’t begrudge Anthropic not wanting to help its competitors; on the other hand, what should be blisteringly clear is that Anthropic does not think that anyone else other than them should even be making frontier LLMs.

What makes this policy all the more remarkable is the fact that it was enacted only two months after Anthropic had that dispute with the Department of War: the latter wanted to use Claude for any legal use, while the former wanted more stringent controls around surveillance and autonomous weapons. What this degradation represented was both the capability and willingness of Anthropic to silently alter its models to achieve its policy preferences. In other words, Anthropic willfully validated some of its critics’ worst fears in terms of being a supply chain risk.

The broader takeaway from that previous episode, however, is that Anthropic believes that they are the ones who should have final say over how Anthropic is used; given that they think only they should be developing leading edge AI, they by extension think that only they should have final say over AI generally. When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone.

Anthropic is best seen as a religious organization. Their employees are true believers in a cause, and on a mission. Perhaps every successful company has a religious aspect at its core — like, maybe, Apple’s is design quality and user-centricism, Microsoft’s is market share with no regard for technical or design elegance, Google’s is market share with high regard for technical elegance, and Meta’s is strip-mining the world’s social graph for profit. These companies tend to attract employees who believe in the company’s core mission, and the employees who believe tend to be the ones who thrive and rise within the companies’ ranks to positions of influence.

But Anthropic feels more like a real religion, where the core tenets must be taken on faith, and the priests (Anthropic employees) have a conviction about them. A religious fervor. If Apple gets too taken away by its cultural fervor for design, they do something silly like make a $20,000 solid gold Apple Watch. So what? If Microsoft or Google get taken away by their shared fervor for market share at all costs, they face antitrust remedies. A stifled market and abusive behavior from a monopolist isn’t good, but doesn’t end the world.

A religious fervor that believes the company is building god-like “super intelligence” that will dwarf human intelligence — and that only the company’s priesthood can be trusted to define, create, control, and gate access to it — is something else entirely. I tend to think the Anthropic true believers are all wet — that LLMs, amazing though they are, are not a path toward “super intelligence”. But, they used to be clearly behind OpenAI in technical capability, then caught up, and now with Mythos/Fable, they are clearly ahead. I still think they’re wrong about where this is heading, but I don’t think we can say we know they’re wrong.