White House science adviser Michael Kratsios said Moonshot, the Chinese company behind the Kimi K3, the largest open-weight LLM available, built its model by copying Anthropic’s Fable LLM while using chips that are not authorized for export to China.

“Large-scale covert industrial distillation aimed at stealing patented American technology and undermining American research is unacceptable,” Kratsios wrote, amid discussions about a ban on Chinese open-weight models that have hit the AI ​​sector. Moonshot did not respond to questions about his training process and Kratsios did not share further details about the sources of his accusations.

Kratsios’ tweet echoed comments from Treasury Secretary Scott Bessent that “we are finding watermarks from our American large language models on many of the Chinese models, and that is unacceptable.” It’s unclear what those watermarks entail, and the Treasury Department did not respond to a query.

However, experts are skeptical that distillation (the process of consulting an LLM to determine its inner workings and copy its capabilities) is responsible for the advanced capabilities displayed by the Kimi K3.

“I don’t think you’ll get a model that strong and that fast after Fable did strictly distillation,” Braden Hancock, a researcher at the Laude Institute and co-founder of Snorkel AI, told TechCrunch. “Frankly, there’s not even time, right? Fable has only been publicly available since July 1. You can’t distill that much data, train a model, and publish it in two weeks.”

“I have been of the opinion that distillation has become less and less impactful over time as Chinese models approach the frontier and the training regimen shifts to [reinforcement learning]”Nathan Lambert, an AI researcher at the Allen AI Institute, said in a podcast published yesterday.”[I]If so, anyone could easily achieve a GLM or a K3 using their data for distillation. But we haven’t seen it, or we won’t see it, just with supervised fine-tuning.”

Performing distillation requires a lab to systematically query its target model to generate data that can be used for subsequent training. Sometimes this involves explicitly asking the model to articulate its chain of thought to understand how it solves problems. Other times, cues and responses from a model are used to train a new model in a process called supervised fine tuning or SFT.

It is this adjustment process that can result in a model seemingly created by a third party claiming to be Claude. In Lambert’s view, fine tuning is where “the model acquires its manners.”

But Lambert says the benefits of SFTs are becoming less important as models become more complex. Distilling Fable-like capabilities would likely require reinforcement learning techniques. In many cases, that means having an agent from the larger model score the responses from the smaller model and make adjustments based on the score.

More advanced techniques also require more significant infrastructure. Large reinforcement learning runs can require tens of millions of agents. Using a frontier lab’s API to do that “would be tremendously expensive and probably a time bottleneck because these models are quite slow and, to be honest, may not even improve performance.”

It seems likely that earlier frontier models contributed to Kimi; Anthropic publicly accused Moonshot, DeepSeek and MiniMax of systematically distilling its models earlier this year. Anthropic said it discovered millions of exchanges between its models and users that it identified at those companies through IP addresses and other metadata. Those queries were “distinct from normal usage patterns and reflected a deliberate extraction of capabilities rather than legitimate use.” Anthropic did not respond to TechCrunch’s inquiries about the Fable distillation.

However, distillation is considered common among AI companies, not just in China. Elon Musk testified earlier this year that his company SpaceXAI distilled OpenAI models to develop Grok and that the practice was common in the industry. The line between distillation and development of synthetic data sets, for example, can be quite blurry.

“[I]In general, Americans underestimate the technical expertise of these Chinese teams,” Hancock said. “One of the founders of Moonshot was a CMU doctoral student. These are legitimate researchers and engineers doing solid work. …if the American models were stopped, I think China’s progress would slow, but it would still continue. Here they are not only taking advantage of the opportunity.”

It’s also difficult to disentangle the second part of Kratsios’ comment: that Moonshot had obtained advanced Nvidia chips, the Grace Blackwell 300, and also accessed GB300-equipped servers in Thailand. Exporting those chips to China is prohibited, but there is a black market, according to Sam Bresnick, a researcher at Georgetown’s Center for Security and Emerging Technology. In May, the founder of Supermicro, an American server maker, was charged with smuggling advanced chips to China.

“I’m a fan of knowing your customers’ laws for data centers around the world,” Bresnick said. “If you allow a company to do large training sessions on your next-generation hardware, there needs to be a reporting mechanism about who that company is and what they’re doing.”

President Joe Biden’s Commerce Department proposed federal “know your customer” rules for data centers in 2024, but no further progress appears to have been made under Donald Trump’s administration. However, exporters who ship advanced chips abroad are supposed to ensure that they are only used for approved purposes.