Distilling The Moat

Whisky Still

The original function of a Web server was to respond to queries by revealing the appropriate part of their internal data. This necessarily meant that repeated queries, for example from a search engine's or an internet archive's web crawler, could extract the server's entire internal data. Since the extracted data had been published on the Web, it was protected by copyright. This has led to many lawsuits, for example against the Internet Archive, Google and others. It is the reason search engines only display "snippets" of the content they collect.

AI companies' intellectual property is their models. They spend vast sums funding the technical and human resources to "train" these models, the racks of GPUs in the data centers, and the hordes of workers labeling images, and having "genuine human conversations" with the nascent model. These expenditures are thought to create a "moat" around the value thus generated, because it would be equally expensive for a competitor to create an equivalent model. It is thhis moat that supports their extraordinary valuations, despite their lack of earnings.

Below the fold I explain why their moat is very shallow.

Like many Web services, the function of an LLM is to respond to queries by revealing the appropriate part of their internal data, i.e. part of their model. Thus, repeated queries could in principle replicate the model. These models are not published, thus not protected by copyright law [ 1 ], and cannot be patented. Their only protection is as trade secrets. Someone using repeated queries would probably be violating the system's terms of service, but this isn't a strong legal protection. The AI companies are not in a strong position to argue that "crawling" their Web servers is illegal because that is how they created their models in the first place, which they argue is fair use.

It turns out that replicating other models by repeated queries is a standard technique in the industry, called "distillation". Fo…