I don't expect Tan's position to be based on any kind of real moral high ground, but his conclusion is correct.

I love the "illicit distillation attacks" framing from the incumbents. There's nothing illicit. There's no attack. You just don't like it because it threatens your market position and business model.

He also notes that the proprietary AI labs didn’t ask permission when they vacuumed up as much human knowledge as they could to train their models.

I think this should desactivate the moral high ground from which Anthropic is trying to speak. That they would want to make distillation orderly IMHO is fair, but to make it illegal is very rich from any AI frontier lab, really.

It's also clear that, as Tan indicates, open-weight models will be (and basically already are) just as good as frontier models. It's all about the harness, baby. We will have two main forks in the road, and two new industries created:

    - AI hardware (NVidia/Cerebras/etc.), the equivalent of Intel/AMD
    - AI software (harnesses, assistants, etc.) the equivalent of Microsoft/Apple

I do not agree with this man all that often, but that is very concisely put.

To him, the true AI doomer scenario is for all the immense power of frontier AI to wind up in the hands of a single powerful, proprietary provider. “The nightmare scenario, the doomer scenario for AI is that there’s just one company,” he said. “It has the best access to capital. It has the best AI researchers. It runs away with it and suddenly there’s one company that’s monolithic. And that would be bad.

Well yes, as I think I said in a previous comment, on the current trajectory OpenAI and Anthropic will really stop releasing models due to distillation and regulatory pressures. Then, they would eat all knowledge work themselves, which would be the end of YC.

Garry Tan and Sam Altman recently did this interview together. They seemed pretty friendly with each other during it. Wonder what Sam Altman would say about Tan advocating for OpenAI’s models to be distilled.

Then again this is the same OpenAI that has gotten into legal trouble recently regarding Apple’s IP so who knows

Back when people made arguments for software privacy, the argument was usually "big business will still pay and consumers wouldn't have paid anyways so it's ok for us to pirate" - I actually think that was fine for business software but terrible for indie games, whose market was 0% businesses.

But in the AI case, it's not like they get to keep some of the value of their investment - it all gets cloned into models that businesses and consumers alike are happy to use. If someone knows how labs could continue to fund data creation and acquisition in this model, please do share!

Preventing token-consumers from developing competing products should be litigated as anti-competitive behavior.

To him, the true AI doomer scenario is for all the immense power of frontier AI to wind up in the hands of a single powerful, proprietary provider.

Isn't this exactly what Dario wanted? He thought he knew what's best for the humanity...

Same thing as Cliff Notes imo. In every other area of manufacturering and tech I can use a machine to build a new machine that competes with the original machine. Should Milwaukee be able to prevent DeWalt from using their drill to make a competing drill? Should Jetbrains ban Eclipse contributors from using their IDE?

It's a good idea. It's not their data to begin with, anyway.

The fact that billions were spent on research to distill the internet should not preclude others from spending 10s of thousands to do the same to these frontier labs.

Time to create a bigger moat than "but we spent so much money doing this ...".

I think that software execs should not incentivize users or other execs to break Terms of Services, or contracts of any kind.

An executive or manager of a company that breaks contracts is worth 0, there's no incentive to do business with them, if you know they will agree to doing or not doing something and then breaking that promise.

The word of a businessman is their most valuable asset, Tan is signalling that he is either misinformed on what Chinese distillation consists of, or that it's ok to do it.

FAQ:

- "But the frontier models do bad things too"

- An argument worthy of a 5 year old, one civil issue doesn't negate the other, bring it to a court if you have an actual claim against OAI or Claude, etc...

- "Companies have the right to reverse engineer"

- Ok, do it, but the moment you are creating 10K accounts in a Distributed fashion (Distributed as in the first D of DDoS), using IP proxies and stolen credit cards or your employees and employee family credit cards, you are not doing it because you believe you have a right, you are doing it despite not having a right to it.

EDIT:

Re(actually)reading the article, Tan's take is a bit more nuanced, he seems to be advocating for regulation to restrict the capacity of Foundation models to restrict usage, on the basis (or to the extent) that it was trained on public data, and therefore it belongs or attributes its success to a wealth of the commons.

My pre-existing quip is against those that want to solve this as-is by breaking the ToS. I think that's a weak version of Free Software position, it's very weak to complain that some software is proprietary and want to use it anyway, the strong FS position is that you don't even want to use it if it's proprietary, you won't catch a FS activist pirating proprietary software, they just don't use it and develop alternatives. Similarly it's not a FS position to distill a proprietary model (where you still wouldn't have source code at any rate).

He elaborated to TechCrunch that this means he wants smaller, American open-weight AI labs to use the same kind of training techniques on American frontier AI labs, giving the U.S. a more robust set of open-weight options that aren’t Chinese.

Those market entrants would face commodity pricing power vs. high capital costs, no?

Maybe there'd be ROI but I think there's another layer or competitive dimension that's neither frontier lab nor distilled model lab.

- theft / distilling (ie cheaper open models that are not from the soon to IPO US corps cheated rather than innovated)

- danger / nat security (ie we can only trust the soon to IPO US corps to shepherd us)

Following news of companies and projects increasingly moving to open weights models.

As AI gets more central to society, we really need to know how the weights were determined.

Open weights isn't just "free as in beer"; it can be "free as in the mystery drug that creepy guy chatting you up at the bar offered you". And maybe even he doesn't even know everything that went into the tablets, since he too was being worked, by an organ-theft ring who will be harvesting both of you tonight.

That's an analogy to get your attention. Your LLM probably isn't going to steal your organs. But in the current environment, it does and will have ideological biases determined by those with direct and indirect influence over it. And there will be a massive market for commercial influence biases (look at how previous generations of adtech invaded almost all technology companies). And there's incentive for military and spying capabilities to be buried in the models, perhaps as long-term sleepers. Maybe some organized crime trojans, too, depending which model you pick up.

In this low-trust environment of the current real world, we need genuine open source models, not closed "open weights", and not mindlessly distilling black boxes gifted by sketchy powerful interests.

Ban data brokers before you ban distillation.

Genuinely fucking crazy we pay money for fast access to autocomplete of stolen human remains.

At the end of the day they were built from data that did not belong to them. So it would be fair that humanity REQUIRES to give back the output of that.

It's a bit like the free software thing: you can still make money from it and providing service to it, but if you build it based on another free stuff the derivative should be free.

Why not do the same for intelligence ?

Similarly, AI companies should be required to allow distillation at a fair price. Fair Use doesn’t make sense as a social contract if it only cuts one way!

They know it won't happen, so arguing for it is 'effectively free' and purely personal marketing.

A bullshit game played by politicians and wannabes.

I would love for a US lab to be at or near the frontier with an open weight model, but it’s going to take some serious elbow grease, and yes some distillation (which btw OAI, anthropic et al, also use distillation of other’s outputs in their training)