Open Models Already Won the Builders. Here's What's Still in Their Way.

Based on the conversations we had at this year's AI Engineer World's Fair, the interesting question isn't whether to move to open weights, but how soon they're going to take over more mainstream usage...

Walk the floor at an AI builder conference in 2026 and you notice something the headlines haven't quite caught up to. A large proportion of the attending teams shipping a real product are optimizing for one or more open-weight models. Not as an experiment, and not to make a point about cost, but as their default. What struck us in those conversations was how the question had matured. Early on it was whether to use an open model at all; now it's which one to serve today, how to serve it, and, most interestingly, how not to get locked into it. The open-weight frontier moves fast, and the best model for a given job can change in a matter of months, so the teams furthest along had started designing for that churn: building against a stable, standard interface so they can swap the model underneath without rebuilding the product around it. Model-agnostic is becoming the default posture.

The usage data points the same way, provided we're being honest about whose usage it is, of course. In OpenRouter's study of more than 100 trillion tokens of real traffic, open-weight models have grown to roughly a third of token volume, and the share going to US-made proprietary models fell from about 70% to 30% inside a year. Of course, OpenRouter is a developer routing tool, so its users skew hard toward the people already inclined to shop models and try open weights the week they ship. That makes it a biased sample of the market as a whole, but we're really talking about a study of users who are building the future for the rest of us. Through this lens, the bias is the point: the population most fluent in this technology has already voted, well ahead of the enterprise average we'll come back to. We think that where the builders go, the average company will necessarily follow; it’s just difficult to determine on what timeline this will hit mainstream headlines.

Curiously, the broader enterprise average is moving in the opposite direction right now, and the reason is not that the models aren't good enough. The models are ready. What most organizations haven't solved is how to run them well and keep them private. That gap — the operational distance between "an open model exists that could halve my bill" and "I'm serving it safely in production" — is the whole story. It's a part of what Palantir's Alex Karp was gesturing at recently when he went after the token economy on CNBC. And over the last year, closing this gap got dramatically cheaper.

This piece is about that gap: why builders crossed it first, what's stopping everyone else, and how a company that assumed private AI was out of reach can now afford to run it.

Weren't we just told AI doesn't pay off?

If you remember spending late 2025 and even early 2026 reading that enterprise AI was a bust, you weren't imagining it. The most-cited number came from an MIT Project NANDA report that found only about 5% of enterprise generative-AI pilots reached rapid, measurable impact on the P&L while the other 95% stalled. That ‘bad news’ figure was spread widely, and in some circles it hardened into a story that AI simply doesn't deliver. This was a convenient data point for commentators in the business of calling “bubble” at various points in the ongoing AI narrative.

Headlines, however, are not really in the business of nuance, and a closer read of the study revealed something narrower. It measured custom, integrated deployments reaching sustained ROI within roughly six months, on the strength of about 150 interviews and a set of public deployments; several credible observers, including Marketing AI Institute's Paul Roetzer, pushed back that the method was opaque and the success bar excluded ordinary efficiency gains. The report's own most useful finding gets less airtime: purchased solutions succeeded about twice as often as internal builds, and the failures traced to a learning gap in workflow integration rather than to weak models. In other words, the pilots that flopped mostly flopped on the wiring, not the intelligence. (We also have thoughts on what constitutes a good vs. 'less good' rollout/adoption strategy in some of these stories…)

Two facts sit awkwardly against the "AI doesn't pay" narrative. Even after the MIT report came out (roughly a year ago as of this writing,) enterprises kept spending — Menlo Ventures put 2025 enterprise AI investment at $37 billion, roughly triple the prior year. And a good deal of the early "it's cheap and it works" optimism had been running on subsidy: the cloud and model providers hand young companies large credit grants — AWS Activate up to around $200,000, Google for Startups up to $350,000 over two years — that typically expire on a 12–24 month clock. For an AI-native company, that window can mask the true cost of inference almost entirely, right up until it doesn't. The teams that kept going, however, didn't conclude AI was overhyped. They went looking for a cost structure they controlled. So, there was never really a “what” problem to be resolved in the ROI story, just one or two “how” problems.

Have builders standardized on open models? If so, then why?

Because two curves crossed. Open-weight models got good enough, and they stayed cheap enough that the difference stopped being ignorable.

The turning point is easy to date: In January 2025, DeepSeek released R1, a reasoning model Stanford's researchers judged roughly on par with OpenAI's o1 at around 90% lower cost and allegedly "trained for pocket change." More than one smart commentator went so far as to label the launch, “China’s Sputnik moment.” The market took it literally, and Nvidia lost about $589 billion of market value in a single session, the largest one-day drop for any company to that point. What DeepSeek proved wasn't purely about economics though. It was that frontier-adjacent quality could come from an open checkpoint you could download and run yourself, and that even if you thought that Deep Seek “wasn’t quite there yet,” it was obvious that it soon would be.

Since then the quality gap has kept closing. Menlo's mid-2025 read had the best open models trailing the frontier by nine to twelve months; by mid-2026, OpenRouter put the lag at three to six months, with several open models landing within a few points of the top closed systems on independent benchmarks like the Artificial Analysis Intelligence Index. The cost spread, meanwhile, never narrowed. Leading open-weight models run somewhere between roughly 10× and 100× cheaper per token than the top closed APIs, depending on the model and who hosts it — a capable open model can sit near $0.27 (or lower!) per million input tokens where a flagship closed model sits at $5 and up, with output tokens spread even wider. Treat those exact figures as a snapshot; the direction is the point here.

This is where a first-hand data point is a helpful illustrator: A vertical SaaS company we spoke with just a few days ago cut its inference costs by more than 40% after moving core workloads onto open-weight models. Its engineering lead told us that building the optimization layer in-house took them further still — to around a 90% reduction on the workloads they tuned. That is one team's result on their traffic, not a number you should expect to inherit. The point here isn't the specific percentage, it's that the lever is real and large enough to reorganize a budget around and that the improved economics are arriving rapidly. Their profile also explains why the shift shows up among builders first: they had the engineering depth to capture it.

One honest note on the usage data: a large share of open-weight token volume today is creative and roleplay traffic, with coding assistance the other dominant use case. "A third of tokens are open" is not the same as "a third of serious enterprise work is open." But coding agents are exactly the high-burn, cost-sensitive workload where the economics tend to hit the hardest, which is precisely why the builders felt it first and why we think the surface hasn’t even started to be scratched on valuemaxxing.

If open models are cheaper and nearly as good, why hasn't everyone switched?

Because the broader enterprise hasn't, but also because it's hard for some teams.  Menlo's surveys show enterprise open-source adoption actually falling through 2025, from around 19% of workloads to somewhere between 11% and 13%. So in the same year that many builders were standardizing on open weights, the average large enterprise was consolidating onto a handful of closed APIs.

Both things are true, and the contradiction is the insight. Enterprises didn't stay closed because the open models were bad. They stayed closed for two reasons Menlo names directly: open models are harder to deploy and operate, and there's a trust question that a model being "open" doesn't automatically answer. The builders had the engineers to handle the first and a higher tolerance for the second. Many organizations have neither. So the value is sitting there, visible and unclaimed, behind an operational and psychological moat.

What are companies actually afraid of when they say they don't "trust" it?

Karp has a knack for creating soundbites, and put the fear in its most quotable form. On CNBC around July 1, 2026, the Palantir CEO argued that enterprises paying per token to frontier labs were getting "no value" while handing over what he called "the weights and alpha" of their business.  He asserted that what companies really want is "control over their compute, their models, their data stack." While Karp was self-admittedly talking his own book, he was still pointing at something real, and "trust" turns out to be three different fears wearing one coat.

Fear one: they're training on my data. This is the loudest fear, but also possibly the weakest. In his interview, Karp appears to imply that investors are content to continue letting the major models hemorrhage money because, similar to negative narratives about social media, the product isn’t necessarily the product, and that companies using these models may be “handing over their alpha.” Let’s be fair though: on their enterprise tiers, the major providers say plainly that they don't train on your inputs by default and OpenAI, Anthropic, and Google all commit to it in writing. The honest concern isn't that they're secretly training on you; it's that you're relying on a contractual promise and a retention window, rather than on architecture, and that the consumer tiers of the same products play by more permissive rules. A promise is only as good as the company keeping it and the terms they can revise. Running the model inside your own boundary replaces the promise with a property of the system: the data doesn't leave, because there's nowhere for it to go.

Fear two: the model gets pulled out from under me. This one is concrete and underrated. Closed models are retired and changed on the provider's schedule, not yours. If your product's behavior is tuned against a specific model, that's a dependency you don't control. An open checkpoint you host doesn't get deprecated. You run the exact weights, unchanged, for as long as you choose. This is the cleanest, least arguable leg of the whole trust case.

Fear three: data sovereignty. For a growing set of organizations the question is jurisdictional — where does the data physically sit, whose law can reach it, who holds the keys. The Linux Foundation's 2025 sovereign-AI research found data sovereignty and control the single most-cited driver of interest, named by roughly 72% of respondents, with about 81% treating open-source software as critical to a sovereign stack. This is the regulated-industry and public-sector version of the trust problem, and it has its own hard rules. (We go deeper on the European side of it in our piece on AI data residency under GDPR and the EU AI Act.)

Open weights fully answer fears two and three on their own because nobody can retire a model you hold, and a model you run in-region satisfies residency by construction. But open weights only answer fear one if you also control where the model runs. Downloading an open model and then serving it through someone else's multi-tenant API gives back most of the privacy you were trying to win. Which is the practical definition of the thing enterprises are reaching for.

What is private AI, exactly?

Private AI is running open-weight models on infrastructure you control — self-hosted on owned or rented GPUs, in a dedicated single-tenant deployment, or inside your own virtual private cloud — so that the data, the model weights, and the physical location all stay inside your trust boundary. Your prompts and outputs never transit a third party's shared service. The privacy isn't a policy you're trusting; it's a property of where the compute lives.

That's the whole idea, and it's why "open" and "private" get conflated but aren't the same. Open is a property of the model. Private is a property of the deployment. You need both to answer all three fears at once, and the deployment half is the part that used to be expensive.

Can a smaller company actually afford to run private AI now?

Until very recently, for most teams the honest answer was no. Today it's yes, but with an asterisk about who does the operating.

Three things have changed. Compute got cheaper: a specialist GPU cloud now often rents an H100 for around a third to a fifth of what the hyperscalers charge for comparable usage. The pricing piece, however, is highly dynamic and not the most interesting, nor the most stable part of the accessible private AI story.

Also, the software got commoditized: open serving stacks like vLLM and SGLang will stand up an OpenAI-compatible endpoint for an open model in about one command, and they're free. And a layer of managed, dedicated-endpoint providers grew up in between, so a team can get single-tenant serving without building the plumbing themselves.

But it's not entirely this simple. Getting a model running is easy now; getting it running well in production is still real work. Serving a demo that turns a first-time vibe-coder into a founder is not a big stretch, even for the less technical among us. Past that point, however, serving business traffic economically means capacity planning, batching and concurrency tuning, quantization choices, KV-cache and memory management, observability, and failover — and the difference between a default configuration and a tuned one is not small; independent benchmarks routinely find naïve setups leaving a quarter or more of their throughput on the floor. That's exactly why the 80–90% results show up at teams with a strong infrastructure engineer or two to spare. The model is free. The expertise to extract the savings is not, and it's the scarcest thing on the market right now.

So the real decision for a smaller organization isn't open versus closed. It's build versus buy the operating layer. A modest production deployment might fit on anywhere from one to eight modern GPUs — call it low thousands to low tens of thousands of dollars a month in raw compute, before you count the engineers — and whether that math beats a closed API depends entirely on your volume and how well the thing is run. If you have the team and the traffic, building it in-house can pay for itself fast, as our earlier example shows. If you don't, the open-model savings can evaporate into ops overhead and idle GPUs. Reserved, owner-operated GPU capacity changes that arithmetic, because the cost floor moves when you're not marking up someone else's hardware — but the operating expertise still has to come from somewhere.

How do you get open-model economics without hiring an inference team?

We'll be direct about our interest here: the reason we've gone this deep on the open-weight shift is the thing we're working on. Everything up to this point is the problem we set out to solve: capable teams can see the savings sitting there, but many still can't reach them because the cost that stays stubborn is the engineering to run open weights well. That's the part we're productizing.

Aurora is opening fixed-cost, optimized inference for open-weight models. The aim is to give a team the economics that the builders captured by hand, and the privacy and control that Karp was flagging, without needing to hire the infrastructure group that usually comes attached. Cost is part of it, but the thing we're actually selling is a bill that doesn't move with your usage and a deployment that stays inside your boundary.

If you're weighing this move and you'd rather compare it on real numbers than take anyone's headline figure, we're opening early access to the first teams now.  We're giving the first teams on the list allocation priority as we open up. It takes a work email and one question about your current inference spend.