Open vs Closed AI Models: Which Should You Build On in 2026?

The capability argument is effectively over, and almost nobody has updated their reasoning.

Epoch AI tracks model ability on a composite measure called the Epoch Capabilities Index. Since January 2026, the most capable open-weight models have trailed the frontier closed models by an average of four months, with an average gap of 8 index points — roughly the distance between GPT-5 and GPT-5.5 (Epoch AI, 2026).

Four months. For most products, that's shorter than the gap between your planning cycle and your launch.

So if you're still choosing closed models because open ones aren't good enough, you're solving last year's problem. And if you're choosing open ones because "open means transparent and cheap," you're about to discover that neither word means what you think.

Key Takeaways

• Open-weight models trail the closed frontier by ~4 months, or 8 ECI points (Epoch AI, 2026).

• The top US model led China's best by just 2.7% as of March 2026 (Stanford HAI).

• Transparency moved backwards: the Foundation Model Transparency Index average fell from 58 to 40.

• "Open weight" is not "open source," and self-hosting is usually more expensive than it looks.

How Far Behind Are Open Models, Really?

Close enough that capability is rarely the deciding factor anymore.

Epoch's comparison puts leading open models — GLM-5, Kimi K2.6, DeepSeek-V3.2 — clustering around 145–151 on the index, against 156–159 for the top closed models including GPT-5.5, Claude Opus 4.7 and Gemini 3.5 Flash. That's a real gap. It's also a gap that closes on its own if you wait a season.

The geopolitical dimension is closer still. Stanford's 2026 AI Index found the top US model leading the best Chinese model by 2.7% as of March 2026, and notes that US and Chinese models "have traded places at the top of the performance rankings multiple times since early 2025." DeepSeek-R1 briefly matched the leading US model back in February 2025.

Since many of the strongest open-weight releases now come from Chinese labs, those two facts are connected: the open ecosystem is competitive largely because a set of well-funded labs chose to release weights.

One honest caveat about all such comparisons — they're benchmark aggregates. A four-month average lag doesn't mean the open model is four months behind on your task. It may be equivalent for extraction and far behind on long-horizon agentic work, which is exactly where the frontier labs concentrate their effort.

"Open Weight" Doesn't Mean Open Source

This distinction trips up a lot of teams, and it has legal and practical consequences.

Open weight means you can download the parameters and run the model. You typically don't get the training data, the training code, or a full account of how it was made. Many of these models also ship under bespoke licences with use restrictions, not standard open-source terms.

Open source, in the sense the term has meant for thirty years, would include the ingredients and freedoms to rebuild and redistribute. Very few frontier-adjacent models qualify.

And here's the finding that should reset expectations. Stanford's Foundation Model Transparency Index — which scores disclosure about data, compute, and process — saw its average score fall from 58 to 40 year on year. Their summary is direct: "the most capable models often disclose the least amount of information."

So the industry got more open in one specific sense (you can download weights) while getting substantially less transparent in every other sense (you can't see what's in them).

Our read: this matters most for the people who invoke openness for compliance reasons. If your case for an open-weight model is "we need to know what's in the system," downloadable weights don't deliver that — you get a large opaque artefact you happen to control, not an auditable one. What open weights genuinely give you is custody: the model runs where you put it, can't be deprecated out from under you, and won't change behaviour because a vendor shipped an update. Custody is valuable and worth paying for. It just isn't transparency, and conflating the two produces architecture decisions that fail their own stated goal.

The Cost Question Isn't What You Think

On paper, open wins easily. One study of list prices across May to September 2025 found open-weight API access averaging around $0.23 per million tokens against $1.86 for closed models — roughly eight times cheaper.

But that's the price of open models served by someone else. It is not the cost of self-hosting, and teams routinely conflate them.

Self-hosting economics turn on utilisation. You rent GPUs by the hour whether or not requests arrive. An endpoint serving bursty traffic at 15% utilisation can easily cost more per token than a frontier API, because you're paying for idle silicon. Add the engineering time to run it — deployment, scaling, upgrades, incident response — and the crossover point is further out than most estimates assume.

That calculation also got harder in 2026, not easier, because GPU capacity tightened and rental prices rose rather than fell.

Three configurations, roughly:

The middle row is the underrated one. Most of the cost benefit of open weights is available without operating any infrastructure, and that's where a lot of teams should land.

What Actually Decides It

With capability roughly neutralised, these are the variables that matter:

That last row is the uncomfortable one, and it follows directly from the transparency data. Whichever way you go, you're unlikely to get a full account of what the model was trained on.

When Open Wins, When Closed Wins

Open weights are the better default when:

• Your data is genuinely prohibited from leaving your infrastructure.

• You're serving high, steady volume where per-token cost dominates.

• You need behaviour to stay identical for years — no silent updates, no deprecations.

• You're fine-tuning enough that owning the base model changes what you can build.

• Vendor concentration risk is a board-level concern.

Closed models are the better default when:

• The task sits at the edge of what any model can do. Four months matters when you're at the frontier.

• You have no platform team, and shouldn't acquire one for this.

• Traffic is unpredictable, so idle capacity would dominate your bill.

• Time to first working version is the binding constraint.

Notice that almost none of these are about benchmark scores. That's the point.

How to Decide for Your Project

1. Check whether you actually need the frontier. Run your real task on a leading open model first. If it passes, the capability question is settled and you can decide on other grounds.

2. Estimate utilisation honestly. Requests per second at peak and at 3am. Self-hosting only pays with a flat, high curve.

3. Separate custody from transparency. Write down which one your requirement actually needs, because open weights deliver the first and not the second.

4. Read the licence. Open weight is not automatically permissive; several carry use restrictions that matter commercially.

5. Build behind an abstraction. Keep model calls behind an interface so switching costs stay low. With a four-month capability gap, you will want to switch.

6. Re-evaluate every two quarters. At this rate of change, an eighteen-month-old model decision is not a decision, it's an inheritance.

[INTERNAL-LINK: what an AI feature actually costs to run → analysis of why AI bills rise as token prices fall]

Frequently Asked Questions

Are open models good enough for production?

For most tasks, yes. Epoch AI measures the leading open-weight models trailing the closed frontier by about four months and 8 index points. The practical test is running your own task on both, since aggregate benchmarks don't tell you about your specific workload.

Is self-hosting cheaper than paying for an API?

Only at sustained high utilisation. Open-weight models are roughly eight times cheaper on list prices, but that reflects hosted access, not running your own GPUs. Idle capacity plus engineering time frequently makes self-hosting the more expensive option for bursty traffic.

Does open weight mean I can audit the model?

No. You get the parameters, usually not the training data or code. Stanford's Foundation Model Transparency Index average fell from 58 to 40, and the report notes the most capable models disclose the least. Open weights give you custody, not auditability.

Are Chinese open models a security concern?

Weights you host yourself don't phone home, so the risk profile differs from an API. The real questions are licence terms, unknown training data, and your own compliance posture — the same opacity problem that applies to closed models, with the difference that you control where it runs.

The Decision Moved, So Should Your Reasoning

For two years the open-versus-closed debate was a capability debate, and closed won it. That's over. Four months and eight index points is not a moat; it's a release cadence.

What replaced it is a less exciting set of questions about custody, utilisation curves, licences and operational capacity. Those don't make headlines, and they're the ones that will determine whether your choice looks smart in eighteen months.

One thing worth carrying out of the transparency numbers, though. The industry is converging on a strange equilibrium: models you can download but can't inspect, and models you can't download and also can't inspect. Whichever you pick, you're building on something nobody outside the lab fully understands — so keep the abstraction layer clean and the switching cost low.

Sources