Model selection for self-hosting has become a procurement problem rather than a research problem. The open-weight field in 2026 is deep enough that several families will clear your quality bar, which means the decision turns on licence terms, hardware footprint, and how well the model fits the specific work you have, not on which model tops a leaderboard this quarter.
The families worth evaluating are Llama 4, Qwen 3 and 3.5, DeepSeek V3 and R1 and their successors, Mistral (particularly Small 3.1 and Large 3), and gpt-oss. Each has a distinct profile, and the differences that matter in production are not the ones the benchmark tables emphasise.
The most common failure we see is a technically excellent selection that legal cannot approve, discovered after the integration work is done. The second most common is a model chosen on aggregate benchmark position that underperforms on the actual task mix, discovered after users start complaining. Both are avoidable with an evaluation sequence that puts the cheap disqualifying checks first.
The families and what distinguishes them
- Llama 4. The broadest tooling and community ecosystem, with open-weight multimodality and context handling in the 1M-token class. It is the safest choice for ecosystem maturity and the least clean on licensing, because Meta’s community licence is not Apache or MIT.
- Qwen 3 and 3.5. The widest size ladder in the field, commonly released under Apache 2.0, which makes it the easiest family for commercial self-hosting and fine-tuning across hardware tiers. When teams ask for a default, this is usually the answer, particularly where hardware is constrained.
- DeepSeek V3, R1, and later releases. Strong on reasoning and code efficiency, with MIT licensing on the models most 2026 guidance points to. The caveats are version-specific: some flagship variants are text-only, and long-context or deployment behaviour varies between releases.
- Mistral. The choice where European vendor provenance carries weight, which for EU-regulated clients is a procurement factor rather than a preference. Mistral Small 3.1 is an Apache 2.0, multimodal, 128K-context option that fits comfortably in single-node deployments.
- gpt-oss. OpenAI’s open-weight family, Apache 2.0 in current listings, with a 120B-class model showing strong benchmark performance and 128K context. Treat it as a serious self-host candidate subject to model-card terms.
Licence terms are the first filter
Open-weight is not a synonym for permissive open source, and this is the check to run before any engineering time is spent.
| Family | Licence posture | Commercial use | What to verify |
|---|---|---|---|
| Llama 4 | Meta community licence | Allowed, conditionally | 700M monthly active user threshold triggers a separate agreement; “Built with Llama” attribution applies in some contexts; acceptable-use policy applies |
| Qwen 3 / 3.5 | Commonly Apache 2.0 | Broadly allowed | Confirm the specific release, since the family spans many variants |
| DeepSeek V3 / R1 and later | Commonly MIT | Broadly allowed | Check for usage-policy overlays on the specific release; confirm whether the variant is text-only |
| Mistral Small 3.1 / Large 3 | Apache 2.0 on selected releases | Broadly allowed | Confirm the exact variant, since multimodal and larger releases can differ |
| gpt-oss | Apache 2.0 in current listings | Broadly allowed | Review model-card-specific restrictions against your policy stack |
The 700M monthly active user clause in the Llama licence sounds irrelevant to most enterprises, and usually is. Where it bites is embedded distribution: if the model ships inside a product with a large end-user base, or inside a platform your customers’ customers touch, that threshold and the attribution requirements become live questions. Have legal read the actual licence for the actual release, and record the decision, because the same question will be asked again at the next model refresh.
Acceptable-use policies deserve equal attention. They are shorter than the licence and easier to breach, particularly for organisations doing anything the policy classifies as high-risk. For air-gapped work there is an additional test: whether the terms permit mirroring the weights into an offline environment at all.
Reading benchmarks without being misled
Public benchmarks are a legitimate shortlisting tool and a poor decision tool. Four caveats matter in procurement.
Leaderboard scores are not deployment performance. A model that scores excellently in the evaluator’s configuration can underperform under your quantization, your batch size, your context length, and your latency budget. INT4 quantization in particular shifts relative rankings, and INT4 is how most enterprises actually run.
Context-window claims are optimistic. An advertised 1M-token window is a capability statement, not an operating point. Once retrieval overhead, prompt size, KV cache pressure, and throughput trade-offs are accounted for, the enterprise-practical window is much smaller, and the cost of running near the advertised limit is usually prohibitive.
Hardware fit reorders the table. Models that win on aggregate scores often assume multi-GPU inference. Smaller Qwen and Mistral variants, or distilled DeepSeek models, frequently deliver better real-world cost per unit of quality on a single node, which is where most private deployments live.
Task mix decides the winner. One family leads on reasoning, another on code, another on multilingual work, another on multimodal input. Choosing by aggregate leaderboard position averages away exactly the distinction you need.
An evaluation sequence that holds up
Run the cheap filters first and the expensive tests last.
- Filter by licence. Eliminate anything whose terms conflict with your commercial, redistribution, attribution, or regulatory position. This costs a lawyer an afternoon and prevents the most expensive category of mistake.
- Match size to hardware. Decide whether you are on a single GPU, multi-GPU node, or cluster, and eliminate models that do not fit at your target quantization with realistic KV cache headroom. This usually cuts the field in half.
- Test on your own workload mix. Build an evaluation set from real tasks: internal-document retrieval, code review, multilingual summarisation, tool-use workflows, long-context extraction, and whatever else your users actually do. Score it the way you will judge production quality.
- Measure operational metrics alongside quality. Latency at p95, tokens per second under concurrency, memory footprint, failure rate, quantization loss against the unquantized baseline, and cost per million tokens. Leaderboards capture none of these, and they determine whether the deployment is affordable.
- Run canary and shadow deployments. Compare candidates against current production traffic and human judgement on hallucination rate, formatting reliability, refusal behaviour, and safety. Benchmark gains do not reliably translate into better operational behaviour, and shadow traffic is where that gap shows up before users find it.
Shortlist by use case
| Use case | Better-fit families |
|---|---|
| General-purpose internal assistant | Qwen 3/3.5, Llama 4 Scout or Maverick, Mistral Small 3.1 |
| Coding and developer copilots | DeepSeek reasoning and code variants, Qwen Coder family, gpt-oss |
| Long-document retrieval | Llama 4, DeepSeek V3-class, large-context Qwen variants |
| Multilingual workloads | Mistral and Qwen families |
| Strict commercial permissiveness | Apache 2.0 and MIT releases: Qwen 3/3.5, selected Mistral and DeepSeek, gpt-oss |
| Maximum ecosystem maturity | Llama 4, accepting the licence constraints |
How we would approach it
For most enterprises the defensible default is a permissively licensed family, Qwen 3 or 3.5 or an Apache 2.0 Mistral or MIT DeepSeek release, sized to run comfortably on hardware you already have or can procure without a special case. Keep Llama 4 for situations where its ecosystem, multimodal support, or context handling outweighs the licence conditions, and treat gpt-oss as a genuine candidate once its model-card terms clear your policy review.
Then build the evaluation set before you build anything else. Fifty to two hundred real tasks with agreed scoring is enough to separate candidates, and it becomes a durable asset: the same set re-runs against the next release, which turns model refresh from a debate into a measurement. Given how fast this field moves, the ability to re-evaluate cheaply is worth more than getting the first choice exactly right.
Finally, keep the roster small and put it behind a gateway. Two or three models routed by task gives you the coverage that a single choice cannot, without the operational drag of maintaining a model zoo that no one has evaluated since the day it was deployed.
FAQ
Does open-weight mean we can use the model commercially without restriction?
No. Open-weight describes availability of the weights, not the licence attached to them. Qwen releases are commonly Apache 2.0, DeepSeek releases are commonly MIT, and selected Mistral and gpt-oss releases are Apache 2.0, all of which are broadly permissive. Llama 4 uses Meta's community licence: commercial use is allowed but conditional, with a 700 million monthly active user threshold that triggers a separate agreement and attribution requirements in some contexts. Check the exact release rather than the family, because terms differ between variants.
How much should we trust public benchmark rankings?
As a filter for a shortlist, not as a decision. Leaderboard scores are produced in conditions that rarely match your deployment: your quantization, batch size, context length, and latency budget all change the ranking. Advertised context windows are particularly optimistic, since a 1M-token claim is not the enterprise-practical window once retrieval overhead and KV cache pressure are accounted for. Use benchmarks to narrow to three or four candidates, then decide on your own task mix with your own evaluation set on your own hardware.
Should we standardise on one model or run several?
Several, routed by task, is the pattern that holds up. Different families win on different work: reasoning and code, multilingual summarisation, long-document retrieval, and agentic tool use rarely have the same winner. Running two or three behind a gateway lets you route by workload and swap a model without touching applications. The cost is operational, since each model is another artefact to evaluate, patch, and monitor, so keep the roster small and deliberate rather than accumulating models as teams discover them.