GPU procurement for private AI

GPU procurement has become the point where private AI programmes either get funded or quietly stop. The technical work is scoped, the architecture is agreed, and then a number appears that no one in the room has a reference point for, attached to a delivery date nobody can confirm. Finance asks whether it should be capex or opex, infrastructure asks where the power is coming from, and the answer to both is that it depends on a utilisation figure the team does not have yet.

The useful starting position for 2026 is that renting or reserving is the default, and buying is the exception you have to justify. Cloud GPU and GPU-as-a-service tends to be cheaper below roughly 60-75% sustained utilisation once power, cooling, staffing, spares, and depreciation are counted. Most enterprise AI workloads, especially in their first two years, run nowhere near that. They are bursty, experimental, and heavily weighted towards evaluation and iteration rather than steady production inference.

That default is not an argument against private infrastructure. It is an argument about how you obtain it. Dedicated capacity, private pods, and sovereign regions all give you isolation and residency guarantees without a purchase order, and the sovereignty requirement that drove the project rarely requires that your organisation holds title to the hardware.

What the hardware actually costs

Street pricing is worth knowing because it anchors every other conversation, including the rental quotes.

• H100 sits at roughly $25,000 to $40,000 per GPU in 2026, with the spread driven by PCIe versus SXM5 form factor and by vendor channel.
• An 8x H100 SXM5 node comes in around $250,000 to $400,000 all-in for the server itself, before networking, racks, or facility work.
• H200 runs approximately $30,000 to $40,000, a premium of about 15-20% over H100.
• B200-era capacity does not have transparent list pricing worth quoting. Supply remains constrained and deals are typically negotiated as dedicated capacity or rack-level agreements rather than per-unit purchases.

The node figure is the one to hold onto. A single 8-GPU server is a third of a million dollars of equipment that depreciates on a schedule shorter than most finance teams assume, sitting in a facility that has to supply and remove a large amount of power. That is the thing being compared against an hourly rate.

Rental benchmarks and the gap between providers

Hourly pricing separates sharply by provider class, and the difference is large enough to change project economics on its own.

• Hyperscalers (AWS, Azure, GCP): roughly $2.50-$4.00 per hour on demand for H100, falling to $1.20-$2.00 reserved.
• Specialist GPU clouds (CoreWeave, Lambda, Together): roughly $1.50-$2.50 on demand, $0.90-$1.50 reserved.
• Spot or preemptible capacity: roughly $0.80-$1.50 per hour, with the interruption risk that implies.

Specialist providers are materially cheaper than hyperscalers for the same silicon, and reserved commitments roughly halve on-demand rates. For workloads that tolerate interruption, and evaluation runs and batch embedding jobs usually do, spot capacity is cheaper again by a wide margin.

The practical implication is that the same workload can differ by a factor of four in cost depending on where and how it is bought. Before modelling a purchase, check whether a committed contract with a specialist provider already lands inside the budget. It frequently does, and it does so without a facility project.

The costs that sit outside the quote

Ownership total cost of ownership is not the server bill, and the gap is where buy-versus-rent analyses go wrong. Power, cooling, facility overhead, support staff, spares, and depreciation all have to be in the model. 2026 writeups that include them find cloud 40-70% cheaper than owning equivalent hardware for many workloads, which is a wide enough margin to survive considerable disagreement about the inputs.

Power and cooling deserve separate attention because they are constraints rather than line items. Dense multi-GPU systems have made procurement a rack-density, electrical-capacity, and liquid-cooling planning problem, not a purchasing problem. A facility that cannot deliver the power per rack or remove the heat cannot host the hardware regardless of budget. Before any purchase discussion advances, get a written answer from facilities on available power per rack and cooling capability for the target configuration. This one question ends a meaningful share of on-premise proposals, and it is far cheaper to ask early.

Staffing is the other underestimated cost. Owned GPU infrastructure needs people who can diagnose driver problems, interconnect failures, and thermal throttling at three in the morning. That capability is either hired, contracted, or absent, and absent shows up as unplanned downtime against the utilisation assumption that justified the purchase.

Lead times and contract risk

High-end GPU supply in 2026 remains constrained, with lead times running from weeks to months depending on configuration and channel. Enterprise deals increasingly take the form of take-or-pay dedicated capacity or reserved private pods precisely because that is what secures delivery.

Those structures move the risk rather than removing it, and four items belong on the legal review list:

• Delivery-date risk. What happens to your programme, and to your payments, if hardware or capacity arrives a quarter late.
• Provider balance-sheet risk. Specialist GPU clouds are cheaper for structural reasons, and some are young companies with heavy capital commitments. Multi-year exclusive dependence deserves the same diligence you would apply to any critical supplier.
• Utilisation minimums. Take-or-pay means you pay for the floor whether or not you use it. Set the floor against a pessimistic forecast, not the business case.
• Early-termination penalties. These decide what a strategy change costs, and they are usually negotiable at signature and never afterwards.

Sovereignty without ownership

For regulated workloads and data residency obligations, the pattern that works is hybrid. Keep the regulated path in a specific facility or sovereign cloud region with contractual guarantees, and run development, evaluation, and non-sensitive workloads on whatever capacity is cheapest.

The market has adapted to this. Private pods, dedicated racks, and GPU-as-a-service from both hyperscalers and specialists now offer isolation and location commitments that shared public cloud could not provide. What matters is contractual specificity: name the facility, name the jurisdiction that governs the operating entity, define who has physical and administrative access, and define what happens to data and to your capacity on termination. A “sovereign” label without those clauses is marketing, and your regulator will read the contract rather than the brochure.

How to approach the decision

Get an honest utilisation forecast before pricing anything. Not the capacity you might need at scale, but GPU-hours per week for the next twelve months based on the workloads that actually exist. Most teams discover the number is far below the level where ownership makes sense, and the procurement question resolves itself.

If the forecast is genuinely high and stable, run the three-year model with your own power price, your facility’s cooling constraints, and a staffing line, then compare against a reserved contract with a specialist provider. Ownership starts to look attractive above roughly 75% sustained utilisation with a three-year horizon and a fleet large enough to justify the operational overhead.

Otherwise, buy a reserved commitment with a specialist provider for the steady baseline, use spot capacity for interruptible work, and revisit in a year with real numbers. The workload profile in year one is a poor predictor of year three, and a contract you can exit is worth more than a marginal saving on a forecast you cannot yet defend.