Used server GPUs for local LLMs in 2026: P40, P100, MI50, A-series and A100
The cheapest memory you can buy for local models is a card that a data center retired. A Tesla P40 gives 24 GB for the price of a budget consumer card; an MI50 gives a terabyte per second of HBM2 for even less. The catch is that none of them were built for a desktop: they are passive, they want a fan shroud and a strong power supply, and the oldest are losing software support. Here is what each family actually gets you today.
Prices are the cheapest live eBay listing that passes our filters, item plus shipping, refreshed every 15 minutes. Speed is our estimate for Qwen3.8 27B · Q4_K_M with 8K of context.
Pascal: cheap memory, fading support
No tensor cores, so prompt processing crawls and the newest quant formats are slow or missing, and llama.cpp and PyTorch are both winding down Pascal support. For chat with a 14B to 30B model at a sane speed they still work, and nothing else is this cheap per GB.
NVIDIA Tesla P40
$273$11/GB
24 GB · 346 GB/s · ~14 tok/s
The cheapest 24 GB with CUDA. No tensor cores and slow FP16, so prefill crawls and newer software is dropping support, but a 14B model at Q6 runs, and it costs less than a game console.
NVIDIA Tesla P100 16GB
$127.63$7/GB
16 GB · 732 GB/s · does not fit
HBM2 bandwidth from 2016 for pocket change. Faster per token than a P40 and it does real FP16, but only 16 GB and no tensor cores, and newer software is dropping Pascal.
AMD Instinct MI50
HBM2 at 1 TB/s for the price of lunch, in two memory sizes. Officially dropped from ROCm, but llama.cpp runs it through Vulkan or older ROCm builds and the community keeps it alive. Budget an afternoon for drivers.
AMD Radeon Instinct MI50 16GB
$244.50$15/GB
16 GB · 1,024 GB/s · does not fit
The eBay bargain: 16 GB of HBM2 at a terabyte per second for less than any 16 GB GeForce. Passive, 300 W and retired from official ROCm support, so expect some driver archaeology, but a 14B model at Q6 runs quickly.
AMD Radeon Instinct MI50 32GB
$600$18/GB
32 GB · 1,024 GB/s · ~31 tok/s
32 GB of HBM2 at a terabyte per second. The 32 GB version has become scarce and sells for several times the 16 GB card; passive, 300 W and retired from official ROCm support, so expect some driver archaeology, but a 32B model at Q4 runs.
Ampere workstation cards
Blower coolers, ECC memory, modest power draw and full current software support. They cost more per GB than the server pulls above and far less trouble; the A4000 in particular fits four to a case.
NVIDIA RTX A4000
$979.99$61/GB
16 GB · 448 GB/s · does not fit
A single-slot 16 GB card that draws 140 W: four of them fit where two consumer cards would. Used workstation pulls are plentiful and cheap.
NVIDIA RTX A5000
$2,325.65$96/GB
24 GB · 768 GB/s · ~30 tok/s
A 3090's memory in a 230 W blower card that stacks in a case. Less bandwidth, but ECC, NVLink and workstation drivers.
NVIDIA RTX A6000
$4,500$93/GB
48 GB · 768 GB/s · ~30 tok/s
48 GB for the least money per gigabyte among NVIDIA cards that still get driver updates. Slower than the Ada and Blackwell cards, but a 70B model at Q4 fits, and two of them bridge with NVLink.
Data-center Ampere and Hopper
80 GB on one board with HBM bandwidth. Passive, 300 W or more, and priced like a used car, but nothing else runs a 70B model at FP8 on a single card.
NVIDIA A100 80GB PCIe
$14,018.71$175/GB
80 GB · 1,935 GB/s · ~76 tok/s
80 GB of HBM2e at 1.9 TB/s on a passively cooled data-center card. A 70B model at Q8 fits. It needs a chassis that pushes air through it, or a bolt-on fan.
NVIDIA H100 80GB PCIe
$27,000$337/GB
80 GB · 2,000 GB/s · ~79 tok/s
The PCIe version of the card the clouds run on. 80 GB at 2 TB/s with FP8 tensor cores; prefill speed in a different league from any workstation card.
Before you buy
Cooling: passive cards need forced air; a 3D-printed shroud and a 40 mm fan is the usual answer, and it is loud. Power: most need an EPS or PCIe 8-pin adapter and a supply with headroom. Drivers: check that the card's architecture is still supported by the software you plan to use, this month. Sellers: our filters require a track record on the expensive cards, but read the listing; "for parts" and "no video output" are not bargains.
More guides
- Best GPUs for local AI in 2026, by budget Every budget tier from $300 up, picked by memory first, priced live.
- Best Mac for local LLMs in 2026 Mac Studio configurations compared for running models, with live prices.
- DGX Spark alternatives in 2026: the same chip, half the price, or more bandwidth GB10 twins, Strix Halo boxes, Apple and used cards, lined up.