Best GPUs for local AI in 2026, by budget
Two numbers decide almost everything about a GPU for local models: how much memory it has, and how fast that memory is. Memory decides which models fit at all; bandwidth decides how many tokens per second you get once they do. Compute matters for prompt processing and for image models, but for chatting with a language model it is a distant third. That ordering is why this page ranks by memory inside each budget, breaks ties on bandwidth, and only then looks at the price.
The prices are the cheapest live eBay listing that passes our filters, item plus shipping, refreshed every 15 minutes. The speed figure is our estimate for Qwen3.8 27B at Q4_K_M with 8K of context; change the model on any product page.
Prices are the cheapest live eBay listing that passes our filters, item plus shipping, refreshed every 15 minutes. Speed is our estimate for Qwen3.8 27B · Q4_K_M with 8K of context.
How to choose
Start from the model you actually want to run and read its memory need on its model page. Add a quarter for context and overhead, then buy the cheapest card that clears that number. If two cards clear it, take the one with more bandwidth. CUDA still has the smoothest software; AMD cards work well in llama.cpp through Vulkan or ROCm but expect to read a few more forum threads; Apple is covered on its own page.
Used data-center cards are the cheapest memory you can buy and the most work to live with: they are passive, so you need a fan shroud, and the oldest ones are losing software support. The used server GPU guide covers the fine print.
Under $300
This is used-card territory. The point is memory per dollar, and nothing new comes close.
NVIDIA Tesla P40
$273$11/GB
24 GB · 346 GB/s · ~14 tok/s
The cheapest 24 GB with CUDA. No tensor cores and slow FP16, so prefill crawls and newer software is dropping support, but a 14B model at Q6 runs, and it costs less than a game console.
AMD Radeon Instinct MI50 16GB
$244.50$15/GB
16 GB · 1,024 GB/s · does not fit
The eBay bargain: 16 GB of HBM2 at a terabyte per second for less than any 16 GB GeForce. Passive, 300 W and retired from official ROCm support, so expect some driver archaeology, but a 14B model at Q6 runs quickly.
NVIDIA Tesla P100 16GB
$127.63$7/GB
16 GB · 732 GB/s · does not fit
HBM2 bandwidth from 2016 for pocket change. Faster per token than a P40 and it does real FP16, but only 16 GB and no tensor cores, and newer software is dropping Pascal.
Under $500
The first band where a current card with a warranty is an option.
NVIDIA GeForce RTX 3060 12GB
$320$26/GB
12 GB · 360 GB/s · does not fit
The cheapest way into CUDA with 12 GB: an 8B model at Q8 or a 14B at Q4. Slow by today's standards, but it works in everything and costs less than a game and a half.
Under $1,000
Here 24 GB comes into reach, and 24 GB is the line that separates 14B models from 30B models at a sensible quant.
AMD Radeon Instinct MI50 32GB
$600$18/GB
32 GB · 1,024 GB/s · ~31 tok/s
32 GB of HBM2 at a terabyte per second. The 32 GB version has become scarce and sells for several times the 16 GB card; passive, 300 W and retired from official ROCm support, so expect some driver archaeology, but a 32B model at Q4 runs.
AMD Radeon RX 7900 XTX
$900$37/GB
24 GB · 960 GB/s · ~29 tok/s
24 GB at 960 GB/s, the same memory and nearly the bandwidth of an RTX 4090, for less. Runs well in llama.cpp on ROCm or Vulkan; CUDA-only tools are the catch.
NVIDIA GeForce RTX 5060 Ti 16GB
$649.99$40/GB
16 GB · 448 GB/s · does not fit
GDDR7 gives this 16 GB card half again the bandwidth of a 4060 Ti, plus FP4 for the newest quant formats. The sensible new budget card for 14B models.
Under $2,000
The choice here is memory against bandwidth and software: AMD sells 32 GB in this band, NVIDIA sells 24 GB that reads faster and runs everything.
AMD Radeon AI PRO R9700
$1,800$56/GB
32 GB · 640 GB/s · ~19 tok/s
32 GB for the price of a 24 GB GeForce, on ROCm. Bandwidth is a third of a 5090's, so expect a third of the speed; the trade is memory per dollar.
NVIDIA GeForce RTX 3090
$1,489$62/GB
24 GB · 936 GB/s · ~37 tok/s
The budget 24 GB card. Nearly the bandwidth of a 4090 for a fraction of the price, and two of them bridge with NVLink for a 70B model at Q4.
NVIDIA RTX 4000 SFF Ada
$1,450.30$72/GB
20 GB · 280 GB/s · ~11 tok/s
20 GB of ECC memory at 70 W in a half-height card. Slow by 2025 standards, but it runs a 14B model at Q6 in a machine with no spare power connector, which is why it still holds its price.
Under $5,000
32 to 48 GB, which opens 70B models at Q4 on a single card.
NVIDIA RTX A6000
$4,500$93/GB
48 GB · 768 GB/s · ~30 tok/s
48 GB for the least money per gigabyte among NVIDIA cards that still get driver updates. Slower than the Ada and Blackwell cards, but a 70B model at Q4 fits, and two of them bridge with NVLink.
NVIDIA RTX PRO 4500 Blackwell
$4,450$139/GB
32 GB · 896 GB/s · ~35 tok/s
32 GB of GDDR7 at 200 W: the same memory as a GeForce RTX 5090 in a quieter, narrower, ECC-backed card.
Money is no object
The 96 GB cards, which run anything short of the very largest open models on one board, at a price that buys a small car.
NVIDIA RTX PRO 6000 Blackwell
$15,900$165/GB
96 GB · 1,792 GB/s · ~71 tok/s
The most memory you can put on one PCIe card: 96 GB of GDDR7 with ECC and 1.8 TB/s of bandwidth. Enough for a 70B model at Q8, or a 120B mixture-of-experts model at Q4 with context to spare. Sold as Workstation Edition (600 W), Max-Q (300 W) and Server Edition.
NVIDIA H100 80GB PCIe
$27,000$337/GB
80 GB · 2,000 GB/s · ~79 tok/s
The PCIe version of the card the clouds run on. 80 GB at 2 TB/s with FP8 tensor cores; prefill speed in a different league from any workstation card.
More guides
- Best Mac for local LLMs in 2026 Mac Studio configurations compared for running models, with live prices.
- DGX Spark alternatives in 2026: the same chip, half the price, or more bandwidth GB10 twins, Strix Halo boxes, Apple and used cards, lined up.
- Used server GPUs for local LLMs in 2026: P40, P100, MI50, A-series and A100 The cheap-memory cards and their fine print: cooling, power, drivers.