gpt-oss · Aug 2025 · 21B total, 3.6B active
gpt-oss-20b: what runs it and what it costs today
At IQ2_M with 8K of context it needs about 9 GB of memory. The cheapest machine that runs it today is the NVIDIA Tesla P100 16GB at $127.63.
By quantization
Memory needed includes the KV cache for 8K of context and runtime overhead. The cheapest machine is the lowest live price, item plus shipping, that fits with room for that context.
| Quant | Needs | Cheapest machine that fits today | Price |
|---|---|---|---|
| FP16 | 44 GB | Framework Desktop, Ryzen AI Max+ 395, 128 GB | $3,464.65 |
| Q8_0 | 24 GB | AMD Radeon Instinct MI50 32GB | $600 |
| Q6_K | 19 GB | NVIDIA Tesla P40 | $273 |
| Q5_K_M | 17 GB | NVIDIA Tesla P40 | $273 |
| Q4_K_M | 15 GB | NVIDIA Tesla P100 16GB | $127.63 |
| Q3_K_M | 12 GB | NVIDIA Tesla P100 16GB | $127.63 |
| IQ2_M | 9 GB | NVIDIA Tesla P100 16GB | $127.63 |
Every machine at IQ2_M 34 fit
| Machine | Memory | Fit at 8K | Est. speed | Price | Per GB |
|---|---|---|---|---|---|
| NVIDIA Tesla P100 16GB card | 16 GB | fits 128K+ ctx | ~390 tok/s | $127.63 eBay | $7/GB |
| AMD Radeon Instinct MI50 16GB card | 16 GB | fits 128K+ ctx | ~420 tok/s | $228.78 eBay | $14/GB |
| NVIDIA Tesla P40 card | 24 GB | fits 128K+ ctx | ~180 tok/s | $273 eBay | $11/GB |
| NVIDIA GeForce RTX 3060 12GB card | 12 GB | fits 66K ctx | ~190 tok/s | $320 eBay | $26/GB |
| AMD Radeon Instinct MI50 32GB card | 32 GB | fits 128K+ ctx | ~420 tok/s | $600 eBay | $18/GB |
| NVIDIA GeForce RTX 5060 Ti 16GB card | 16 GB | fits 128K+ ctx | ~240 tok/s | $617.99 eBay | $38/GB |
| NVIDIA GeForce RTX 4060 Ti 16GB card | 16 GB | fits 128K+ ctx | ~150 tok/s | $656.27 eBay | $41/GB |
| AMD Radeon RX 7900 XTX card | 24 GB | fits 128K+ ctx | ~390 tok/s | $900 eBay | $37/GB |
| NVIDIA RTX A4000 card | 16 GB | fits 128K+ ctx | ~240 tok/s | $1,079.94 eBay | $67/GB |
| NVIDIA GeForce RTX 3090 card | 24 GB | fits 128K+ ctx | ~500 tok/s | $1,420 eBay | $59/GB |
| NVIDIA RTX 4000 SFF Ada card | 20 GB | fits 128K+ ctx | ~150 tok/s | $1,510.30 eBay | $75/GB |
| AMD Radeon AI PRO R9700 card | 32 GB | fits 128K+ ctx | ~260 tok/s | $1,820 eBay | $56/GB |
| NVIDIA GeForce RTX 4090 card | 24 GB | fits 128K+ ctx | ~540 tok/s | $2,083.48 eBay | $86/GB |
| NVIDIA RTX A5000 card | 24 GB | fits 128K+ ctx | ~410 tok/s | $2,325.65 eBay | $96/GB |
| NVIDIA RTX PRO 4000 Blackwell card | 24 GB | fits 128K+ ctx | ~360 tok/s | $2,827.16 eBay | $117/GB |
| NVIDIA RTX PRO 4000 Blackwell SFF card | 24 GB | fits 128K+ ctx | ~230 tok/s | $2,899 eBay | $120/GB |
| Framework Desktop, Ryzen AI Max+ 395, 128 GB desktop box | 128 GB | fits 128K+ ctx | ~100 tok/s | $3,464.65 eBay | $27/GB |
| GMKtec EVO-X2, Ryzen AI Max+ 395, 128 GB desktop box | 128 GB | fits 128K+ ctx | ~100 tok/s | $3,649.99 eBay | $28/GB |
| NVIDIA RTX PRO 4500 Blackwell card | 32 GB | fits 128K+ ctx | ~480 tok/s | $4,450 eBay | $139/GB |
| NVIDIA RTX A6000 card | 48 GB | fits 128K+ ctx | ~410 tok/s | $4,500 eBay | $93/GB |
| Apple Mac Studio M4 Max, 128 GB desktop box | 128 GB | fits 128K+ ctx | ~270 tok/s | $4,899 eBay | $38/GB |
| NVIDIA GeForce RTX 5090 card | 32 GB | fits 128K+ ctx | ~950 tok/s | $4,920.99 eBay | $153/GB |
| ASUS Ascent GX10 desktop box | 128 GB | fits 128K+ ctx | ~140 tok/s | $6,099.99 eBay | $47/GB |
| Acer Veriton GN100 desktop box | 128 GB | fits 128K+ ctx | ~140 tok/s | $6,367.19 eBay | $49/GB |
| HP ZGX Nano AI Station desktop box | 128 GB | fits 128K+ ctx | ~140 tok/s | $6,440.40 eBay | $50/GB |
| Gigabyte AI TOP ATOM desktop box | 128 GB | fits 128K+ ctx | ~140 tok/s | $6,999 eBay | $54/GB |
| NVIDIA RTX PRO 5000 Blackwell card | 48 GB | fits 128K+ ctx | ~710 tok/s | $7,099.99 eBay | $147/GB |
| NVIDIA RTX 6000 Ada card | 48 GB | fits 128K+ ctx | ~510 tok/s | $7,698.43 eBay | $160/GB |
| NVIDIA DGX Spark desktop box | 128 GB | fits 128K+ ctx | ~140 tok/s | $7,799.98 eBay | $60/GB |
| Apple Mac Studio M3 Ultra, 256 GB desktop box | 256 GB | fits 128K+ ctx | ~400 tok/s | $8,499 eBay | $33/GB |
| NVIDIA A100 80GB PCIe card | 80 GB | fits 128K+ ctx | ~1,000 tok/s | $15,299 eBay | $191/GB |
| NVIDIA RTX PRO 6000 Blackwell card | 96 GB | fits 128K+ ctx | ~950 tok/s | $16,499 eBay | $171/GB |
| Apple Mac Studio M3 Ultra, 512 GB desktop box | 512 GB | fits 128K+ ctx | ~400 tok/s | $18,566.48 eBay | $36/GB |
| NVIDIA H100 80GB PCIe card | 80 GB | fits 128K+ ctx | ~1,100 tok/s | $26,995.21 eBay | $337/GB |
Estimates assume the whole model in memory with llama.cpp-style inference; see how we estimate. Prices are the cheapest live eBay listing that passed our filters, item plus shipping.