A100 80GB vs RTX PRO 6000 for local AI
Today the A100 80GB is cheaper: $15,299 against $16,499 for the RTX PRO 6000. Per gigabyte of memory the RTX PRO 6000 wins, $171 against $191. For Qwen3.8 27B · Q4_K_M the A100 80GB is faster, about 76 against 71 tokens per second. The RTX PRO 6000 holds more, 96 GB against 80 GB, and fits 18 of our 28 reference models at Q4_K_M where the A100 80GB fits 17.
| NVIDIA A100 80GB PCIe | NVIDIA RTX PRO 6000 Blackwell | |
|---|---|---|
| Lowest price today, delivered | $15,299 on eBay | $16,499 on eBay |
| Price per GB | $191/GB | $171/GB |
| Memory | 80 GB HBM2e | 96 GB GDDR7 ECC |
| Memory bandwidth | 1,935 GB/s | 1,792 GB/s |
| FP16 compute, dense | 312 TFLOPS | 504 TFLOPS |
| Reference models that fit at Q4_K_M | 17 of 28 | 18 of 28 |
| Power | 300 W | 600 W |
| Form factor | Dual slot, passive, needs chassis airflow | Dual slot, full height (Workstation Edition) |
| Interface | PCIe 4 | PCIe 5 |
| Software platform | cuda | cuda |
| Launched | 2021 | 2025 |
| Launch price | n/a | $8,565 |
Qwen3.8 27B · Q4_K_M by quantization
| Quant | A100 80GB | RTX PRO 6000 |
|---|---|---|
| FP16 | ~23 tok/s | ~22 tok/s |
| Q8_0 | ~44 tok/s | ~41 tok/s |
| Q6_K | ~57 tok/s | ~53 tok/s |
| Q5_K_M | ~66 tok/s | ~61 tok/s |
| Q4_K_M | ~76 tok/s | ~71 tok/s |
| Q3_K_M | ~95 tok/s | ~88 tok/s |
| IQ2_M | ~140 tok/s | ~130 tok/s |
Fit means the weights plus an 8K context cache fit in memory; speed is the estimated decode rate. Prices are the cheapest live eBay listing that passed our filters, item plus shipping. How we estimate.