Nemotron · Aug 2026 · 31.6B total, 3.5B active
Nemotron 3.5 Lightning 30B-A3B: what runs it and what it costs today
At FP16 with 8K of context it needs about 65 GB of memory. The cheapest machine that runs it today is the Framework Desktop, Ryzen AI Max+ 395, 128 GB at $3,618.73. With 49 GB of the weights in system RAM, the NVIDIA Tesla P100 16GB at $127.63 also runs it, at about 7.1 tokens per second.
By quantization
Memory needed includes the KV cache for 8K of context and runtime overhead. The cheapest machine is the lowest live price, item plus shipping, that fits with room for that context.
| Quant | Needs | Cheapest machine that fits today | Price |
|---|---|---|---|
| FP16 | 65 GB | Framework Desktop, Ryzen AI Max+ 395, 128 GB | $3,618.73 |
| Q8_0 | 35 GB | Framework Desktop, Ryzen AI Max+ 395, 128 GB | $3,618.73 |
| Q6_K | 27 GB | AMD Radeon Instinct MI50 32GB | $600 |
| Q5_K_M | 24 GB | NVIDIA Tesla P40 | $269.99 |
| Q4_K_M | 21 GB | NVIDIA Tesla P40 | $269.99 |
| Q3_K_M | 17 GB | NVIDIA Tesla P40 | $269.99 |
| IQ2_M | 12 GB | NVIDIA Tesla P100 16GB | $127.63 |
Every machine at FP16 18 fit
| Machine | Memory | Fit at 8K | Est. speed | Price | Per GB |
|---|---|---|---|---|---|
| Framework Desktop, Ryzen AI Max+ 395, 128 GB desktop box | 128 GB | fits 128K+ ctx | ~17 tok/s ~17 at 32K | $3,618.73 eBay | $28/GB |
| GMKtec EVO-X2, Ryzen AI Max+ 395, 128 GB desktop box | 128 GB | fits 128K+ ctx | ~17 tok/s ~17 at 32K | $3,649.99 eBay | $28/GB |
| Apple Mac Studio M4 Max, 128 GB desktop box | 128 GB | fits 128K+ ctx | ~40 tok/s ~39 at 32K | $4,899 eBay | $38/GB |
| ASUS Ascent GX10 desktop box | 128 GB | fits 128K+ ctx | ~23 tok/s ~23 at 32K | $5,899.99 eBay | $46/GB |
| Apple MacBook Pro M5 Max, 128 GB laptop | 128 GB | fits 128K+ ctx | ~44 tok/s ~43 at 32K | $5,952.43 eBay | $46/GB |
| Apple Mac Studio M5 Max, 128 GB desktop box | 128 GB | fits 128K+ ctx | ~44 tok/s ~43 at 32K | $5,999 eBay | $46/GB |
| Acer Veriton GN100 desktop box | 128 GB | fits 128K+ ctx | ~23 tok/s ~23 at 32K | $6,367.19 eBay | $49/GB |
| HP ZGX Nano AI Station desktop box | 128 GB | fits 128K+ ctx | ~23 tok/s ~23 at 32K | $7,028.73 eBay | $54/GB |
| NVIDIA DGX Spark desktop box | 128 GB | fits 128K+ ctx | ~23 tok/s ~23 at 32K | $7,300 eBay | $57/GB |
| Gigabyte AI TOP ATOM desktop box | 128 GB | fits 128K+ ctx | ~23 tok/s ~23 at 32K | $7,416.62 eBay | $57/GB |
| AMD Ryzen AI Max+ PRO 495 desktop, 192 GB desktop box | 192 GB | fits 128K+ ctx | ~18 tok/s ~18 at 32K | $8,099.99 eBay | $42/GB |
| Apple Mac Studio M3 Ultra, 256 GB desktop box | 256 GB | fits 128K+ ctx | ~56 tok/s ~55 at 32K | $8,549.99 eBay | $33/GB |
| Apple Mac Studio M5 Ultra, 256 GB desktop box | 256 GB | fits 128K+ ctx | ~75 tok/s ~74 at 32K | $13,499 eBay | $52/GB |
| NVIDIA A100 80GB PCIe card | 80 GB | fits 128K+ ctx | ~110 tok/s ~110 at 32K | $14,018.71 eBay | $175/GB |
| NVIDIA RTX PRO 6000 Blackwell card | 96 GB | fits 128K+ ctx | ~100 tok/s ~100 at 32K | $16,499 eBay | $171/GB |
| Apple Mac Studio M3 Ultra, 512 GB desktop box | 512 GB | fits 128K+ ctx | ~56 tok/s ~55 at 32K | $16,999 eBay | $33/GB |
| NVIDIA H100 80GB PCIe card | 80 GB | fits 128K+ ctx | ~110 tok/s ~110 at 32K | $27,180.38 eBay | $339/GB |
| NVIDIA RTX Spark N1X laptop, 128 GB laptop | 128 GB | fits 128K+ ctx | ~25 tok/s ~25 at 32K | no live price | |
| NVIDIA Tesla P100 16GB card | 16 GB | with 49 GB in system RAM | ~7.1 tok/s ~7.1 at 32K | $127.63 eBay | $7/GB |
| AMD Radeon Instinct MI50 16GB card | 16 GB | with 49 GB in system RAM | ~7.2 tok/s ~7.1 at 32K | $246 eBay | $15/GB |
| NVIDIA Tesla P40 card | 24 GB | with 41 GB in system RAM | ~7.9 tok/s ~7.9 at 32K | $269.99 eBay | $11/GB |
| NVIDIA GeForce RTX 3060 12GB card | 12 GB | with 53 GB in system RAM | ~6.6 tok/s ~6.5 at 32K | $289.94 eBay | $24/GB |
| AMD Radeon Instinct MI50 32GB card | 32 GB | with 33 GB in system RAM | ~10 tok/s ~10 at 32K | $600 eBay | $18/GB |
| NVIDIA GeForce RTX 5060 Ti 16GB card | 16 GB | with 49 GB in system RAM | ~7.0 tok/s ~7.0 at 32K | $617.99 eBay | $38/GB |
| NVIDIA GeForce RTX 4060 Ti 16GB card | 16 GB | with 49 GB in system RAM | ~6.9 tok/s ~6.8 at 32K | $648.41 eBay | $40/GB |
| AMD Radeon RX 7900 XT 20GB card | 20 GB | with 45 GB in system RAM | ~7.6 tok/s ~7.6 at 32K | $699.99 eBay | $34/GB |
| NVIDIA Tesla V100 32GB PCIe card | 32 GB | with 33 GB in system RAM | ~10 tok/s ~10 at 32K | $729 eBay | $22/GB |
| AMD Radeon RX 9070 XT 16GB card | 16 GB | with 49 GB in system RAM | ~7.1 tok/s ~7.0 at 32K | $799.97 eBay | $49/GB |
| AMD Radeon RX 7900 XTX card | 24 GB | with 41 GB in system RAM | ~8.3 tok/s ~8.3 at 32K | $900 eBay | $37/GB |
| NVIDIA RTX A4000 card | 16 GB | with 49 GB in system RAM | ~7.0 tok/s ~7.0 at 32K | $950 eBay | $59/GB |
| NVIDIA GeForce RTX 3090 card | 24 GB | with 41 GB in system RAM | ~8.4 tok/s ~8.4 at 32K | $990 eBay | $41/GB |
| NVIDIA GeForce RTX 5070 Ti 16GB card | 16 GB | with 49 GB in system RAM | ~7.2 tok/s ~7.1 at 32K | $1,007.98 eBay | $62/GB |
| Intel Arc Pro B70 32GB card | 32 GB | with 33 GB in system RAM | ~9.0 tok/s ~8.9 at 32K | $1,398 eBay | $43/GB |
| NVIDIA GeForce RTX 5080 16GB card | 16 GB | with 49 GB in system RAM | ~7.2 tok/s ~7.2 at 32K | $1,500 eBay | $93/GB |
| NVIDIA RTX 4000 SFF Ada card | 20 GB | with 45 GB in system RAM | ~7.3 tok/s ~7.2 at 32K | $1,511.06 eBay | $75/GB |
| AMD Radeon AI PRO R9700 card | 32 GB | with 33 GB in system RAM | ~9.7 tok/s ~9.6 at 32K | $1,950 eBay | $60/GB |
| NVIDIA GeForce RTX 4090 card | 24 GB | with 41 GB in system RAM | ~8.4 tok/s ~8.4 at 32K | $2,083.48 eBay | $86/GB |
| NVIDIA RTX A5000 card | 24 GB | with 41 GB in system RAM | ~8.4 tok/s ~8.3 at 32K | $2,325.65 eBay | $96/GB |
| NVIDIA RTX PRO 4000 Blackwell card | 24 GB | with 41 GB in system RAM | ~8.3 tok/s ~8.3 at 32K | $2,659.99 eBay | $110/GB |
| NVIDIA RTX PRO 4000 Blackwell SFF card | 24 GB | with 41 GB in system RAM | ~8.1 tok/s ~8.0 at 32K | $2,899 eBay | $120/GB |
| NVIDIA RTX PRO 4500 Blackwell card | 32 GB | with 33 GB in system RAM | ~10 tok/s ~10 at 32K | $4,450 eBay | $139/GB |
| NVIDIA RTX A6000 card | 48 GB | with 17 GB in system RAM | ~17 tok/s ~17 at 32K | $4,500 eBay | $93/GB |
| NVIDIA GeForce RTX 5090 card | 32 GB | with 33 GB in system RAM | ~10 tok/s ~10 at 32K | $4,920.99 eBay | $153/GB |
| NVIDIA RTX PRO 5000 Blackwell card | 48 GB | with 17 GB in system RAM | ~18 tok/s ~18 at 32K | $7,099.99 eBay | $147/GB |
| NVIDIA RTX 6000 Ada card | 48 GB | with 17 GB in system RAM | ~17 tok/s ~17 at 32K | $7,684.05 eBay | $160/GB |
| Apple Mac mini M5 Pro, 64 GB desktop box | 64 GB | does not fit | $3,757.78 eBay | $58/GB |
Estimates assume llama.cpp-style inference for one user. Cards that cannot hold the weights run with the overflow in system RAM, assuming 64 GB, which is slower; see how we estimate. Prices are the cheapest live eBay listing that passed our filters, item plus shipping.