NVIDIA RTX 4080 — Ada Lovelace at two-thirds the 4090 price.

The RTX 4090 costs $0.29/hr with 24 GB and 1.01 TB/s bandwidth; the RTX 4080 costs $0.27/hr with 16 GB and 717 GB/s. For models under 16 GB, the 4080 delivers 80-90% of the 4090's throughput at 93% of the price — the 4090's edge is marginal. For models between 16-24 GB (14B FP16, 27B INT4), only the 4090 works. If your model fits in 16 GB, the 4080 is the smarter buy.

In INT8, a 13B model needs ~13 GB, leaving 3 GB for KV cache. This works for low-concurrency use but is tight for production with multiple simultaneous users. In INT4, 13B needs ~7 GB, which gives 9 GB of comfortable headroom. FP16 at ~26 GB exceeds the 16 GB VRAM. For 13B production serving, INT4 quantization on the RTX 4080 is practical; INT8 is borderline.

The T4 costs $0.18/hr vs $0.27/hr for the RTX 4080. The RTX 4080 delivers 3-4x the inference throughput thanks to Ada Lovelace tensor cores and 2.2x higher bandwidth (717 vs 320 GB/s). If you're serving enough traffic to saturate a T4, upgrading to an RTX 4080 gives you the headroom at a 50% price premium for 300% more throughput. For light traffic, the T4 is cheaper.

LoRA fine-tuning of 7B models fits in 16 GB with gradient checkpointing. Full fine-tuning is limited to 3B-5B models due to the VRAM constraint from optimizer states. For any serious fine-tuning of 13B+ models, the RTX 4090 (24 GB) or A100 (80 GB) is necessary. The RTX 4080 is best classified as an inference card that can handle light fine-tuning.