NVIDIA RTX A4000 — 16 GB professional power in a single slot.

The RTX A4000 (Ampere, $0.17/hr) and T4 (Turing, $0.18/hr) are priced nearly identically. The A4000 delivers ~2x the tensor throughput thanks to Ampere architecture, supports BF16 that the T4 lacks, and has higher bandwidth (448 vs 320 GB/s). The T4 uses less power (70W vs 140W) and has wider cloud availability. For inference performance at the same price, the A4000 is the better card.

In INT4 quantization, yes — 13B INT4 needs ~7 GB, leaving 9 GB for KV cache. In INT8, 13B needs ~13 GB, leaving only 3 GB — workable for low-concurrency use but tight for production. In FP16, 13B needs ~26 GB, which exceeds the 16 GB VRAM. For comfortable 13B serving, consider the 20 GB RTX 4000 Ada or 24 GB A5000.

No. The RTX A4000 is an Ampere-generation professional card (GA104 chip). The 'RTX 4000' branding was later reused for the Ada Lovelace professional lineup (RTX 4000 Ada, AD104 chip). They are different GPUs from different generations. The RTX A4000 is Ampere; the RTX 4000 Ada is Ada Lovelace with higher performance and FP8 support.

With vLLM or TGI serving Mistral 7B in FP16, expect 30-40 tokens/second per request on the A4000. With INT8 quantization, throughput rises to 50-65 tokens/second. These numbers assume single-user latency — batched serving with multiple concurrent users increases total throughput at the cost of per-user latency. For interactive chat, the A4000 handles 5-10 concurrent conversations at acceptable latency.