NVIDIA A40 — 48 GB with ECC for the price of a coffee.

The A40 ($0.35/hr) and A6000 ($0.33/hr) are priced similarly with the same 48 GB VRAM and Ampere architecture. Key differences: the A40 has ECC memory (error correction) and is passively cooled for data centers; the A6000 has higher bandwidth (768 vs 696 GB/s) and display outputs for workstation use. For inference reliability, the A40's ECC is the tiebreaker. For raw throughput, the A6000's extra bandwidth gives it a slight edge.

The L40/L40S use Ada Lovelace architecture (one generation newer) with FP8 support and higher tensor throughput. They cost more ($0.51-$0.57/hr) but deliver roughly 40% better inference performance per GPU-hour. If your workload is latency-sensitive, the L40S wins on speed. If it's throughput-at-minimum-cost, the A40 wins on price.

Yes. A40 servers support up to 4 or 8 GPUs connected via PCIe 4.0. NVLink is not available on the A40 (unlike the A100), so inter-GPU bandwidth is limited to PCIe speeds (~32 GB/s per direction). This is fine for independent model instances but too slow for tensor-parallel inference on models sharded across cards. Use multi-A40 for multi-model hosting, not multi-GPU sharding.

For fine-tuning models up to ~15B parameters in BF16, the A40 is usable. LoRA and QLoRA fine-tuning of 30B-70B models works within the 48 GB envelope. Full pre-training at scale should use A100 or H100 for the HBM bandwidth advantage (2.0+ TB/s vs 696 GB/s). The A40 is a training-capable inference card, not a training-first card.