24GB
VRAM
GDDR6X memory
AIVory · GPU Marketplace
Live spot pricingNVIDIA RTX 4090 — consumer king, inference powerhouse.
The widest spot availability and lowest entry price for serious AI inference. Ada Lovelace architecture, 24 GB GDDR6X, and from $0.29/hr on the spot market. The pragmatic choice for startups and indie developers who need GPU compute without enterprise budgets.
At a glance
RTX 4090 specifications.
Key hardware specs that determine what workloads this GPU handles.
1.01 TB/s
Memory Bandwidth
peak throughput
450W
TDP
thermal design power
Ada Lovelace
Architecture
NVIDIA GPU architecture
Spot pricing
RTX 4090: live hourly rates.
Every provider offering this GPU on the spot market, sorted cheapest first.
Prices in USD per GPU-hour · spot instances · sorted cheapest first
Recommended models
AI models that run well on RTX 4090.
Tested model-GPU pairings with notes on why each is a good fit.
Mistral Small 24B
Fits perfectly in 24 GB in INT8. One of the strongest models at this parameter count, running at full speed on consumer hardware.
View model pricing →
Mistral Nemo 12B
Runs in FP16 with 12 GB of headroom for KV cache. Ideal for real-time chat inference with long conversation histories.
View model pricing →
Qwen 3.5 27B (4-bit)
4-bit quantization brings this 27B model to ~16 GB, fitting on the 4090 with room for concurrent request batching.
View model pricing →
Gemma 2 9B
Runs in full FP16 with plenty of headroom. High throughput for lightweight chat, classification, and summarization tasks.
View model pricing →
Use cases
What the RTX 4090 is built for.
-
Development and prototyping before scaling up
The RTX 4090 is the fastest way to validate that your model, prompt templates, and serving infrastructure work correctly before committing to expensive data center GPUs. At $0.29-0.35/hr, you can iterate on inference configurations, benchmark latency, and stress-test your pipeline for a few dollars a day. When you are ready to scale, move to A100 or H100 with confidence.
-
Running sub-24B models in production at minimal cost
Models under 24B parameters — Mistral Small 24B, Gemma 2 9B, Llama 3.2 3B, Phi-3 14B — run natively on the RTX 4090 in FP16 or INT8. For startups and small teams serving hundreds to low thousands of requests per minute, a single 4090 at $0.30/hr often beats API pricing from OpenAI or Anthropic within the first week of operation.
-
Horizontal scaling with many cheap GPUs
For embarrassingly parallel workloads — batch embedding generation, document classification, dataset labeling — ten RTX 4090s at $3/hr total can outperform a single A100 at $1.50/hr while providing redundancy. If one instance gets reclaimed, nine continue. The spot market's deep 4090 supply makes large horizontal deployments practical.
FAQ
Common questions.
RTX 4090 vs A100 for inference — which is better?
The 4090 wins on cost for any model that fits in 24 GB. On a per-dollar basis, the 4090 delivers roughly 2x the inference throughput of an A100 for models under 24B parameters. The A100 wins when you need more VRAM (for 70B models), when you need NVLink for multi-GPU setups, or when spot availability in your required region favors data center GPUs.
Why is the RTX 4090 so cheap on spot markets?
Three factors: consumer supply volume (NVIDIA sold millions of 4090s to gamers and creators), the post-crypto mining GPU glut (repurposed mining rigs entering the rental market), and the lack of enterprise SLAs (providers price consumer GPUs lower because they carry less support overhead). The hardware itself is excellent for inference — only the market dynamics make it cheap.
What models don't fit on a 24 GB RTX 4090?
Anything above ~20B parameters in FP16 will not fit. Llama 3.3 70B requires aggressive 4-bit quantization (GPTQ/AWQ) to squeeze into 24 GB, and even then KV cache for long contexts can push it over. Mixtral 8x7B (46.7B) does not fit. For models in the 24-70B range, you need a 48 GB card (L40S, A6000) or an 80 GB card (A100, H100).
How reliable are RTX 4090 spot instances compared to reserved?
Interruption rates on 4090 spot instances are typically 2-10%, depending on provider and time of day. Vast.ai reports around 2% interruption probability for well-priced 4090 offers. RunPod's community cloud runs closer to 10%. For fault-tolerant workloads (batch inference, async processing), this is perfectly acceptable. For latency-sensitive serving, pair spot 4090s with auto-restart and a load balancer.
Rent a RTX 4090. Right now.
Spot pricing, per-second billing, no commitment.
Browse the live marketplace, pick your GPU, deploy in one click. Credits from $10.