AIVory Smart Inference Documentation
Smart Inference is an OpenAI-compatible HTTP gateway that routes every request to the cheapest working provider at that moment. Change your base_url, keep the same SDK, keep the same model names, pay less.
- OpenAI-compatible - keep your existing SDK, prompts, and streaming handlers
- Cheapest-first routing - scored per request, with automatic fallback if the chosen provider fails
- Pay as you go - top up from $10, default $100 monthly cap (adjustable)
- Per-request receipts - cost, chosen model, routing score, and latency on every response
Note
Start with the quickstart - sign up, top up, and swap your base_url in under five minutes. Quickstart
Documentation sections
Quickstart
Your first request in curl, Python, or TypeScript
API reference
Endpoints, request and response schemas, streaming format
SDK compatibility
Which OpenAI chat parameters we accept and which we ignore
How routing works
Scoring, health filtering, failover, and what we never compromise on
Response headers
Read the per-request receipt we attach to every response
GPU Spot Marketplace
Rent spot GPUs, browse offers, and deploy models on bare metal
Billing and credits
Pay-as-you-go credits, caps, auto-recharge, and refunds
Errors
HTTP status codes, error shapes, and what each one means
Model catalogue
Canonical model IDs, tiers, and how to list them live
The short version
from openai import OpenAI
client = OpenAI(
base_url="https://smart.aivory.net/v1",
api_key="sk-aivory-...",
)
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": "Ship it."}],
)
That’s the whole migration. Everything else on this site — routing, failover, credits — lives behind that one URL.