AIVory Smart Inference  ·  Providers

You have spare capacity. See what it is worth.

AIVory Smart Inference routes every customer request to the cheapest provider that can serve it. Send a test key and we benchmark your endpoint — latency, throughput and error rate, per model, from the regions you serve. You keep those numbers whether or not we ever route to you.

Smart Inference is in early access and the pool is still small. That is the honest argument for joining now rather than later: less competition for the traffic there is, and a longer health record by the time the volume arrives.

Get benchmarked Read the requirements

We reply within two business days.

How volume is allocated

Three steps, then the router decides.

There is no placement to negotiate and no tier to buy. You pass verification, the router scores you on every request, and the traffic follows the score.

You pass verification

We check that your endpoint is OpenAI-compatible, that your model metadata is accurate, and that your prices are readable by machine. We then benchmark latency and throughput from the regions you serve.

The router scores you per request

For each request the router removes every provider that lacks a needed capability, has too small a context window, or is unhealthy. It then picks the cheapest candidate that meets the reliability and speed requirements.

Volume follows the score

Win on price in a region and you get that region's requests. Health probes run continuously, with a five-minute grace period, so a short network blip does not remove you from the pool.

Integration requirements

Six things, before we can benchmark you.

These are not a wish list. Each one is something the router or the billing system reads directly, so a gap here blocks the review.

An OpenAI-compatible endpoint

/v1/chat/completions with streaming. We do not write per-provider adapters, so the request and response shapes have to match.

A precise model list

Exact model IDs, context window and max output per model. Also whether each one supports tool calling, JSON mode and vision, and the quantization you serve it at. The router filters on these fields, so an optimistic answer costs you traffic when requests fail.

Machine-readable prices

Either prices inline in /v1/models, or a price endpoint we can poll. Tell us which, and name the fields. A price list in a PDF cannot be routed on.

Usage on every response

Streams must honour stream_options {"include_usage": true}, and non-streaming responses must always return usage. We bill our customers on those numbers, so we cannot estimate them.

Rate limits and 429 behaviour

Requests per minute, tokens per minute and concurrency, per key. Tell us what a 429 looks like and whether you send Retry-After. The router needs to know when to fail over rather than retry.

A status page and a contact

One URL we can watch, and one human we can reach during an incident. Health probes tell us that you are down; they do not tell us when you will be back.

Set your expectations

We are not a directory.

Here is the deal in full, both halves of it. It is fairer to say this on the page than in the third email.

What you do get

  • Demand that arrives already routed. No sales cycle, no customer support load, no billing relationship to run.
  • A benchmark of your endpoint, run by us, with the numbers sent back to you. Useful even if we never route to you.
  • Regional priority you can earn. If your latency holds up in a region we under-serve, that is a direct argument for us to prioritise routing there.
  • A pricing position that is judged on numbers, not on how long you have been in the market.

What you do not get

  • A listing page, a badge, or a logo on our site.
  • Your name in front of the customer. By default customers see a pool label, not a provider name. That may change later, but plan for it as it is today.
  • A guaranteed share of traffic. Nothing is reserved and nothing is sold.
  • A marketing relationship. We do not run joint campaigns.

What happens next

The review, end to end.

  1. You send the application and a test key

    The key needs a little eval credit on it. Without traffic we cannot measure anything.

  2. We benchmark your endpoint

    Time to first token, throughput and error rate, per model, from the regions you serve. We check that your reported usage matches what we count.

  3. We come back with the numbers

    You get the benchmark results, including where you sit against the pool. This happens whether or not we go ahead.

  4. We agree commercial terms

    See below. This is the step that decides whether routed volume works for both sides.

  5. You go live behind the router

    We start with a small share of traffic and widen it as the health record builds.

The commercial question

Public rate, or a partner rate?

We sell to the customer and we pay you. Our margin sits between your price and theirs, so your price is the input that decides whether the route is ever competitive.

If your published price is your rate

That is a workable starting point. You compete on the same terms as everyone else in the pool, and you win the requests where you are cheapest for the capability the customer asked for.

If you can offer a partner rate

Say so in the application. Routed volume arrives without you paying to acquire it, and a rate that reflects that usually moves you up the score in far more requests than a marketing budget would.

Questions

Before you apply

Do you pay per token or per GPU hour?

Both, depending on what you supply. If you run an OpenAI-compatible inference API, we route token traffic to you and settle on the usage your endpoint reports. If you rent out GPUs, that is the spot marketplace side, and it settles on instance time. Tell us which one you are in the application, or both.

How much volume can we expect?

We cannot promise a number, and we will not pretend otherwise. What we can tell you is the mechanism: the router re-scores every request, so a price change on your side shows up in your traffic within the hour rather than at the next contract review. A provider who becomes cheapest on one popular model can take a large share of it overnight, and lose it the same way.

Will customers see our name?

Not by default. Customers see a pool label rather than a provider name. We may change that later, but plan around how it works today. This is why we say volume follows your numbers, not your logo.

Do you require exclusivity?

No. Sell the same capacity anywhere else you like. We only ask that your published prices stay current, because the router polls them and a stale price means we route on numbers that are wrong.

We are new and small. Is that a problem?

No. The router does not weight providers by brand or by how long they have been trading. It filters on capability and health, then picks on price. A new provider with an honest model list and a working endpoint competes on the same terms as anyone else.

Can we offer interruptible or spot capacity?

Yes. Some requests are served by reclaimable capacity, and the response carries an X-SI-Spot header so the customer knows. Say in the application which of your capacity is interruptible and what your reclaim notice looks like.

How long does the review take?

It depends almost entirely on how complete your first message is. With a working test key and an accurate model list we can usually return benchmark numbers within a week. Without them the review does not start at all.

What happens if we go down?

Health probes take you out of the candidate list, after a five-minute grace period so a short blip does not remove you unnecessarily. In-flight requests fail over to the next candidate, and you come back automatically once the probes recover. That is why we ask for a status page and a human to contact.

Get benchmarked

There are two ways in. Take the short one if you want to move now, or the full one if you want the review to start immediately.

The short version

Your company, your API base URL, the regions you serve, and a test key with a little eval credit on it. That is enough for us to start measuring, and we will ask for the rest when we need it.

Send the short version

The full version

All six at once. Nothing to chase, so the technical review starts the day it arrives.

  1. A test key with some eval credit on it. Without traffic we cannot measure anything.
  2. Your model list. Exact model IDs, context window and max output per model. Also tool calling, JSON mode and vision support, and the quantization you serve each one at.
  3. How we read your prices. Either inline in /v1/models, or a price endpoint we can poll. Name the fields.
  4. Usage on every response. Confirm that streams honour stream_options {"include_usage": true}, and that non-streaming responses always return usage. We bill on those numbers.
  5. Rate limits per key. Requests per minute, tokens per minute and concurrency, plus your 429 behaviour.
  6. A status page and an incident contact we can reach.

One more thing, and it is the one that decides everything: are the prices you publish your public rate, or is there a partner rate for routed volume?

Send the full version

Both buttons open your mail client with the questions ready to fill in. You can also write to [email protected] directly. We reply within two business days.

Send the key.

We will come back with benchmark numbers. That is the fastest way to find out whether this works for both of us.

Apply to the pool