This Tiny LLM Does Nothing but Judge

For years I have wanted a model whose only job is to decide, and TypeSafe AI just released one. Jev does not chat and it does not write. You hand it text and a typed question, and it hands back a choice from a list, a score on a scale, or a yes/no, each carrying a number that says how much to trust it.

I tested it in the private beta. It is the first model in a long while where my reaction was not “that is a little better than the last one” but “this is a different tool”.

What is Jev, and how is it different from a normal LLM?

Jev is tiny and specialized, and that is the whole design rather than a compromise. Every other model I run is general. It can write an email, refactor a function, and then argue with itself for a minute about a two-way decision. Jev cannot do any of that. You give it a state, which is text or an object or an array, plus a set of typed questions, and it answers all of them in one call.

There are three question types and nothing else. A choice picks one option from a set you describe. A score puts the input on a scale you define. A yes/no question comes back as a probability. Nothing is parsed out of prose, because there is no prose to parse.

If you have ever typed “respond with only the word YES and nothing else” into a system prompt, you already know why that matters. That line is a chat model being talked into behaving like a function. Jev is the function.

Why does the confidence number matter so much?

Because it tells you when not to trust the answer, and that is what makes a model safe to put in front of something. Every answer arrives with a confidence from 0 to 1, which is how peaked the distribution is. Below 0.5 means it is genuinely unsure.

That one number is what lets code decide without a human in the loop. You set a floor, and anything under it falls through to whatever you were doing before. The model does not have to be right every time. It has to be honest about when it is unsure.

Mostly it is.

What does Jev refuse to do?

It does not chat, it does not write, and it does not reason at you for two minutes. It is not your Fable, Mistral or Astra, and it is not trying to be. No sentence at the end, no summary, no edit, no explanation of its own answer.

That reads like a restriction until you go through your own list of jobs and count how many genuinely needed prose. Most of mine did not. They needed a decision, and the prose was packaging I had to unwrap before I could use it.

What can you actually use it for?

Micro-decisions, and there are far more of them in your code than you think. Working out what the user wants so you can route them to the right place. Classifying, as in this is that. Judging whether a direction is good before you spend real money going down it. Working out where to click on a screen.

None of those are glamorous and all of them are everywhere. They are the small decisions software makes on the way to an answer, and almost all of them are currently handled by either a pile of regular expressions or a general model doing a job several sizes below its pay grade.

How fast is it on a real job?

Fast enough that the tail stops being a problem, which matters more than the average. The case I benchmarked hardest is an intent gate: every turn a user types gets sorted into a model request, off topic, or fine, and that gate sits in front of the answer, so it has a hard time budget. 69 cases, three repetitions, 207 runs.

Jev got every single one right in all three rounds. The median was 250 ms and the 95th percentile landed between 453 and 688 ms. Out of everything in that bench it was the only one that never once blew the 1.5 second budget.

Let me be honest about what that does and does not prove. Once I repaired the incumbent I was comparing against, it reached 0.986 to 1.000 on the same set, so the accuracy gap is small and I would not sell you on it. The gap that actually mattered was the tail. A gate that is right almost every time but occasionally takes two seconds is a gate you end up deleting.

The second job had no incumbent at all: one yes/no question per slide, checked against the source document the deck was built from, 40 slides in a single call. It came back in 1.41 seconds and cost about a tenth of a cent. It caught claims I had read straight past myself, which is the real reason I trust it here, rather than any number in my bench.

Where does it confidently get things wrong?

It judges meaning well and characters badly, and the line between those two is sharper than I expected. Ask it whether a claim is supported by a source document and it is excellent. Ask it what is literally on the page and it guesses.

I found this the dumb way. I asked it whether each of forty short texts I had written contained an em dash. Not one of them did. It said yes to eleven, and it was not hedging when it said so.

That is not a bug so much as a category error on my part. Counting characters is a lexer’s job, and I handed it to a model because the model was right there. The same goes for arithmetic and dates. It cannot count, so anything that hinges on counting stays in code where it belongs.

The subtler version of the same lesson: confidence is not accuracy. It will hand you a 1.00 on a judgment call where two reasonable people would pick different answers, because the distribution really is peaked, and a peaked distribution is not the same thing as a correct one. Ask one question at a time, keep each question to a single fault, and treat a confident answer to a vague question as your fault rather than the model’s.

What are the hard limits?

Three, and the first one rules out more candidates than the other two combined. It cannot work on images or audio. State is text, objects or arrays, so anything that scores a rendered picture is out. That was exactly the fast judge job I assumed Jev would walk into, and it cannot touch it. Fast at judging does not mean fast at my judging.

Second, there is no prose output, which sounds obvious until you walk your own candidate list and find how many of them need one human-readable sentence at the end.

Third, it is not OpenAI compatible. Its own endpoint, its own body shape. You cannot swap a model name in a config file, so every use is new code. That is a real cost and worth knowing before you get excited. It is also why Smart Inference cannot route it today: our router speaks the OpenAI shape across providers, and Jev deliberately does not.

Why is it priced like this?

Because the price is the argument, not a launch discount. Jev costs $0.042 per million input tokens, and output is free, which TypeSafe describe as too cheap to meter.

They named it after William Stanley Jevons, which is not subtle and is not meant to be. His paradox says that making a resource cheaper raises total consumption instead of lowering it. At that price, decisions nobody would route through a model today become things you call a thousand times inside a single request. The transactions are tiny and so are the costs, which tells you they are playing a long game rather than a quarterly one. I find that a more convincing reason to start a model company than another chat window.

Does this open the gate for a whole industry?

I think it does, and that is the part I actually care about. I have been saying for years that small specialized models should be the norm, and it is genuinely good to see somebody build the industry that way instead of shipping one more general model with a bigger context window.

I do not always want a reasoning model arguing with itself for a minute. Sometimes I want a confident answer in 250 milliseconds, and until now the only way to get one was to take something enormous and talk it down into a single word. Jev gives me a new lever to pull, which is not something I have been able to say in a while. If it works, there is no reason for it to be the only one.

Kudos to the team at TypeSafe AI. Anyway, I will keep you posted as always…

FAQ

What is Jev? Jev is TypeSafe AI’s System-One model. It answers typed questions about text and returns a choice, a score, or a yes/no probability, with a confidence value on every answer. It does not generate text.

Can Jev replace a chat model? Only for typed decisions. Use it when the answer is one of a fixed set, when code acts on the answer, when the input is text or JSON, and when a confidence score would let that code pick a fallback. If any of those four fail, use a chat model.

How much does Jev cost? $0.042 per million input tokens, with output free. A 40 question call over a 31.7k token input cost about a tenth of a cent.

How fast is Jev? In my intent gate bench the median was 250 ms and the 95th percentile ran between 453 and 688 ms across 207 runs.

What can Jev not do? It cannot read images or audio, it cannot output prose, and it is not OpenAI compatible, so every integration is new code. It also cannot count, so never ask it about numbers, dates, or literal characters.