ornotto · a book and a Python package

Ask a local model to pick an answer, not to write one

Send a message and a closed list of answers. The model returns one of them and a probability for every alternative, with no generated text to parse, repair or retry.

See the results Read the book

320methods measureda model, read one way on one engine or remote API

67queries in 30 languageseach asking for one of five tasks

65/67best local scoreRune q5 on pcdServer, level with jev in both modes

21.9 msfastest at 60/67 or bettergliner25-decide on Core ML CPU/GPU, 61/67

How a decision works

1. A state

   A message, a ticket or a slice of application data. The model reads it once.

2. A question and its answers

   A named question with a closed set of answers, such as docs, python or fea.

3. A probability for each answer

   The highest one is the choice. The others show how close the alternatives came.

Would you say a linear classifier hallucinates?

Diogo Almeida, TypeSafe, September 2026

Almeida asked this on jev's launch thread, when the discussion turned to whether a model that writes no text can hallucinate. None of the readouts in this book can return an answer outside your list. Each of them can still put a high probability on the wrong one, which is what chapter 9 measures.

What we measured

One router question, asked of every method: which of five tasks does a FontLab user want? The chart ranks the best eight by translated score, then direct score and speed. Repeated measurements of TypeSafe Jev occupy one rank; both runs remain in the full report. Local and remote runs retain their execution labels. Explore the speed/accuracy frontier, or read all 320 rows and every table.

For a fast first pass, laya on MLX scores 52/67 in 6.9 ms on translated text. Set your own accuracy floor and compare Pareto frontiers. Rune Q4 now measures 124.8 ms on GPU, against 487.1 ms with CPU weights. Rune Q5 full Metal adds 64/64 at 153.3 ms, 6.3× faster with one fewer correct answer in each mode than its mixed run. Ornith Splash scores 62/63 at 322.4/328.5 ms. C = CPU, G = GPU, C+G = mixed, R = remote; all other device codes are explained in the full report.

Correct answers out of 67, translated queries. Bars start at zero.

1. typesafe/jev-1.13 OpenRouter API · R · native choice65364 ms
2. rune-mradermacher-26b-a4b q5pcdServer · C+G65962 ms
3. jaredpalmer/kev-4b OpenRouter API · R · native choice651653 ms
4. rune-26b-a4b q3slot (llama-server) · G65367 ms
5. qwen3.5-4b-hmm q8pcdServer · G6488 ms
6. qwen3.5-4b-hmm q4pcdServer · G6493 ms
7. qwen3.5-4b-hmm bf16pcdServer · G6495 ms
8. togethercomputer/tev1-4b-experimental OpenRouter API · R · native choice64339 ms

+Show the full top 15

Each row is one ranked method; the two TypeSafe Jev measurements share one model entry. Rows are ranked by translated score, then direct score, then speed. Click a column header to sort.

The 15 best methods by translated score, then direct score, then speed. Click to sort.

Engine	C/G	Model	Family	Quant	GB	ms/query	Translated	Direct
OpenRouter API	R	typesafe/jev-1.13	dedicated			364.4	65/67	65/67
pcdServer	C+G	rune-mradermacher-26b-a4b	dedicated	q5	19.13	962.3	65/67	65/67
OpenRouter API	R	jaredpalmer/kev-4b	dedicated			1,653.0	65/67	65/67
slot (llama-server)	G	rune-26b-a4b	dedicated	q3	13.52	366.6	65/67	57/67
pcdServer	G	qwen3.5-4b-hmm	fine-tuned	q8	4.48	87.6	64/67	65/67
pcdServer	G	qwen3.5-4b-hmm	fine-tuned	q4	2.71	92.6	64/67	65/67
pcdServer	G	qwen3.5-4b-hmm	fine-tuned	bf16	8.42	94.8	64/67	65/67
OpenRouter API	R	togethercomputer/tev1-4b-experimental	dedicated			339.3	64/67	65/67
ollaya	G	winnow-12b	dedicated	q8_0	12.67	891.1	64/67	65/67
pcdServer	G	rune-mradermacher-26b-a4b	dedicated	q3	13.29	146.1	64/67	64/67
pcdServer	G	rune-mradermacher-26b-a4b	dedicated	q5	19.13	153.3	64/67	64/67
dohnuts (Metal)	G	decider-35b-a3b	dedicated	q4	21.17	266.3	64/67	64/67
System One server	G	neohorse-4b	dedicated	q8	5.17	273.1	64/67	64/67
slot (llama-server)	G	decider-35b-a3b	dedicated	q4	21.17	299.1	64/67	64/67
dohnuts (Metal)	G	jpt-4b	dedicated	q8	4.48	335.6	64/67	64/67

Local engines and remote models, one API

The package runs two open-source C++ engines on llama.cpp behind one Python API, drives ollaya if you install it, supports optional native Laya MLX/xDecision encoders, and calls the nine measured OpenRouter endpoints remotely. Chapter 3 explains how each one reads an answer.

dohnuts

Runs models trained for this job and reads their answer at the slot they were trained to fill. decider-0.8b scores 62/67 in about 51 ms.

How dohnuts reads an answer

pcdServer

Runs any chat GGUF and scores only the tokens of the answers you allow. Qwen3.5-4B-Hmm scores 65/67 on the original text in 88 ms.

How pcdServer constrains the answer

ollaya

A separate binary with its own model registry; ornotto drives it when it is installed. Ten of its models were measured, and winnow-12b scores 65/67 on the original text.

Setting up ollaya

The book in twelve chapters

Concepts

1. Deciding, not generating A model that points at one of the answers you gave it.
2. The shape of a decision The request and the answer in System One terms, and what changes on pcdServer.
3. Ways to read an answer Each readout takes its answer from a different place in the model.
4. Dedicated, fine-tuned and vanilla models Three families, sorted by where the answer is read.

Benchmarks

1. How we measured One router question, 67 queries, labels fixed before the run.
2. Results All 320 methods against jev, the hosted reference.
3. Quantization and size What precision changes: file size, memory and the answers that survive.
4. Speed, memory and caching One forward pass per decision, and what an engine avoids prefilling twice.
5. Confidence and fallback Whether the probabilities mean what they say, and what a fallback buys.

The package

1. The ornotto package What each object takes, and how the engines start and stop.
2. Typed decisions and pydantic-ai A Literal, a bool or a pydantic model as the question.
3. Choosing, building and contributing Which engine and model for which job, and how to send a change upstream.

Try it in two lines

uv add ornotto
python -c 'import ornotto; print(ornotto.choose("Build a kern feature for A V W T", ["docs", "python", "fea"]).value)'   # fea

The first call downloads decider-0.8b (0.81 GB) from Hugging Face and starts dohnuts on a loopback port. Later calls reuse both. For a yes/no question with its probability, use check:

answer = ornotto.check("Write me a script that renames glyphs", "Does the user want code?")
answer.value, answer.probability   # (True, 0.6)

Chapter 10 covers the rest of the API, and uv add "ornotto[pydantic-ai]" adds the pydantic-ai integration.