Work note 01 of 04
An AI-written rule engine that still answers deterministically
Eligibility across four dimensions, with far too many combinations to write a case for each — solved by moving the language model out of the request path and having it produce storable rules instead of verdicts.
- Context
- B2B2C platform, numerous markets worldwide
- My role
- Lead engineer on this component, in a small team alongside the client
- Status
- Live and in use.
- Stack
- PayloadCMS
- TypeScript
- zod
- OpenAI
- REST
The requirement was eligibility. Given who the customer is, who the seller is, and where and when an interaction takes place, decide whether it qualifies. The client wanted that to stay flexible rather than be fixed in code, because writing a case for each combination of those four dimensions was not something anyone could do up front.
The first attempt, and why it failed
The obvious approach put a language model directly in the request path: pass it the customer information, ask it for a verdict. It did not survive contact with reality.
Calling a model over REST was far too slow for a decision that sits on every single interaction. And — the part that actually ruled it out — the results were not deterministic. A decision engine that can return two different answers for the same input is not a decision engine.
What I did instead
The model moved out of the request path entirely. Instead of asking it for a verdict, I had it produce an expression: a rule, in a defined grammar, that could be stored once and then evaluated at run time against the facts of a particular interaction.
That split the problem along a much better seam. Generation happens once, up front, where it can be reviewed before it takes effect. Evaluation is ordinary deterministic code — fast enough to sit on every interaction, and guaranteed to answer the same way every time.
Getting the model to emit valid rules
Moving the model one step back concentrates all the remaining difficulty in a single place: getting it to emit expressions of exactly the right shape. Valid functions rather than plausible-looking inventions, and a correct understanding of what the data can actually look like.
The answer was to stop writing that knowledge down more than once. One set of
zod schemas defines the shape, and three things consume it:
- validation of the data at run time,
- the generated OpenAPI documentation,
- and a structured JSON description passed into the system prompt, so the model knows the grammar it is writing for.
Because there is a single definition, the validator, the published documentation and the model’s own instructions cannot drift apart. The usual failure mode here is three descriptions of the same schema slowly disagreeing; with one source there is nothing to disagree.
What it generalises to
When a model is too slow or too non-deterministic to sit in the request path, it is often still the right tool one step back. Use it to produce an artefact instead of an answer: an artefact can be reviewed before it takes effect, stored, reused, and evaluated identically every time.
And when a model has to produce something schema-shaped, that schema should be the one the rest of the system already uses — not a second description of it written into a prompt.