Open source · my own project · 2026
Jev-AR: Arabic intent routing in 10 ms
A 322M-parameter model that routes Gulf-Arabic messages to the right agent, tool or knowledge base — in one forward pass, locally, with routes supplied at request time.
- 60-route accuracy
- 83.4%
- Latency (p50, GPU)
- ~10 ms
- Unseen-domain RAG routing
- 38 / 40
Role
Sole author: data, training, evaluation, release
The problem
Every agent, RAG app and support bot starts with the same decision: which route should this message go to? A tool, a knowledge base, a workflow, a human queue.
For Arabic, and especially for Gulf Arabic as people actually write it — Emirati, Saudi, MSA, and Arabic mixed with English — the usual options are poor:
- A fixed intent classifier has to be retrained every time the route list changes.
- An LLM with the routes in the prompt is accurate, but adds hundreds of milliseconds and a per-call cost to every single message, and sends user text to a remote API.
I wanted something in between: LLM-like flexibility on the route list, classifier-like speed and cost, running on your own hardware.
What I built
Jev-AR is a 322M-parameter decision model. You pass the user's message and the list of possible routes at request time. It scores every route jointly in one forward pass and returns:
- the best route,
- a probability for every route, and
- an
escalateflag for when it isn't confident enough to act.
router.route("وش آخر أخبار أسعار الذهب اليوم؟", tools)
# {'route': 'Web search', 'confidence': 0.88, 'escalate': False, 'probabilities': {...}}
New route list? No retraining.
How it works
- One sequence. The message and the routes are joined into one input, with a marker token before each route.
- Read together. An mmBERT-base encoder (307M) reads the whole sequence, so each route is judged against the message and against the other routes.
- One score per route. A 15M decision head scores each route at its marker. A temperature-scaled softmax — the temperature fitted on development data — turns scores into probabilities. Below 0.72 confidence,
escalateis set.
Because routes are part of the input rather than fixed output classes, the model has to learn what a route means, not where it sits. I trained on randomised, shuffled and reworded route lists — route order, subsets, languages, names, descriptions and paraphrases — so it can't lean on position or exact wording. Shuffling the 60-route list changes 5.0% of its answers.
Training data
- ~37K Arabic routing examples and ~44K multi-domain routing examples
- 320 synthetic applications
- Soft targets: probability distributions from Qwen3.8-27B, distilled into the small model
Jev-AR is fine-tuned from Laya multilingual, an open decision model with the same architecture. The Arabic training takes it from 42.0% to 83.4% on 60 routes, and from 13 to 38 of 40 on the unseen-domain test.
Results
Evaluated on Jev-AR Bench: 4,974 held-out queries in MSA, Emirati, Saudi and code-switched Arabic.
| Model | Params | Deployment | 60 routes | 20 routes | Latency (p50) |
|---|---|---|---|---|---|
| Qwen3.8-27B (LLM, routes in prompt) | 27B | GPU server | 90.0 | 94.0 | 634 ms |
| Jev 1.13 (closed API) | undisclosed | API | 86.2 | 91.4 | 370 ms |
| Jev-AR | 322M | local | 83.4 | 90.0 | 10 ms |
| mmBERT-base, fine-tuned classifier | 307M | local | 76.5 | 79.8 | 6 ms |
| multilingual-E5-base + logistic regression | 278M | local | 72.4 | 77.4 | 4 ms |
| Laya multilingual (starting point) | 322M | local | 42.0 | 56.4 | 11 ms |
Local models ran at batch 1 on an RTX 5090 Laptop GPU; API latencies are client-side round trips. 95% confidence interval for Jev-AR on 60 routes: 81.2–85.4.
Unseen-domain routing. On a kind of RAG app never seen in training — 6 routes, each question routed over an English and an Arabic route list (40 decisions) — Jev-AR scores 38 / 40, with zero answers changed when the list is reversed.
The trade-off in one line: 6.6 points behind a 27B LLM, at about 1/60th of the latency, with no per-call cost and no data leaving the machine. On CPU it runs in 0.15–0.45 s on a laptop.
Where it fits
Agents message → Jev-AR → tool / sub-agent / workflow
RAG query → Jev-AR → knowledge base → retrieval → LLM
Support message → Jev-AR → billing / claims / FAQ / human
Low-confidence decisions escalate to an LLM or a person — the router handles the easy majority cheaply and hands off the rest.
Reception
In the first week after release (25 Sep – 1 Oct 2026), 12 people in 7 countries requested access to the weights, mostly for research, and the benchmark was downloaded 436 times.
Limitations I'm honest about
- The training data is synthetic and hasn't yet been validated by native speakers.
- Confidence is calibrated on the built-in routes, so treat it as approximate on custom lists.
- Completely unseen route semantics remain the area I most want to improve.
Weights are CC BY-NC 4.0 (non-commercial use, testing and evaluation); code is Apache-2.0; the benchmark is CC BY 4.0. Commercial licences are available — get in touch.
Working on something similar?
I'm open to senior and staff roles and to select advisory engagements.