Skip to content
Atmane Ayoub
← All work

Open source · my own project · 2026

Jev-AR: Arabic intent routing in 10 ms

A 322M-parameter model that routes Gulf-Arabic messages to the right agent, tool or knowledge base — in one forward pass, locally, with routes supplied at request time.

60-route accuracy
83.4%
Latency (p50, GPU)
~10 ms
Unseen-domain RAG routing
38 / 40

Role

Sole author: data, training, evaluation, release

The problem

Every agent, RAG app and support bot starts with the same decision: which route should this message go to? A tool, a knowledge base, a workflow, a human queue.

For Arabic, and especially for Gulf Arabic as people actually write it — Emirati, Saudi, MSA, and Arabic mixed with English — the usual options are poor:

  • A fixed intent classifier has to be retrained every time the route list changes.
  • An LLM with the routes in the prompt is accurate, but adds hundreds of milliseconds and a per-call cost to every single message, and sends user text to a remote API.

I wanted something in between: LLM-like flexibility on the route list, classifier-like speed and cost, running on your own hardware.

What I built

Jev-AR is a 322M-parameter decision model. You pass the user's message and the list of possible routes at request time. It scores every route jointly in one forward pass and returns:

  • the best route,
  • a probability for every route, and
  • an escalate flag for when it isn't confident enough to act.
router.route("وش آخر أخبار أسعار الذهب اليوم؟", tools)
# {'route': 'Web search', 'confidence': 0.88, 'escalate': False, 'probabilities': {...}}

New route list? No retraining.

How it works

  1. One sequence. The message and the routes are joined into one input, with a marker token before each route.
  2. Read together. An mmBERT-base encoder (307M) reads the whole sequence, so each route is judged against the message and against the other routes.
  3. One score per route. A 15M decision head scores each route at its marker. A temperature-scaled softmax — the temperature fitted on development data — turns scores into probabilities. Below 0.72 confidence, escalate is set.

Because routes are part of the input rather than fixed output classes, the model has to learn what a route means, not where it sits. I trained on randomised, shuffled and reworded route lists — route order, subsets, languages, names, descriptions and paraphrases — so it can't lean on position or exact wording. Shuffling the 60-route list changes 5.0% of its answers.

Training data

  • ~37K Arabic routing examples and ~44K multi-domain routing examples
  • 320 synthetic applications
  • Soft targets: probability distributions from Qwen3.8-27B, distilled into the small model

Jev-AR is fine-tuned from Laya multilingual, an open decision model with the same architecture. The Arabic training takes it from 42.0% to 83.4% on 60 routes, and from 13 to 38 of 40 on the unseen-domain test.

Results

Evaluated on Jev-AR Bench: 4,974 held-out queries in MSA, Emirati, Saudi and code-switched Arabic.

ModelParamsDeployment60 routes20 routesLatency (p50)
Qwen3.8-27B (LLM, routes in prompt)27BGPU server90.094.0634 ms
Jev 1.13 (closed API)undisclosedAPI86.291.4370 ms
Jev-AR322Mlocal83.490.010 ms
mmBERT-base, fine-tuned classifier307Mlocal76.579.86 ms
multilingual-E5-base + logistic regression278Mlocal72.477.44 ms
Laya multilingual (starting point)322Mlocal42.056.411 ms

Local models ran at batch 1 on an RTX 5090 Laptop GPU; API latencies are client-side round trips. 95% confidence interval for Jev-AR on 60 routes: 81.2–85.4.

Unseen-domain routing. On a kind of RAG app never seen in training — 6 routes, each question routed over an English and an Arabic route list (40 decisions) — Jev-AR scores 38 / 40, with zero answers changed when the list is reversed.

The trade-off in one line: 6.6 points behind a 27B LLM, at about 1/60th of the latency, with no per-call cost and no data leaving the machine. On CPU it runs in 0.15–0.45 s on a laptop.

Where it fits

Agents    message → Jev-AR → tool / sub-agent / workflow
RAG       query   → Jev-AR → knowledge base → retrieval → LLM
Support   message → Jev-AR → billing / claims / FAQ / human

Low-confidence decisions escalate to an LLM or a person — the router handles the easy majority cheaply and hands off the rest.

Reception

In the first week after release (25 Sep – 1 Oct 2026), 12 people in 7 countries requested access to the weights, mostly for research, and the benchmark was downloaded 436 times.

Limitations I'm honest about

  • The training data is synthetic and hasn't yet been validated by native speakers.
  • Confidence is calibrated on the built-in routes, so treat it as approximate on custom lists.
  • Completely unseen route semantics remain the area I most want to improve.

Weights are CC BY-NC 4.0 (non-commercial use, testing and evaluation); code is Apache-2.0; the benchmark is CC BY 4.0. Commercial licences are available — get in touch.

Arabic NLPRoutingAgentsRAGEvaluation

Working on something similar?

I'm open to senior and staff roles and to select advisory engagements.