Classifier model comparison

Classifier models answer a fixed question with a fixed set of answers instead of writing text. Decision models answer any yes/no, choice, or score question you write. Guardrail classifiers answer one built-in question: is this content safe, or is this an attack. Model routers use a classifier to pick which model answers each request.

Facts come from each vendor's own docs and model cards, linked below. Latency, price, and accuracy figures are the vendor's or source's claims, not independent tests. A head-to-head benchmark on this site's use cases is planned.

Decision models

5 models · any typed question

Model	Question types	Max options	Access	Open weights	Price	Latency (as reported)	Status
Jev 1.13TypeSafe AI	Yes/no, Choice, Score	255	API, OpenRouter	No	$0.042/1M input tokens (output free)	154 ms median (vendor)	Not stated (Eden AI: alpha)
OpenAI Decisions API (GPT-6 Luna)OpenAI	Yes/no, Choice, Score	—	API	No	$0.10/1M input tokens	~10x faster than Responses (vendor)	Public beta
ClefCloudflare	Yes/no, Choice, Score	—	API, Open weights, Local	Yes	$0.24/1M input tokens	209.3 ms median (vendor)	Released; beta not stated
Clef FlashCloudflare	Yes/no, Choice, Score	—	API, Open weights, Local (Ollama)	Yes	—	38.8 ms median (vendor)	Released (hosted + open weights)
Nimble 9BBespoke Labs	Yes/no, Choice, Score	26	Open weights, Local (Ollama)	Yes	Free (open weights)	Under 100 ms on M5 Max (vendor)	Available in Ollama library

Guardrail classifiers

12 models · safety and attack detection

Model	Classifies	Output	Size	Open weights	License	Access
Llama Guard 4Meta	Prompt/response safety, 14 categories	'safe'/'unsafe' + category codes	12B (dense)	Yes	Llama 4 Community License	Hugging Face (gated)
ShieldGemma (2B / 9B / 27B)Google	Policy violation; 4 harm types	'Yes'/'No' + optional score	2B (9B, 27B siblings)	Yes	Gemma license (gated)	Hugging Face (gated)
ShieldGemma 2Google	Image safety, 3 policies	Yes/No probability per policy	4B	Yes	Gemma license (gated)	Hugging Face (gated)
Granite Guardian 4.1 8BIBM	Harm, hallucination, custom criteria	Yes/No in <score> tags	8B	Yes	Apache 2.0	Hugging Face, Ollama
Nemotron 3.5 Content SafetyNVIDIA	Prompt/response safety (Aegis 2.0)	User/Response Safety + categories	4B (Gemma-3-4B-it base)	Yes	OpenMDW v1.1 + Gemma Terms	Hugging Face, NVIDIA NIM, build.nvidia.com
Qwen3Guard-Gen (0.6B / 4B / 8B)Alibaba (Qwen)	3 severity levels, 8 categories	Safety/Categories/Refusal lines	0.6B, 4B, 8B	Yes	Apache 2.0	Hugging Face, vLLM / SGLang
gpt-oss-safeguard (20b / 120b)OpenAI	Developer-supplied written policy	Classification + reasoning trace	20b (21B total); 120b (117B total)	Yes	Apache 2.0 + gpt-oss usage policy	Hugging Face, Ollama, vLLM
OpenAI Moderation API (omni-moderation-latest)OpenAI	13 categories; text and image	JSON flags + scores 0-1	—	No	Proprietary API	API
Llama Prompt Guard 2 (86M / 22M)Meta	Jailbreak/prompt-injection attempts	Binary: benign vs malicious	86M (22M English variant)	Yes	Llama 4 Community License	Hugging Face (gated)
ProtectAI deberta-v3-base-prompt-injection-v2Protect AI	Prompt injection in input text	Binary: 0 benign / 1 injection	~200M (DeBERTa-v3-base)	Yes	Apache 2.0	Hugging Face
Azure AI Content Safety (incl. Prompt Shields)Microsoft	Text/image harm; Prompt Shields	Per-category severity levels	—	No	Proprietary API	API, Content Safety Studio
Shieldstral 1.0Mistral AI	Content vs natural-language policy	Yes/no token + softmax score	3B (4B in file metadata)	Yes	Apache 2.0	Hugging Face, Mistral API

Model routers

8 routers · pick a model per request

Model	Routes between	Decides	Access	Open weights	Price	Status
TypeSafe Jev RouterTypeSafe	OpenRouter model ecosystem	Model + reasoning effort	OpenRouter API	—	Variable (OpenRouter lists -1)	Live; 1M context
OpenRouter Auto RouterOpenRouter	OpenRouter models by task type	Model, by cost tier	OpenRouter API	No	Selected model's rate; no fee	Live (beta track too)
Not DiamondNot Diamond	Models you configure	Which model to call	REST API, Python / TS SDK	No	$0.05 per 1M tokens routed	Live
RouteLLMLMSYS (lm-sys)	Strong vs weak model	Strong or weak model	Python library, OpenAI-compatible server	Yes	Free; pay the models	Open source
Azure Foundry Model RouterMicrosoft	OpenAI, Claude, xAI, DeepSeek +	Model + routing mode	Foundry deployment	No	See Azure pricing page	Live; v2025-11-18
Amazon Bedrock Intelligent Prompt RoutingAmazon Web Services	Two models, same family	Which of two models	Bedrock console, AWS CLI / SDK	No	See Bedrock pricing	GA
vLLM Semantic RoutervLLM project	Your configured backends	Model per call, by policy	GitHub, Envoy / Kubernetes	Yes	Free; pay your backends	Open source; v0.4
Arch-RouterKatanemo	Models mapped to your routes	Route (domain/action) per query	Hugging Face, Arch gateway	Yes	Free; pay your models	Open weights

Published results

As reported by each source. Different test sets, so scores are not comparable across rows.

Jev 1.13: Red Hat prompt injection (EvalHub NeMo Guardrails), accuracy, Jev-1.13.0 via API 86.35% (rank 4 of 9) (source); Red Hat content safety (toxicity-profanity-safety), accuracy 86.20% (rank 1 of 9) (source); Nimble authors' held-out set agreement (Bespoke Labs figure) 93.21% for Jev 1.13.0 (source); Nimble authors' mean accuracy over 13 public datasets (Bespoke Labs figure) 76.0% (source); Cloudflare Jev Decision Index, e.g. BANKING77 macro-F1 (Cloudflare figure) 79.74 (Clef 94.20) (source); MediaPipe Decision Maker test (author-reported on madewithjev.com) 92% for Jev vs 75% for MediaPipe alone (source)
Clef: Cloudflare-run Jev Decision Index, BANKING77 macro-F1 (vendor figure) 94.20 (Jev 79.74) (source); Cloudflare-run Jev Decision Index, BFCL case exact (vendor figure) 98.47 (Jev 95.75) (source); Cloudflare-run Jev Decision Index, When2Call accuracy (vendor figure; Jev ahead) 72.37 (Jev 80.97) (source); Decision Index 0.2.1 (chance-corrected, community page) not listed for Clef in fetched table (source)
Clef Flash: Cloudflare-run Jev Decision Index, BANKING77 macro-F1 (vendor figure) 90.93 (Jev 79.74) (source); Cloudflare-run Jev Decision Index, CLINC150+OOS macro-F1 (vendor figure) 66.77 (Jev 89.27) (source)
Nimble 9B: Mean accuracy across 13 public datasets (Bespoke Labs figure) 75.7% (Jev 1.13: 76.0%) (source); Held-out set agreement (Bespoke Labs figure) 90.12% (292/324) vs Jev 93.21% (source); Public benchmark averages by type (Bespoke Labs figure) choice 81.6%, boolean 80.2%, score 54.6% (source); Decision Index 0.2.1 (Nimble 9B v2 repo; chance-corrected) 39.57 (ECE 0.024) (source)
Llama Guard 4: Meta in-house test set, English, output filtering (vendor claim): recall 69%, FPR 11%, F1 61% F1 61% (source); Meta in-house test set, multilingual (vendor claim) recall 43%, FPR 3%, F1 51% (source)
ShieldGemma (2B / 9B / 27B): ShieldGemma-2B optimal F1 / AU-PRC, SG Prompt (vendor claim) 0.825 / 0.887 (source); ShieldGemma-2B, OpenAI Mod 0.812 / 0.887 (source); ShieldGemma-2B, ToxicChat 0.704 / 0.778 (source); ShieldGemma-2B, SG Response 0.743 / 0.802 (source)
ShieldGemma 2: Google internal synthetic benchmark (~500 examples/policy), optimal F1 (vendor claim): sexually explicit 88.6, dangerous 93.7, violence/gore 85.0 85.0-93.7 F1 (source)
Granite Guardian 4.1 8B: OOD safety aggregate F1 (model card) 0.79 no-think / 0.78 think (source); LM-AggreFact RAG avg balanced accuracy 0.760 no-think / 0.764 think (source); FC Reward Bench balanced accuracy 0.79 no-think / 0.78 think (source)
Nemotron 3.5 Content Safety: Model card (vendor): Aegis 2.0 / Wildguard prompt F1 ~0.85-0.86 see card (source); Red Hat benchmark, prompt injection, default vs custom policies (third party) 69.37% vs 84.84% accuracy; median 239.6/240.4 ms (source); Red Hat benchmark, content safety 84.67% default / 85.07% custom accuracy; ~230-242 ms (source)
Qwen3Guard-Gen (0.6B / 4B / 8B): Paper claims state-of-the-art prompt/response classification (vendor claim); no numbers on the model card or arXiv abstract. Shieldstral's card lists Qwen3Guard-8B best on WildGuardTest prompt F1 88.2 and BeaverTails response 85.9 (third-party table) 88.2 / 85.9 F1 (source)
gpt-oss-safeguard (20b / 120b): OpenAI internal multi-policy benchmark, accuracy (vendor claim): gpt-oss-safeguard-120b vs GPT-5 46.3% vs 43.2% (source); Shieldstral card table (third party): OpenAI Moderation prompt F1 84.0; WildGuardTest response 80.7; HarmBench response 88.2 for GPT-OSS-Safeguard-20B 80.7-88.2 F1 (source)
Llama Prompt Guard 2 (86M / 22M): 86M direct jailbreak detection, English AUC (vendor claim) 0.998 (source); 86M recall at 1% FPR, English 97.5% (source); 86M multilingual AUC 0.995 (source)
ProtectAI deberta-v3-base-prompt-injection-v2: Model card (vendor): 20,000 held-out prompts, accuracy / F1 95.25% / 95.49% (source); Red Hat benchmark (third party), prompt-injection accuracy; Qwen3.6-35B scored 89.31% 89.01% (source)
Shieldstral 1.0: Model card (vendor): WildGuardTest prompt F1 88.1, ToxicChat 84.1, Aegis v2 86.2, HarmBench 99.4 84.1-99.4 F1 (source); Red Hat benchmark, prompt injection (Table 1; appendix conflicts) 72.02%; median 187.9 ms (source); Red Hat benchmark, content safety (Table 2; appendix conflicts) 74.80%; median 191.9 ms (source)
RouteLLM: Cost reduction at 95% of GPT-4 performance on MT Bench (vendor) up to 85% (source)
Amazon Bedrock Intelligent Prompt Routing: Cost reduction without losing accuracy (vendor) up to 30% (source)
vLLM Semantic Router: Accuracy gain on MMLU-Pro vs direct vLLM inference (authors' paper) +10.2 points (source); Latency cut vs direct vLLM inference (authors' paper) 47.1% (source)

Model details and sources

Jev 1.13 TypeSafe AI

Jev is TypeSafe's closed, hosted decision model that returns probabilities for yes/no, pick-one and score questions at $0.042 per million input tokens.

Released: 2026-09-15
Question types: Yes/no, Choice, Score
Max options: 255
Context: 64k tokens per request (state + all questions); 32k for state plus longest question; text only
Latency: Vendor-reported (via OpenRouter Ori Eval, as cited on madewithjev.com): median 154 ms vs 860 ms for GPT-5.6 Luna. Cloudflare measured Jev median 524.1 ms (p95 536.0 ms) across 43 benchmarks; Red Hat measured median 348.1 ms (prompt injection) and 360.4 ms (content safety) over a remote API from a UK client.
Price: $0.042 per 1M input tokens; output tokens free (direct API). Same $0.042/1M listed on Eden AI's page for typesafe/jev-latest.
Access: Hosted API (api.typesafe.ai/v1/systemone), OpenRouter (typesafe/jev-1.13)
Open weights: No
Status: Direct API: status not stated in docs. Eden AI page describes its Jev decisions endpoint as alpha (shapes, IDs, pricing may change without deprecation).

Sources: Jev API docs (endpoint, question types, 255 choice options, score 2-10 levels), Jev models page (64k/32k context, pricing, rate limits, text only), Made with Jev launch page (15 Sep 2026, latency/other claims, weights unpublished), Eden AI decisions docs (typesafe/jev-latest, alpha warning, 255 options, price), OpenRouter Jev recipe (confirms typesafe/jev-1.13 id), Red Hat benchmark, Cloudflare Clef blog, Ollama Nimble page

OpenAI Decisions API (GPT-6 Luna) OpenAI

OpenAI's Decisions API answers predicate, choice and score questions with probabilities using the gpt-6-luna model, currently in public beta.

Question types: Yes/no, Choice, Score
Context: Not stated numerically; accepts text and inline base64 images (no hosted image URLs or file_id)
Latency: Vendor claim: about 10x faster than the Responses API; no absolute figures. Note: OpenRouter's Ori Eval, as cited on madewithjev.com, reports median 860 ms for GPT-5.6 Luna (third-party-reported, different setup).
Price: $0.10 per 1M input tokens; no output, cache-read or cache-write charges; regional processing premiums and long-context multipliers apply
Access: Hosted API (POST /v1/decisions)
Open weights: No
Status: Public beta; OpenAI expects GA 'in the coming weeks'

Sources: OpenAI Decisions guide, Made with Jev (Luna latency from OpenRouter eval)

Clef Cloudflare

Cloudflare's open-weight 27B multimodal decision model, hosted on Workers AI at $0.24 per million input tokens.

Released: 2026-10-01
Question types: Yes/no, Choice, Score
Size: 27B (post-trained from Qwen3.8-27B)
Context: 65,536 tokens on Workers AI (blog says 64k); multimodal, up to 4 images per request; 1-64 questions per request
Latency: Vendor-reported by Cloudflare across 43 eval benchmarks: median 209.3 ms, p95 238.6 ms
Price: $0.24 per 1M input tokens; no charge for output tokens (Workers AI model page)
Access: Hosted API (Workers AI @cf/cloudflare/clef), Open weights (Hugging Face), Local via SGLang or Ollama
Open weights: Yes
License: Apache-2.0
Status: Released as hosted Workers AI model and open weights; beta status not stated on model page

Sources: Cloudflare blog announcement (date, latency, benchmarks, Apache 2.0), Workers AI Clef model page (price, 65,536 context, limits), Hugging Face Clef (27B, Apache-2.0, /v1/systemone via SGLang), llmconfigurator guide (Ollama 0.35 added clef, clef-flash)

Clef Flash Cloudflare

The smaller, faster Cloudflare decision variant, with a vendor-reported median latency under 40 ms.

Released: 2026-10-01
Question types: Yes/no, Choice, Score
Size: Backbone Qwen3.5-9B (total parameter count not stated)
Latency: Vendor-reported by Cloudflare across 43 eval benchmarks: median 38.8 ms, p95 122.4 ms
Access: Hosted API (Workers AI @cf/cloudflare/clef-flash), Open weights (Hugging Face), Local via Ollama
Open weights: Yes
License: Apache-2.0
Status: Released as hosted Workers AI model and open weights

Sources: Cloudflare blog announcement, Workers AI Clef page (clef-flash accepted as model param; no price/limits given), Hugging Face Clef (Clef-Flash is smaller variant), llmconfigurator guide (Ollama 0.35)

Nimble 9B Bespoke Labs

Nimble is an Apache-2.0 9B open decision model from Bespoke Labs that runs locally through Ollama's Jev-compatible System One endpoint.

Question types: Yes/no, Choice, Score
Max options: 26
Size: 9B (fine-tuned from Qwen3.5-9B)
Context: 256K window listed by Ollama, but each question's prompt must fit 8,192 tokens; up to 64 questions and 64 KiB request body
Latency: Vendor claim (Ollama page): answers in under 100 ms on a MacBook Pro with M5 Max
Price: Free to run locally (open weights)
Access: Open weights / local (ollama pull nimble), Ollama /v1/systemone endpoint
Open weights: Yes
License: Apache-2.0
Status: Available in Ollama library (Ollama 0.35 added it, 29 Sep 2026)

Sources: Ollama Nimble library page, llmconfigurator guide (Ollama 0.35 date, Index score)

Llama Guard 4 Meta

Meta's 12B multimodal safe/unsafe classifier that names which of 14 hazard categories were violated.

Classifies: Prompt and response safety against 14 MLCommons-based hazard categories (S1 Violent Crimes ... S14 Code Interpreter Abuse); text and images
Output: Text: 'safe' or 'unsafe' followed by violated category codes (e.g. 'unsafe S9')
Size: 12B (dense, pruned from Llama 4 Scout)
Languages: English plus multilingual text; evaluated on French, German, Hindi, Italian, Portuguese, Spanish, Thai
Access: Hugging Face (gated)
Open weights: Yes
License: Llama 4 Community License

Sources: Hugging Face model card

ShieldGemma (2B / 9B / 27B) Google

Google's Gemma-2-based text policy classifier that answers Yes/No on four harm types.

Released: 2024-07-31
Classifies: Whether a prompt or response violates a supplied policy; 4 harm types: sexually explicit, dangerous content, hate speech, harassment
Output: Text starting 'Yes' or 'No'; optional score = probability from Yes/No token logits
Size: 2B (listed as 3B in safetensors metadata); 9B and 27B siblings
Languages: English only
Access: Hugging Face (gated)
Open weights: Yes
License: Gemma license (gated, accept Google terms)

Sources: Hugging Face model card (release date from Gemma release log; paper 2024-07-31), Gemma release log

ShieldGemma 2 Google

Google's image-only safety classifier built on Gemma 3 4B.

Released: 2025-03-10
Classifies: Image safety against 3 policies: sexually explicit, dangerous content, violence/gore
Output: Probability from Yes/No next-token likelihood per policy
Size: 4B
Access: Hugging Face (gated)
Open weights: Yes
License: Gemma license (gated)

Sources: Hugging Face model card, Gemma release log (ShieldGemma 2 listed 2025-03-10 with Gemma 3)

Granite Guardian 4.1 8B IBM

IBM's Apache-licensed 8B judge for harm, RAG groundedness and agent tool-call risks, English only.

Released: 2026-04-29
Classifies: Harm (social bias, jailbreaking, violence, profanity, unethical behavior), RAG hallucination (context relevance, groundedness, answer relevance), function-calling hallucination, bring-your-own criteria
Output: Yes/No inside <score></score>; optional <think> reasoning
Size: 8B
Languages: English only
Access: Hugging Face, Ollama (per third-party guide)
Open weights: Yes
License: Apache 2.0

Sources: Hugging Face model card (release 'April 2026'), Granite 4.1 family date 2026-04-29 (third-party coverage)

Nemotron 3.5 Content Safety NVIDIA

NVIDIA's 4B multilingual, multimodal guard that also accepts custom policies.

Released: 2026-06-02
Classifies: Prompt and response safety (text and optional image) using Aegis 2.0 taxonomy, plus custom user-defined policies with optional reasoning
Output: Text lines: User Safety (safe/unsafe), Response Safety, Safety Categories (comma list); optional <think> trace in custom-policy mode
Size: 4B (Gemma-3-4B-it base)
Context: 128K tokens
Languages: 12: English, Arabic, German, Spanish, French, Hindi, Japanese, Thai, Dutch, Italian, Korean, Chinese
Latency: No numeric latency on model card
Access: Hugging Face, NVIDIA NIM container (NGC), build.nvidia.com
Open weights: Yes
License: OpenMDW License v1.1 plus Gemma Terms of Use (NGC page lists NVIDIA Nemotron Open Model License instead)

Sources: Hugging Face model card, Search listing for NGC/NIM and license discrepancy, Red Hat article

Qwen3Guard-Gen (0.6B / 4B / 8B) Alibaba (Qwen)

Alibaba's multilingual guard family that returns a three-level verdict plus categories, with a streaming variant.

Released: 2025-10-16
Classifies: Prompt and response safety with 3 severity levels (Safe/Controversial/Unsafe); 8 categories (Violent, Non-violent Illegal Acts, Sexual Content, PII, Suicide & Self-Harm, Unethical Acts, Politically Sensitive Topics, Copyright Violation), Jailbreak (input only), refusal detection for responses
Output: Text lines: 'Safety:', 'Categories:', and 'Refusal:' for responses. A Stream variant adds token-level moderation during generation
Size: 0.6B, 4B, 8B
Context: Serving examples use 32768 tokens (not a stated spec)
Languages: 119 languages and dialects
Access: Hugging Face, vLLM / SGLang
Open weights: Yes
License: Apache 2.0

Sources: Hugging Face model card, arXiv technical report (date 2025-10-16), Shieldstral card comparison table

gpt-oss-safeguard (20b / 120b) OpenAI

OpenAI's open-weight reasoning models that judge content against whatever policy text you give them.

Released: 2025-10-29
Classifies: Any developer-supplied written policy applied at inference time to messages, completions or full chats
Output: Classification with an inspectable reasoning trace (harmony format)
Size: 20b (21B total, 3.6B active); 120b (117B total, 5.1B active)
Latency: Reasoning model, slower and heavier than small classifiers (vendor says classifiers trained on tens of thousands of labels can beat it on complex tasks)
Access: Hugging Face, Ollama, LM Studio, vLLM, Groq via HF Inference Providers, Azure AI catalog
Open weights: Yes
License: Apache 2.0 plus gpt-oss usage policy

Sources: Hugging Face model card, Launch coverage (date 2025-10-29, benchmark), OpenAI technical report page (not fetched; referenced via search), OpenAI help center: not served via API

OpenAI Moderation API (omni-moderation-latest) OpenAI

OpenAI's free hosted moderation endpoint returning per-category flags and scores for text and images.

Classifies: 13 categories: harassment, harassment/threatening, hate, hate/threatening, illicit, illicit/violent, sexual/minors, self-harm (+intent, +instructions), sexual, violence, violence/graphic; text and image
Output: JSON: flagged boolean, per-category booleans, per-category scores 0-1, applied input types
Context: Image files up to 20 MB; text limit not stated
Price: Free to use (per OpenAI docs)
Access: OpenAI API /moderations endpoint
Open weights: No
License: Proprietary API

Sources: OpenAI moderation guide

Llama Prompt Guard 2 (86M / 22M) Meta

Meta's tiny prompt-attack detector with a 512-token window.

Classifies: Direct jailbreak / prompt-injection attempts in prompts
Output: Binary label benign vs malicious
Size: 86M (mDeBERTa-based, listed 0.3B on Hub); 22M English-focused variant
Context: 512 tokens (split longer inputs)
Languages: 86M: English, French, German, Hindi, Italian, Portuguese, Spanish, Thai; 22M for English-only
Latency: 92.4 ms per classification on A100, 512 tokens (86M, model card)
Access: Hugging Face (gated)
Open weights: Yes
License: Llama 4 Community License (gated)

Sources: Hugging Face model card, Meta Prompt Guard docs (22M/86M sizes)

ProtectAI deberta-v3-base-prompt-injection-v2 Protect AI

A small English-only DeBERTa classifier for prompt injection; the card says the project is archived.

Classifies: Prompt injection in input text
Output: Binary label 0 benign / 1 injection-detected
Size: ~200M (DeBERTa-v3-base)
Context: 512 tokens used in examples (not stated as limit)
Languages: English only
Latency: Red Hat benchmark: median 54.1 ms (Table 1; appendix says 80.4 ms), on MacBook Pro M1 CPU
Access: Hugging Face
Open weights: Yes
License: Apache 2.0

Sources: Hugging Face model card (archived notice), Red Hat benchmark article

Azure AI Content Safety (incl. Prompt Shields) Microsoft

Microsoft's hosted moderation and prompt-attack API with severity scores.

Classifies: Text and image harm detection (sexual, violence, hate, self-harm) with multi-severity levels; Prompt Shields for user-input attacks; protected material; groundedness (preview); task adherence; custom categories
Output: Per-category severity levels; Prompt Shields returns attack detection
Languages: Trained/tested on Chinese, English, French, German, Spanish, Italian, Japanese, Portuguese; protected material, groundedness and custom categories (standard) are English only
Price: Free F0 tier: 5,000 text records and 5,000 images per month (Prompt Shields included in text). Standard S0 per-1,000 prices not shown on the pricing page as fetched
Access: Azure AI Content Safety API, Content Safety Studio
Open weights: No
License: Proprietary API

Sources: Azure docs overview, Azure pricing page

Shieldstral 1.0 Mistral AI

Mistral's 3B policy-adaptive multimodal safety scorer.

Released: 2026-08-04
Classifies: Text, image or text+image content against a natural-language policy
Output: Single yes/no token; softmax over logits gives a continuous score
Size: 3B (4B in file metadata)
Context: Trained to 32k tokens (recommended limit)
Languages: 12: English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic, Russian
Latency: No latency figures; fits in 16GB VRAM
Access: Hugging Face, Mistral API (per third-party coverage)
Open weights: Yes
License: Apache 2.0

Sources: Hugging Face model card, Release date 2026-08-04 (third-party coverage), Red Hat article

TypeSafe Jev Router TypeSafe

Picks the best model and reasoning effort for each request, balancing quality, speed and cost (vendor).

Released: 2026-09-25
Routes between: Models across the OpenRouter ecosystem (specific models not named on the page)
Decides: Which model and which reasoning effort answers each request
How it decides: Runs on Jev, TypeSafe's first System One model; adapts as the conversation evolves (vendor)
Price: Not stated; OpenRouter lists price as -1 (variable)
Access: OpenRouter API (typesafe/jev-router), Chat Completions endpoint
Status: Live on OpenRouter; 1M context listed

Sources: OpenRouter model page, OpenRouter models API (created 1790363560 = 2026-09-25; context 1000000; pricing -1), OpenRouter API guide

OpenRouter Auto Router OpenRouter

Routes each prompt by task type using what the OpenRouter community collectively spends on each model.

Released: 2023-11-08
Routes between: Ranked candidates per task type from models on OpenRouter, filtered by your allow/exclude lists, cost tier and policies
Decides: Which model answers (cost tier low/medium/high/xhigh/max); beta track tests new behaviors
How it decides: Classifies each prompt into about 30 task types, then ranks models by their share of OpenRouter spend for that task over a trailing 7 days; docs do not mention Not Diamond (vendor)
Price: No additional fee; pay the selected model's standard rate
Access: OpenRouter API (openrouter/auto), openrouter/auto-beta (early access)
Open weights: No
Status: Live; beta track available

Sources: OpenRouter Auto Router docs, OpenRouter models API (created 1699401600 = 2023-11-08)

Not Diamond Not Diamond

Hosted router that recommends the best model per request, with optional custom routers and prompt optimization.

Routes between: Models you configure; pre-trained or custom router (full list not on docs page)
Decides: Which model to call; returns a recommendation, you make the call via your own gateway
How it decides: Pre-trained router or custom routers trained on your data; mechanism not detailed
Price: $0.05 per million tokens routed (pay-as-you-go, vendor pricing page); enterprise custom
Access: REST API, Python SDK, TypeScript SDK
Open weights: No
Status: Live; pay-as-you-go access by application per pricing page

Sources: Not Diamond pricing, Not Diamond docs, Third-party summary (no open-source or self-host found)

RouteLLM LMSYS (lm-sys)

Open-source framework that routes each query between a strong and a weak model using routers trained on preference data.

Released: 2024-06-26
Routes between: Two models: a strong and a weak one you choose
Decides: Whether to send each query to the strong or the weak model
How it decides: Routers trained on preference data: mf (matrix factorization), sw_ranking (weighted Elo), bert classifier, causal_llm classifier
Price: Free software; you pay the models you call
Access: Open-source Python library, OpenAI-compatible server
Open weights: Yes
License: Apache-2.0
Status: Open source; repo active (5.6k stars)

Sources: RouteLLM GitHub, Paper (v1 2024-06-26)

Azure Foundry Model Router Microsoft

A trained model deployed like any Foundry model that picks the underlying LLM per prompt, with quality, balanced and cost modes.

Routes between: About 38 models from OpenAI, Anthropic, xAI, DeepSeek, Meta and Fireworks (current version 2025-11-18), or a subset you pick
Decides: Which model answers; routing mode Quality, Balanced (default) or Cost; reasoning vs non-reasoning model
How it decides: Trained language model that analyzes complexity, reasoning and task type in real time; routing decision uses text input only
Price: Input prompts charged at the rate on the Azure pricing page; amount not retrieved
Access: Foundry model deployment, Global Standard / Data Zone Standard
Open weights: No
Status: Live; version 2025-11-18 active, older versions frozen

Sources: Microsoft Learn: model router

Amazon Bedrock Intelligent Prompt Routing Amazon Web Services

Routes between two models in the same family by predicting which will give the better response for the cost.

Released: 2025-04-22
Routes between: Two models within one family (default routers use Nova or Llama; configured routers add models such as Claude)
Decides: Which of two same-family models answers, using a response-quality-difference threshold and a fallback model
How it decides: Predicts the response quality of each model for the prompt, then picks the best quality/cost combination; English-optimized
Price: Routing price not stated on page; see Bedrock pricing page
Access: Bedrock console, AWS CLI / SDK (CreatePromptRouter)
Open weights: No
Status: GA since 2025-04-22

Sources: AWS docs, AWS product page, AWS GA announcement

vLLM Semantic Router vLLM project

Open-source decision layer that routes each request to a model based on policy and request complexity.

Routes between: Configured backends across local, private and cloud models
Decides: Which model handles each call, per your quality, latency and cost policy
How it decides: Policy-driven decision layer; earlier project posts describe a lightweight classifier (ModernBERT) judging intent and complexity
Price: Free software; you pay your backends
Access: Open-source (GitHub), Envoy / Kubernetes integration
Open weights: Yes
License: Apache-2.0
Status: Open source; v0.4 reported 2026-09-24

Sources: GitHub repo, Paper, vLLM blog

Arch-Router Katanemo

Small open-weights model that routes a query to the model matching your own routing preferences.

Routes between: Models you map to your own domain/action routes
Decides: Which user-defined route (and so which model) fits the query
How it decides: 1.5B generative model fine-tuned from Qwen2.5-1.5B-Instruct that maps a query to your domain and action policy, returning a JSON route name
Price: Free weights; you pay the models you route to
Access: Hugging Face weights, Arch gateway (open source)
Open weights: Yes
License: Katanemo license (katanemo-research)
Status: Open weights; paper published 2025-06-19

Sources: Hugging Face model card, Paper

Data as JSON: /api/models.json. Spot an error? Tell us.