Bosun is built for teams running agent memory or RAG pipelines who need a cheap, fast layer to prune stale or unsupported graph edges instead of a general-purpose LLM API. Each pair it judges tops out around 8,192 tokens, so it fits paragraph-length findings, not full documents. There's no vendor API to call: everything runs on infrastructure you control.
Bosun is Hanno Labs' programmable judge model for agent knowledge graphs, released in two parameter sizes. Rather than scoring against a fixed query like a standard reranker, it applies a natural-language rule supplied at inference time, correctly reversing its verdict when the rule is negated, with 0.945 WarrantBench steerability.
Provider: Hanno Labs · Family: Bosun
More about Hanno Labs on HokAI
Context window: 8,192 tokens
Input modalities: text · Output: text
About Bosun
Bosun is a programmable judge model released June 11, 2026 by Hanno Labs, a small AI research lab focused on causal intelligence and knowledge systems. Available in two sizes -- 0.6B (Bosun-XS) and 4B (Bosun-4B) -- both are LoRA fine-tunes of Qwen3-Reranker, trained to evaluate whether connections in an agent's knowledge graph are warranted: supported by evidence, non-redundant, and still factually current. Unlike general-purpose rerankers that score document relevance against a fixed query, Bosun is programmed at inference time by a natural-language instruction defining the specific rule to apply, allowing reprogramming per batch without fine-tuning. Version 1.1 expanded training data and added support for directional and typed-edge judgment, enabling the model to assess asymmetric relationships like supersession, dependency, support, and contradiction.
On the PAWS paraphrase adversarial benchmark, Bosun-4B v1.1 scores 0.91, compared to Gemini 3.1 Flash Lite at 0.81 and a blind baseline near 0.53. On FollowIR, which measures how well models adjust retrieval behavior based on per-query instructions, Bosun-4B achieves +17.9 in preference-MRR, placing first on that leaderboard; most standard retrievers score near or below 0 because they ignore the instruction signal entirely. On WarrantBench steerability -- the ability to flip a judgment when the instruction is negated -- Bosun-4B scores 0.945 and Bosun-XS scores 0.94, vastly outperforming Gemini 3.1 Flash Lite (0.575). On e-CARE causal direction, Bosun-4B reaches 0.85, closely matching Gemini (0.86). The one area where Bosun trails frontier LLMs is ANLI adversarial NLI: 0.57 for the 4B and 0.44 for XS, versus Gemini's 0.74, a known gap for smaller specialized models.
Both Bosun variants process input via the Qwen3-Reranker template: an instruction block defining the rule, a fixed query string, and a document block containing the two findings being compared. The model outputs a single calibrated probability P = sigmoid(logit_yes - logit_no) per pair. Effective max sequence length is approximately 8,192 tokens per pair, making Bosun well suited for paragraph-length findings but not multi-page documents. Long-context recall in the traditional sense does not apply since the model produces a scalar output, not a sequence.
Bosun is a text-in, score-out model. It accepts a text instruction and two text findings, and outputs a float in the range 0 to 1. It does not support vision, audio, structured JSON generation, or chat interaction. The instruction-following capability is the key differentiator: the model can be reprogrammed with any natural-language rule per batch, reversing its judgment when the rule is negated (0.97 negation accuracy for XS) and generalizing to rules never seen during training (0.95 novel-rule accuracy). Version 1.1 added directional typed-edge judgment -- supersession, depends-on, supports, and contradicts -- which v1.0 could not distinguish from symmetric co-occurrence.
GGUF quantizations are available at Hanno-Labs/bosun-xs-GGUF and Hanno-Labs/bosun-4b-GGUF for CPU and edge deployment. On a modern CPU, Bosun-XS can evaluate thousands of graph edges per minute, since each inference call is a single forward pass producing one scalar, not a generated sequence.
Both models are available on HuggingFace as LoRA adapters (adapter_model.safetensors and adapter_config.json) loadable via PEFT on top of the Qwen3-Reranker base. GGUF quantizations (f16, Q8_0, Q4_K_M) support llama.cpp, Ollama, and other local inference runtimes. The serving.json file in each repository contains the exact prompt template, yes and no token IDs, and max sequence length for fully reproducible inference. There is no official hosted API, no Bedrock or Vertex deployment, and no official SDK beyond the PEFT-based reference implementation.
As a specialized scoring model, Bosun does not produce natural language outputs and is not subject to the same jailbreak or prompt injection risks as chat models. The model outputs a calibrated float; its attack surface is narrow. Hanno Labs has not published a system card, red-teaming report, or safety benchmark for Bosun. The license places no restrictions on use case, so deployers are entirely responsible for ensuring the judgment rules they supply are appropriate for their application.
Bosun is a strong fit for teams building agent memory systems with knowledge graphs, where it acts as a fast, cheap pruning layer removing stale, redundant, or unsupported edges. RAG pipelines that need instruction-tuned filtering -- where the acceptance rule changes per query -- benefit from Bosun's runtime reprogrammability over static embedding similarity. Teams that should not use Bosun include those who need a generative chat model, those processing documents above 8,000 tokens per pair, those needing vision or audio, and those where ANLI-class adversarial NLI robustness is critical.
As a self-hosted model, data governance is entirely under the deployer's control. No data leaves the premises without the operator explicitly sending it. Hanno Labs has not disclosed SOC 2, ISO 27001, HIPAA, or GDPR certifications for Bosun; for regulated deployments, the user organization's own compliance posture applies. Training data for v1.1 included DialAM-2024 argument edges, NLI datasets, PAWS paraphrase pairs, e-CARE and COPA causal data, deduplicated hard-negatives, completeness examples, and synthetic directional data.
Bosun launched as v1.0 on June 11, 2026 with WarrantBench steerability of 0.935 for XS. Version 1.1, released approximately one week later, expanded the training blend and added directional typed-edge judgment: the model can now reason about asymmetric relationships that v1.0 could not distinguish. GGUF builds were added at the same time for local CPU deployment. The WarrantBench dataset is available at github.com/Hanno-Labs/warrantbench. Rapid iteration from v1.0 to v1.1 within two weeks of launch suggests active development continues.
Pricing
Free and open-source under Apache 2.0: no subscription, no per-token fee, no rate limit. The only expense is your own compute: about 0.5 GB RAM for Bosun-XS or 2.5 GB for Bosun-4B at Q4_K_M quantization, both CPU-capable. A GPU batch of 100,000 pairs through Bosun-4B on a leased A100 runs roughly $0.30.
Key Features
- Runtime-Programmable Rules: Supply any natural-language rule in the instruction block to reprogram judgment criteria per batch with no fine-tuning required.
- WarrantBench Steerability: Reliably reverses its judgment when the instruction is negated, the core property needed for trustworthy automated graph curation.
- PAWS Paraphrase Detection: 0.91: The benchmark most aligned with catching duplicate or redundant claims when pruning a knowledge graph.
- FollowIR Instruction-Following: First Place: Ranks first among evaluated retrievers on per-query instruction following, where most models ignore the instruction signal entirely.
- Directional Edge Judgment: Evaluates asymmetric relationships including supersession, dependency, support, and contradiction, not just symmetric co-occurrence.
- GGUF CPU Inference: Q4_K_M builds run on CPU with no GPU required, light enough for a laptop or a small VM.
Pros
- Places first on the FollowIR instruction-following leaderboard at +17.9 p-MRR, the property that matters most when the acceptance rule changes per query in a RAG pipeline.
- Strong paraphrase and duplicate-edge detection, exactly the skill needed to keep a knowledge graph from accumulating redundant claims.
- Runs on CPU via GGUF quantization, with no GPU, no API cost, and no rate limit once it is on hardware you control.
Cons
- Trails frontier general-purpose LLMs on adversarial NLI benchmarks, where lexical entailment matters more than explicit rule-matching.
- No official hosted API: every deployment is self-managed via PEFT or GGUF, which needs in-house ML infrastructure to run reliably.
- English-only training data; performance on non-English findings is untested and likely worse.
Benchmarks
- Anli: 0.57 vendor-reported · 25 Jun 2026
- Paws: 0.91 vendor-reported · 25 Jun 2026
- E Care: 0.85 vendor-reported · 25 Jun 2026
- Followir Pmrr: 17.9 vendor-reported · 25 Jun 2026
- Negation Accuracy: 0.97 vendor-reported · 25 Jun 2026
- Cross Domain Bridging: 0.83 vendor-reported · 25 Jun 2026
- Novel Rule Generalization: 0.95 vendor-reported · 25 Jun 2026
- Warrantbench Steerability: 0.95 vendor-reported · 25 Jun 2026
A benchmark is an exam, not the job. Scores transfer unevenly between tasks, so weigh the one closest to your workload and read every figure with its source.
Frequently Asked Questions
What does Bosun actually cost?
Bosun has no pricing plans in the traditional sense: it is a free, self-hosted model, not a subscription. The only ongoing cost is the compute you run it on, and that is modest: Bosun-XS fits in about half a gigabyte of memory, and Bosun-4B in roughly 2.5. A GPU batch of 100,000 pairs on a leased A100 costs roughly $0.30.
Is Bosun free to use?
Yes, both variants ship under an open license, free to run, change, and share commercially, with no royalty and no separate enterprise tier to unlock. Because inference runs entirely on your own hardware, there is no usage cap, no seat limit, and no data ever leaves your infrastructure unless you send it somewhere yourself.
What should you use instead of Bosun?
Gemini 3.1 Flash-Lite and other frontier LLM APIs like GPT-4o mini can act as ad-hoc graph judges, and unlike Bosun both can also generate fluent text if the pipeline needs that too. Teams already running a vector database sometimes fall back to plain cosine-similarity reranking instead of an instruction-following judge, trading Bosun's per-query rule flexibility for a simpler setup. Either approach costs real money per call; Bosun's tradeoff is your own infrastructure instead of a bill.
Bosun or Gemini 3.1 Flash-Lite: which should you pick?
Pick Bosun when the job is graph-edge judgment specifically: it holds up better on paraphrase detection and reliably reverses its verdict when a rule is negated, which Gemini does not do as consistently. Pick Gemini if your workload leans on adversarial NLI, where it has a real edge over Bosun-4B, or if the pipeline needs the model to also write fluent text. Gemini bills per token; Bosun has no per-token price once you have paid for the hardware to run it.
What does it take to start using Bosun?
Download the LoRA adapter weights for Bosun-XS or Bosun-4B from HuggingFace and load them via PEFT on top of the exact Qwen3-Reranker base model named in adapter_config.json, or skip PEFT and pull a GGUF build for llama.cpp or Ollama instead. Each repository ships a serving.json file with the prompt template and the yes/no token IDs your inference code needs to read the score correctly. From there, write your rule as a plain-language instruction, pass in the two findings, and read back the probability.
Top Alternatives
- Gemini 3.1 Flash-Lite: Pick Bosun for graph-edge judgment at zero API cost; pick Gemini 3.1 Flash-Lite ($0.25/$1.50 per million tokens) if the pipeline also needs to generate fluent text, not just a score.
- GPT-4o mini: Pick Bosun if you are running high-volume batch judgments and can self-host; pick GPT-4o mini ($0.15/$0.60 per million tokens) if you would rather pay per call than manage your own inference stack.
HokAI guides covering Bosun
- The Quirks Field: 373 Sentences No Vendor Would Publish: HokAI publishes 373 quirk entries across 102 of 103 model pages, each caveat checked today against the vendor's own model card, license, or terms page.