Performance
How current AI models perform on legal research tasks, shown as research on the field rather than as claims about Justinian. The figures are reproduced from the HAQQ Sovereign AI benchmark page.
Third-party benchmark results, shown as research on the field. They are not a claim about Justinian.
Reliability against quality per dollar
Each numbered marker is one model scored on the Legal Research Bench. Further right is a higher reliability proxy. Higher up is more quality for each dollar, as an index. The vertical axis stops at 20%.
- 1Gemini 4 Argon85.0% / 1.2%
- 2MiniMax-M383.9% / 12.1%
- 3Grok 4.771.0% / 1.3%
- 4Claude Sonnet 5.553.0% / 0.4%
- 5GPT-6 Astra49.0% / 0.5%
- 6GPT-6.1 Sol46.0% / 1.6%
- 7Claude Opus 5.541.0% / 0.3%
- 8Claude Fable 5.127.4% / 0.3%
Not plotted, no published reliability rate: MiMo V2.6 Flash, GLM 5.3 Flash, MiMo V2.6 Pro, DeepSeek V4.1 Flash, Muse Spark 1.3 Max
Quality on the Legal Research Bench
Quality score in percent, highest first.
- Muse Spark 1.3 Max55.3%
- Claude Fable 5.155.3%
- Gemini 4 Argon54.8%
- Claude Opus 5.550.5%
- Claude Sonnet 5.548.1%
- MiMo V2.6 Pro47.1%
- Grok 4.747.1%
- GLM 5.3 Flash45.2%
- DeepSeek V4.1 Flash41.4%
- GPT-6 Astra39.4%
- GPT-6.1 Sol38.5%
- MiMo V2.6 Flash38.0%
- MiniMax-M329.8%
Cost per task
Cost in US dollars to run one task, lowest first.
- MiMo V2.6 Flash$0.07
- GLM 5.3 Flash$0.12
- MiMo V2.6 Pro$0.18
- DeepSeek V4.1 Flash$0.25
- MiniMax-M3$0.34
- Muse Spark 1.3 Max$0.59
- GPT-6.1 Sol$3.38
- Grok 4.7$4.92
- Gemini 4 Argon$6.45
- GPT-6 Astra$10.58
- Claude Sonnet 5.5$14.83
- Claude Fable 5.1$23.06
- Claude Opus 5.5$25.16
View the data as a table
| Model | Quality | Cost per task | Quality per $ | Quality per $ index | Reliability proxy |
|---|---|---|---|---|---|
| MiMo V2.6 Flash | 38.0% | $0.07 | 542.6 | 74.6% | Not published |
| GLM 5.3 Flash | 45.2% | $0.12 | 376.6 | 51.8% | Not published |
| MiMo V2.6 Pro | 47.1% | $0.18 | 261.8 | 36.0% | Not published |
| DeepSeek V4.1 Flash | 41.4% | $0.25 | 165.4 | 22.7% | Not published |
| Muse Spark 1.3 Max | 55.3% | $0.59 | 93.7 | 12.9% | Not published |
| MiniMax-M3 | 29.8% | $0.34 | 87.7 | 12.1% | 83.9% |
| GPT-6.1 Sol | 38.5% | $3.38 | 11.4 | 1.6% | 46.0% |
| Grok 4.7 | 47.1% | $4.92 | 9.6 | 1.3% | 71.0% |
| Gemini 4 Argon | 54.8% | $6.45 | 8.5 | 1.2% | 85.0% |
| GPT-6 Astra | 39.4% | $10.58 | 3.7 | 0.5% | 49.0% |
| Claude Sonnet 5.5 | 48.1% | $14.83 | 3.2 | 0.4% | 53.0% |
| Claude Fable 5.1 | 55.3% | $23.06 | 2.4 | 0.3% | 27.4% |
| Claude Opus 5.5 | 50.5% | $25.16 | 2.0 | 0.3% | 41.0% |
Source: sovereign.haqq.ai/benchmark, captured October 16, 2026.
Quality and cost per task come from the Vals.ai Legal Research Bench. The reliability proxy is 100 minus a hallucination rate from Artificial Analysis, a general proxy rather than a measure of legal reliability. The quality per dollar index is a relative scale set on the source page, not a quality score.
Macro architecture
Justinian is a system built around frontier models, not a single model. This is how its parts fit together.
Justinian® understands your legal question, targets your exact jurisdiction, and crafts an answer backed by verified case law and line-by-line citations.
Product & Interfaces
One platform, every surface
- HAQQ Chat
- HAQQ eFirm
- Mobile App
- eBar
- Client Portal
- API / MCP
Security & Governance
- Cross-Matter Isolation
- Audit Trails
- RBAC
- Anonymization
JUSTINIAN AGENTIC HARNESS
The Ontology of Intelligence
Resolved before the model call
- Prompt Rewrite
- Intent Classification
- Language Detection
- Entity Extraction
- Document Intake + OCR
- Tool Routing
- Model Routing
- Guardrails
Then the agentic loop
- Sub-Agent Orchestration
- Memory
- Deep Reasoning
- Lookahead + Rollback
- Citation Audit
- Human-in-the-Loop
Architected to route across
- OpenAI
- Anthropic
Context & Knowledge
Your firm's institutional memory
- Playbooks
- Policies
- Memos
- Templates
- Precedents
- Clause Libraries
- Matter History
Data & Integrations
Live legal sources
- Web Search
- Statutes
- Case Law
- Gazettes
- Regulatory DBs
- USPTO
- EUIPO
- WIPO
Firm
Your legal operating system
- Matters
- Contracts
- Documents
- Hearings
- Tasks
- Calendar
- Financial
- KYC
- Leads
- Emails
Jurisdictions
Multi-region coverage
- MENA
- Europe
- North America
- South America
- Asia-Pacific
- Africa
Legal Capabilities
- Drafting
- Redlining
- Due Diligence
- Compliance Reports
- Litigation
- Analysis
- Case Summaries
Benchmark framework
How we build and run a legal benchmark: the task prompts, the scoring rubric, the public frameworks it draws on, how averages are computed, and where it stops.
Two evaluations, two scales
The task benchmark
11 legal task categories. On the work a lawyer actually hands over, who produces the better output?
The cross-domain study
25 benchmarks in 5 domains: legal, tax, journalism, general capability, and safety and values. Against the frontier models on their own ground, where does a legal-specialised engine stand?
The two scales are not interchangeable. A score out of 50 and a score out of 100 measure different work on different rubrics, and must never be read as comparable.
How a score is produced
- 1
Same prompt, same task
Each system is given the same prompt for a task a lawyer would recognise: draft this NDA, research this question, explain this provision to a client.
Task benchmark - 2
A fifty-point rubric
The output is scored against a fifty-point rubric. Partner-readiness and source linking are the two criteria that separate a legal evaluation from a general one.
Task benchmark - 3
Public frameworks
The cross-domain study draws on public frameworks rather than instruments we wrote ourselves, so the legal columns can be checked by someone outside HAQQ.
Cross-domain study - 4
All-pass grading
A task counts only when every criterion in its rubric is met. A near-miss scores zero, not partial credit, because output that is ninety per cent correct still has to be checked line by line.
Cross-domain study - 5
How averages work
A domain average is the mean of that domain's benchmarks. The overall average is the mean of twenty-four benchmarks, leaving out adversarial testing. A blank cell is excluded from every average and is never treated as a zero.
Cross-domain study - 6
Report and analysis
Rows where HAQQ is not first are shown exactly like rows where it is: same weight, same position, no reordering and no omission. Where a figure is derived, the derivation is stated.
Both
What the rubric covers
- Sharia
- Statute
- Forum
- Clause construction
- Risk
- Hallucination
- Formatting
- Brevity
- Partner-readiness
- Source linking
Partner-readiness
An answer that is correct but needs an hour of restructuring before a partner would sign it has not done the job.
Source linking
A citation that does not resolve is worse than no citation, because it transfers the checking cost to the reader without telling them.
Public frameworks the study draws on
Legal reasoning across a broad task taxonomy
All-pass rubric grading of long-horizon legal work
Independent industry evaluation of legal AI systems
Arabic legal reasoning, which most legal benchmarks do not cover
Tasks drawn from large-firm practice
Contract clause extraction and understanding
Citation accuracy and resolvability
Multilingual legal capability
Cross-vendor methodology
What this benchmark does not do
It is not independent
HAQQ builds it, runs it, and competes in it. That is why the methodology is published and the per-task results are visible rather than summarised.
It does not cover every practice area
The task benchmark is weighted toward corporate and commercial drafting, research and client-facing explanation. Other areas are covered thinly or not at all.
It does not cover every system
Several legal AI vendors publish no evaluations and offer no self-serve access. A system absent from a table was not beaten by us; it was not run.
It is a point in time
Every entrant ships new versions. A score is the version named at the run date and nothing more.
It measures output, not deployment
Security posture, data residency, support and integration depth decide whether a system can be used at all, and none of them appear in the benchmark.
Historical data
The legal benchmarks that have shaped how AI models are measured, from the first public datasets in 2021 to 2026.
Scores are not shown. Each benchmark uses its own tasks and metrics, so scores from different benchmarks cannot be compared. Every entry links to its primary source.
2021
CUAD
Contract review: highlighting the clauses in a contract that matter to a human reviewer, with over 13,000 expert annotations.
Primary source: arXiv 2103.06268
CaseHOLD
Multiple-choice questions that ask a model to identify the relevant holding of a cited case, with over 53,000 questions.
Primary source: arXiv 2104.08671
LexGLUE
A collection of datasets for evaluating models in a standardised way across a range of English legal language understanding tasks.
Primary source: arXiv 2110.00976
2023
LegalBench
Legal reasoning in large language models: 162 tasks covering six types of legal reasoning, built collaboratively with legal professionals.
Primary source: arXiv 2308.11462
LawBench
Legal knowledge in language models at three cognitive levels: memorising, understanding and applying legal knowledge, across 20 tasks.
Primary source: arXiv 2309.16289
2024
LegalBench-RAG
Evaluates the retrieval step of retrieval-augmented generation in the legal domain, with a focus on retrieving minimal, highly relevant text segments.
Primary source: arXiv 2408.10343
LexEval
A Chinese legal benchmark with 23 tasks and 14,150 questions, which also examines the ethical issues in applying language models to law.
Primary source: arXiv 2409.20288
2025
VLAIR, legal research
Legal research report from Vals AI: AI legal research products and ChatGPT scored on 200 US legal research questions against a lawyer baseline.
Primary source: vals.ai
ALARB
Arabic legal argument reasoning: over 13,000 commercial court cases from Saudi Arabia, with the facts, the court's reasoning, the verdict and the cited clauses.
Primary source: arXiv 2510.00694
2026
LegalCiteBench
Citation reliability in legal language models: closed-book citation recovery, citation verification and case matching.
Primary source: arXiv 2605.10186
Engine information
The technical facts about the Justinian engine, in one place.
- Type
- The proprietary Legal AI engine behind HAQQ Chat, eFirm and the mobile app, built for lawyers and for everyone who is not one.
- Powers
- Not a wrapper. One proprietary engine powers AI chat, practice management, eBar, payments and mobile. Orchestration, retrieval, memory and safety are ours, not a model vendor's.
- Frontier models
- A third-party frontier model still performs the final generation, and we would rather say so plainly than imply we trained our own. The orchestration, retrieval, jurisdiction logic, memory and guardrails around it are ours, and that is the layer carrying the legal value.
- Decisions before the model call
- Ten, resolved before a frontier model is ever called
- Model routing
- Architected to route across: OpenAI, Anthropic, Google
- Changing models
- Swap underlying models without re-engineering. Proprietary orchestration layer ensures consistent quality across any frontier LLM.
- Jurisdictions
- Jurisdiction-aware analysis grounded in local statutes, case law, and regulatory frameworks across 200+ legal systems.
- Grounding
- Proprietary legal ontology, clause-level analyzers, and constraint engines ensure every output is legally defensible.
- Language handling
- Yours, and separately the language the governing law is written in.
- Deployment
- Deploy in HAQQ's cloud, inside your own cloud tenancy, or fully on-premise.
- Data protection
- In HAQQ Legal AI Chat, personal details are swapped for made-up stand-ins before anything is sent to an external AI model provider. Encryption in transit and at rest. Compliant with GDPR, SOC 2 and ISO 27001, not certified.
- Human oversight
- Every output needs professional review before it reaches a client or a court.
- Parameters
- Not disclosed
- Context length
- Not disclosed
- Supported languages
- Not disclosed
- Knowledge cutoff
- Not disclosed
- Hardware requirements
- Not disclosed
Only facts already published on haqq.ai/justinian are listed. A field without a published fact reads Not disclosed.
Applications
Justinian powers HAQQ's applications. The Legal AI chat is available today, with more on the way.
Legal AI chat
Work through legal questions and documents with Justinian, in your browser or on your phone.
Coming soon
AI research behind Justinian
Open-ended research projects, kept like books on a shelf: libraries, skills, tools and studies you can open and use.
Our commitments
Your matters never train a model.
Client files, matters and conversations stay yours and are kept separate per firm.
Your data lives where you say.
HAQQ runs in the region you pick, and the platform is the same wherever it runs.
Every answer can be checked.
Research, review and drafting are tied to the sources behind them, so a lawyer can verify the work.
We would rather say we do not know.
When the engine cannot find support, it is designed to say so instead of producing a source.
The lawyer stays accountable.
HAQQ assists legal professionals, and lawyers remain responsible for professional judgment and the final work product.
We publish how we measure.
Our benchmark method states its limits, and a blank is never counted as zero.