HAQQ Labs
The HAQQ Legal Benchmark
Three evaluations, all run and published by HAQQ itself rather than an outside lab: a 50-point task benchmark over the work a lawyer hands over, a cross-domain study that places a legal-specialised engine against the frontier models on their own ground, and a head-to-head HAQQ ran between three frontier models (Justinian is not one of them). The methodology for each is public so you can check it yourself. Justinian leads 10 of the 10 measured practice areas below.
HAQQ's own benchmark
Benchmarked across major practice areas
Each of the 10 areas below is the mean of the measured task categories mapped to it, over every model the run scored rather than the shorter list the homepage shows (out of 50 per model).
Source: HAQQ's published 50-task legal AI benchmark, run August 2026
Overall coverage
50-point evaluation covering Sharia, statute, forum, clause, risk, hallucination, formatting, brevity, partner-readiness, and source linking
- 1HAQQ (Justinian)49/50
- 2Anthropic: Claude Fable 545/50
- 3Anthropic: Claude Opus 4.843/50
- 4Mike OS42/50
- 5DeepSeek: DeepSeek V4 Pro40/50
Corporate & Commercial
Contract drafting, commercial agreements, shareholder agreements, consultancy terms, and NDAs
- 1HAQQ (Justinian)47/50
- 2Anthropic: Claude Fable 543/50
- 3Anthropic: Claude Opus 4.841/50
- 4Harvey Tenet41/50
- 5Mike OS40/50
Disputes
Research under an adversarial posture, and the memoranda that carry an argument to a partner
- 1HAQQ (Justinian)48/50
- 2LexisNexis +AI42/50
- 3Anthropic: Claude Fable 541/50
- 4Thomson Reuters CoCounsel41/50
- 5Anthropic: Claude Opus 4.840/50
Regulatory & Public
Statute and forum reasoning against a published rulebook, with the sources cited
- 1HAQQ (Justinian)47/50
- 2Anthropic: Claude Fable 543/50
- 3Anthropic: Claude Opus 4.841/50
- 4LexisNexis +AI41/50
- 5Mike OS39/50
IP & Technology
Licence agreements and the clause work that sits under them
- 1HAQQ (Justinian)46/50
- 2Anthropic: Claude Opus 4.843/50
- 3Anthropic: Claude Fable 542/50
- 4Spellbook41/50
- 5Legora40/50
Employment & Mobility
Employment agreements, drafted to the jurisdiction rather than to a template
- 1HAQQ (Justinian)48/50
- 2Anthropic: Claude Fable 543/50
- 3Harvey Tenet42/50
- 4Mike OS41/50
- 5Legora40/50
Real Estate & Projects
Long-form transactional drafting: commercial terms, grants of use, and schedules
- 1HAQQ (Justinian)45/50
- 2Anthropic: Claude Opus 4.843/50
- 3Anthropic: Claude Fable 542/50
- 4Harvey Tenet41/50
- 5Mike OS39/50
Private Client
Explaining the law in plain language, to someone who is not a lawyer
- 1HAQQ (Justinian)46/50
- 2OpenAI: GPT-5.6 Sol Pro45/50
- 3Anthropic: Claude Fable 543/50
- 4Google: Gemini 3.1 Pro41/50
- 5Anthropic: Claude Opus 4.839/50
Industry-Specific
Sector agreements where the terms of art come from the industry, not the statute
- 1HAQQ (Justinian)44/50
- 2Anthropic: Claude Fable 542/50
- 3Anthropic: Claude Opus 4.841/50
- 4Legora41/50
- 5Mike OS39/50
MENA-Native Law
Sharia, regional statute and forum, answered in Arabic and cited to a primary source
- 1HAQQ (Justinian)49/50
- 2Anthropic: Claude Fable 545/50
- 3Anthropic: Claude Opus 4.843/50
- 4Mike OS42/50
- 5DeepSeek: DeepSeek V4 Pro40/50
Cross-domain, out of 100
Twenty-five benchmarks across legal, tax, journalism, general capability, and safety and values, drawing on Stanford LegalBench, Harvey LAB, VLAIR, ALARB, BigLaw Bench, CUAD, LegalCiteBench and LawBench. The full table sorts by any column and filters by domain. HAQQ's own run, not independently audited, completed August 2026.
Justinian® is the best or joint-best score in 23 of 25. 6 of those are ties, marked =, and 20 are won by less than a point, so on most rows this is parity with the frontier rather than a lead. The 2 it does not lead (Harvey LAB, Long Context) are shown unchanged rather than dropped.
| Benchmark | HAQQ (Justinian) | Anthropic: Claude Opus 4.8 | Thomson Reuters CoCounsel | Google: Gemini 3.1 Pro | Z.ai: GLM 5.2 | OpenAI: GPT-5.6 Sol Pro | Kimi: K3 | Anthropic: Claude Sonnet 5 | Snowdon 1.0-Large | Qwen: Qwen3.7 Plus | DeepSeek: DeepSeek V4 Pro |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Overall average | 83.2 | 79.5 | 78.5 | 78.0 | 77.2 | 76.5 | 75.7 | 75.7 | 73.6 | 73.0 | 71.9 |
| Legal | |||||||||||
| Stanford LegalBench | 85.7 | 81.8 | 82.3 | 84.3 | 82.9 | 82.3 | 83.3 | 81.4 | 82.8 | 78.8 | 76.8 |
| Info. Retrieval | 57.2 | 51.2 | 53.6 | 53.8 | 53.6 | 53.8 | 53.0 | 47.9 | 51.2 | 49.0 | 51.5 |
| Reasoning | 78.2 | 75.2 | 73.2 | 77.8 | 73.1 | 68.4 | 74.8 | 73.1 | 70.8 | 66.5 | 70.1 |
| Classification | 74.7 | 70.5 | 70.9 | 74.2 | 70.2 | 70.0 | 71.6 | 70.4 | 69.8 | 68.7 | 67.8 |
| Doc. Processing & RAG | 83.2 | 78.9 | 79.7 | 78.2 | 76.5 | 78.6 | 79.0 | 74.7 | 71.2 | 74.1 | 73.7 |
| Summarisation | 90.0= | 88.3 | 90.0= | 89.4 | 90.0= | 86.3 | 87.7 | 84.8 | 87.9 | 82.5 | 89.1 |
| Contract Under. | 77.4 | 74.4 | 71.0 | 75.9 | 73.6 | 74.9 | 77.2 | 69.6 | 71.7 | 68.4 | 68.3 |
| Human Queries | 89.7 | 85.4 | 89.2 | 86.4 | 87.3 | 88.6 | 61.4 | 84.3 | 87.1 | 86.7 | 86.4 |
| Deep Research | 90.9 | 90.8 | 88.9 | 80.6 | 89.0 | 85.9 | 89.8 | 86.0 | 78.4 | 87.3 | 84.0 |
| Harvey LAB | 86.8 | 86.9 | 85.7 | 55.5 | 84.6 | 76.1 | 83.7 | 80.9 | 56.3 | 70.6 | 83.1 |
| Legal average | 81.4 | 78.3 | 78.4 | 75.6 | 78.1 | 76.5 | 76.1 | 75.3 | 72.7 | 73.3 | 75.1 |
| Tax | |||||||||||
| Deep Research | 86.3 | 85.8 | 82.4 | 76.0 | 83.5 | 80.9 | 85.1 | 82.0 | 70.4 | 78.7 | 80.7 |
| Tax Q&A | 88.9 | 86.9 | 87.9 | 84.5 | 85.7 | 86.6 | 88.3 | 88.7 | 88.2 | 85.1 | 82.8 |
| Tax average | 87.6 | 86.3 | 85.1 | 80.2 | 84.6 | 83.8 | 86.7 | 85.4 | 79.3 | 81.9 | 81.8 |
| Journalism | |||||||||||
| Deep Research | 84.6 | 82.3 | 80.9 | 83.0 | 79.0 | 66.3 | 84.5 | 74.1 | 76.2 | 77.2 | 78.6 |
| Journalism average | 84.6 | 82.3 | 80.9 | 83.0 | 79.0 | 66.3 | 84.5 | 74.1 | 76.2 | 77.2 | 78.6 |
| General | |||||||||||
| Factuality | 82.9 | 71.4 | 73.7 | 82.6 | 66.2 | 68.3 | 69.5 | 59.8 | 71.7 | 76.2 | 69.1 |
| Long Context | 75.6 | 75.2 | 75.3 | 75.0 | 75.9 | 70.0 | 73.5 | 70.7 | 74.6 | 53.0 | 69.5 |
| Multilingualism | 85.8 | 83.2 | 78.4 | 85.7 | 81.7 | 82.7 | 84.9 | 79.5 | 74.2 | 77.6 | 78.4 |
| Instr. Following | 91.7 | 86.1 | 91.4 | 84.8 | 89.4 | 89.0 | 85.8 | 86.8 | 87.3 | 85.6 | 89.2 |
| Writing | 80.7= | 79.3 | 80.3 | 78.5 | 78.1 | 79.1 | 78.2 | 78.7 | 79.9 | 77.9 | 80.7= |
| Reasoning | 76.2= | 73.7 | 68.4 | 74.8 | 67.3 | 71.6 | 76.2= | 66.8 | 66.8 | 66.6 | 65.0 |
| General Agent | 89.2 | 83.4 | 89.1 | 84.4 | 77.5 | 75.6 | 61.0 | 74.1 | 87.1 | 85.5 | 72.0 |
| Coding | 66.8= | 57.4 | 39.9 | 50.0 | 56.0 | 50.8 | 66.8= | 57.4 | 40.9 | 43.9 | 45.6 |
| Maths | 98.7= | 98.7= | 94.0 | 97.4 | 91.2 | 97.7 | 96.2 | 85.9 | 95.5 | 94.9 | 94.9 |
| General average | 83.1 | 78.7 | 76.7 | 79.2 | 75.9 | 76.1 | 76.9 | 73.3 | 75.3 | 73.5 | 73.8 |
| Safety / Values | |||||||||||
| Political Neutrality | 97.9 | 82.8 | 97.3 | 93.3 | 91.0 | 83.8 | 33.5 | 82.3 | 85.3 | 51.5 | 31.3 |
| Robustness | 78.5 | 78.2 | 60.3 | 65.3 | 50.4 | 68.3 | 72.7 | 77.1 | 42.2 | 66.7 | 38.1 |
| Adversarial Testing | 95.9= | - | 93.4 | - | 93.6 | - | 87.3 | - | 78.8 | 95.9= | 81.1 |
| Safety / Values average | 90.8 | - | 83.6 | - | 78.3 | - | 64.5 | - | 68.7 | 71.4 | 50.2 |
An empty cell means the model was not run on that benchmark: it is not a zero, and it is left out of that model's averages. Sort and filter it on /justinian.
Head-to-head: GPT-6 Astra, Claude Opus 5, GPT-5.6 Luna Pro
A benchmark HAQQ ran and is not a competitor in: Justinian is not one of the three models below, deliberately. 41 prompts across 20 practice areas, 9 measured properties, $66.55 of metered inference, reasoning effort pinned to medium for every model. Run 2026-09-10, tracked as GRO-1641.
Composite is out of 35 and the per-practice-area figure below is out of 20. Neither is the 50-point or 100-point scale above, and none of the three should be compared directly. The OpenAI model in this study, GPT-5.6 Luna Pro, is not the OpenAI model in the /50 run above (GPT-5.6 Sol Pro): a different model, a different study.
Quality leaderboard, composite out of 35
| Model | Prompts | Quality | Accuracy | Style | Insight | Speed | Composite |
|---|---|---|---|---|---|---|---|
| GPT-6 Astra | 41/41 | 9 | 8.634 | 4.951 | 4.463 | 3 | 30.05 |
| Claude Opus 5 | 37/41 | 9.108 | 8.108 | 4.946 | 4.973 | 1 | 28.13 |
| GPT-5.6 Luna Pro | 41/41 | 8.732 | 8.22 | 4.951 | 4.024 | 5 | 30.93 |
Cost per answer
A price list is quoted per million tokens; nobody buys a token, they buy an answer. Cost here is what OpenRouter actually billed for a median answer.
| Model | Price per M out | Tokens/answer | Cost/answer | vs. cheapest |
|---|---|---|---|---|
| GPT-6 Astra | $50 | 5,201 | $0.2622 | 11.4x |
| Claude Opus 5 | $25 | 18,190 | $0.4566 | 19.8x |
| GPT-5.6 Luna Pro | $1.2 | 16,356 | $0.0231 | 1x |
Arabic and French: the sharpest finding
Six questions across five jurisdictions, asked in the language a local practitioner would use. Both OpenAI models cited the correct provision and then misstated what it says: a citation checker would pass that answer; a client would not.
| Model | Language fidelity | Legally correct | Right provision cited |
|---|---|---|---|
| GPT-6 Astra | 100% | 66.7% | 100% |
| Claude Opus 5 | 100% | 100% | 100% |
| GPT-5.6 Luna Pro | 100% | 66.7% | 100% |
| Jurisdiction · language | GPT-6 Astra | Claude Opus 5 | GPT-5.6 Luna Pro |
|---|---|---|---|
| SA · AR | wrong Incorrectly asserts 30-day notice for a monthly-paid worker, contradicting Art. 75. | correct | wrong Incorrectly adds an employer/employee distinction not present in Art. 75. |
| AE · AR | correct | correct | correct |
| FR · FR | correct | correct | correct |
| FR · FR | correct | correct | correct |
| EG · EN | wrong Downplays the consent requirement; ground truth requires both consent and a licence. | correct | wrong Omits/downplays the consent requirement; discusses only the licence. |
| QA · AR | correct | correct | correct |
Long context
A planted four-clause conflict in a Master Services Agreement, tested at 25,000 / 100,000 / 200,000 tokens. All three models resolved it correctly at every depth. Cost of that identical answer at 200k: GPT-6 Astra $2.01, GPT-5.6 Luna Pro $0.15.
Refusal calibration
Twelve requests (six a lawyer must refuse, six a lawyer must answer). All three models: 100% of the harmful requests handled and 100% of the legitimate ones actually answered.
Reasoning effort (GPT-6 Astra)
The only model here with a user-facing effort dial. Same 10 prompts at each setting.
Determinism
Same three questions, five repeats each, a fixed seed where a model accepts one. No model reproduced an identical answer across five runs.
By practice area, quality + accuracy out of 20
| Practice area | GPT-6 Astra | Claude Opus 5 | GPT-5.6 Luna Pro |
|---|---|---|---|
| AI/Tech Regulation | 17 | 17 | 16.5 |
| Arbitration | 17.5 | 17 | 17 |
| Banking/Finance | 17 | 16.5 | 16.5 |
| Bankruptcy & Insolvency | 18.5 | 19 | 16.5 |
| Construction | 18 | 18 | 16.5 |
| Contract & Commercial | 18 | 17.333 | 17.667 |
| Corporate Governance | 18 | 18 | 16.5 |
| Criminal/White Collar | 17.5 | 17.5 | 16 |
| Data Privacy | 18 | 17 | 18 |
| Employment Law | 18 | 17.5 | 18 |
| Environmental/ESG | 17 | 17.5 | 16 |
| IP/Tech Law | 17.5 | 17 | 17.5 |
| Immigration | 17.5 | 17.5 | 17.5 |
| Insurance | 17.5 | 17 | 16 |
| International Trade | 18 | 17 | 16.5 |
| M&A | 18 | 18 | 17.5 |
| Real Estate | 17.5 | 15.5 | 17.5 |
| Regulatory Compliance | 17 | 16.5 | 16.5 |
| Securities | 17.5 | 17.5 | 17 |
| Tax | 17.5 | 17 | 17.5 |
Bold marks the leader in each row (ties bold more than one). This is HAQQ's own run, dated 2026-09-10 and tracked as GRO-1641. HAQQ is not a model in it, and the citation-integrity suite from this run is not reported here pending a harder trap set (see the source module's header for why).