Z.ai announced GLM-Legal on October 12, 2026, a legal-domain agentic model built on the GLM-5.3-Flash foundation. The headline number is 19.17% all-pass on Harvey LAB, the legal industry's open benchmark for real law firm workflows. That reads like a failing grade until you look at how the benchmark grades.
The base model scores 6.67%. The delta, a 12.5 percentage point lift, is the actual claim Z.ai is making, and an independent evaluator reproduced it. On Vals AI's Harvey LAB leaderboard, GLM-Legal ranks seventh out of 79 entries.
Harvey LAB is not a multiple choice test. It contains 1,250 tasks across 24 practice areas, and every task carries expert-written rubric criteria, more than 75,000 of them in total. A task passes only when every criterion passes. That is all-pass grading, and it is the reason scores compress toward single digits for most models.
Vals AI runs the held-out set with two LLM judges and internet disabled. Agents receive file tools plus docx, pptx and xlsx skills, so the model has to produce a usable deliverable, not just an answer. Under that harness, GLM-Legal passes 92.5% of individual criteria, in line with the 90% to 95% range of the leaders. The gap between 92.5% and 19.17% is the all-pass penalty: a single missed criterion fails the whole task.
The base architecture matters here. GLM-5.3-Flash is Z.ai's first natively multimodal GLM-5 model, with 320B total parameters and 18B active, mixing sparse and linear attention to cut long-context serving cost. Inputs can be text, images, video or files, and the output is text. A legal workflow that ingests a 40-document disclosure bundle depends on that long-context efficiency more than on raw parameter count.
The Launch Chart Problem and the Training Stack Behind It
Z.ai's launch chart shows GLM-Legal ahead of Kimi K3 at 12.92%, Claude Fable 5 at 11.25% and GPT-6 Astra at 5.42%. Those figures match Vals AI. What the chart leaves out is the six models ranked higher, led by Muse Spark 1.2 at 25.42%. Rank seven of 79 is a real result; a chart that stops at rank seven is a marketing choice.
On LegalBench, Vals AI scores GLM-Legal at 86.23%. Claude Fable 5 leads at 88.56% and Kimi K3 sits at 86.21%. Z.ai also claims parity with leading frontier systems on PLawBench, which evaluates Chinese legal consultation, case analysis and document drafting, reporting 77.3. The company published no numbers of its own for either benchmark, so the Vals AI runs are the load-bearing evidence.
The training recipe has three stages. Domain mid-training expands the legal corpus so the model absorbs statutory rules and interpretive frameworks inside dense domain context. Post-training then runs reinforcement learning across tens of thousands of simulated legal environments, where the model learns to plan analysis and generate deliverables under concrete constraints. The third stage converts production feedback into training signal: legal professionals grade outputs on whether every critical issue and edge case was addressed and whether the deliverable can be used without manual rework, and those judgments become loss signals for the next cycle.
The unusual part is the agent harness. Z.ai describes harness architecture and model training as co-evolving in both directions. Edge cases surfaced during training expose bottlenecks in the harness; harness fixes then expose the model to more realistic constraints. That loop is why the published samples look like work product rather than essays. In one evidence analysis run across 40 documents from a software project dispute, the model read a September 30 client report claiming the project was on track and 60% complete against a September 8 internal warning that the milestone could not honestly be met, with an internal snapshot recording roughly 41% completion. It connected those records to a CEO instruction to keep invoicing and cited sections 2 and 5 of the master services agreement, which provide that payment does not waive rights and that warranties survive payment, while noting that the remaining claim elements still needed testing.
Deployment is where the story cools. API access runs through a consultation application aimed at law firms, legal tech companies and in-house teams. No price is published. No GLM-Legal repository exists on Hugging Face. The base model is available now at $0.15 per 1M input tokens and $0.50 per 1M output tokens, and Vals AI benchmarked both models at $0.075 and $0.25. Z.ai says GLM-Legal is coming to AutoClaw, its desktop agent platform, but gave no date.
- Task coverage: 1,250 tasks spread across 24 practice areas
- Grading depth: 75,000+ expert-written rubric criteria, all-pass required
- Judge setup: two LLM judges, internet disabled, file tools plus docx, pptx and xlsx skills
- Independent runner: Vals AI holds the held-out set and the leaderboard
- Separate criteria pass rate from task pass rate. 92.5% versus 19.17% is an artifact of all-pass grading, not a contradiction.
- Check the leaderboard window. Ask where the six omitted higher-ranked models sit before quoting the chart.
- Confirm the deliverable path. A benchmark score is not a Word document your team can send to a client.
That last point is the whole game for legal AI. Buyers are not purchasing benchmark percentages; they are buying work product that survives partner review. GLM-Legal's 19.17% is the strongest all-pass figure an open-weight legal model has posted on Harvey LAB, and the harness co-training explains why its output arrives formatted. Whether firms get to run it depends on an application form, an unpublished price, and a desktop app that has not shipped the model yet.
Comments