Harvey Legal Agent Benchmark
Vals AI / Harvey / Professional work · Processes legal materials and delivers reviewable Word, Excel, and slide files.
What it measures and how
Fixes Harvey's private set, six file tools, and docx/xlsx/pptx skills, without internet. Uses the mean whole-task pass rate from two judges, not criterion-level rates. Fable 5's 11.25% with Opus 4.8 fallback cannot represent one model. The separate 10.42% without fallback requires common-policy configuration review.
How this evidence is used
Under review: complete legal-work evidence is a priority. Stable exports, configurations, and redisplay terms are pending. Source listing only; no automatic collection or scoring.
Evaluation limits & attribution
Fallbacks and judging conditions affect comparability. Roughly 90% criterion success is not whole-task completion. Multiple Vals specialists are not independent evaluators and cannot increase total weight by their count.
Data license: Official results are readable. Harvey's open tasks and code do not automatically license Vals private-test results. Redisplay terms remain unresolved.
Results are published by Vals AI / Harvey. Original scores and the NexGateHub consensus index use different scales and cannot be added directly.