FinanceBenchmark
FinanceBenchmark / Professional work · Completes pricing, capital, and risk-control tasks verified by numerical answers and code execution.
What it measures and how
Task sets v2/v3 and environment versions stay separate, retaining three attempts per task. The site calls at least one success Tasks passed and also Pass@1; this is not mean single-attempt accuracy. Finance Index is a separate aggregate.
How this evidence is used
Under review: could add exact-answer and code-verified finance evidence. Metric definitions need checking; the official Sol JSON returned 404 in this check. Source listing only; no automatic collection or scoring.
Evaluation limits & attribution
One success in three attempts is not single-attempt accuracy. Intervals must match metrics, and rolling aliases need version checks. Public tasks and download buttons do not establish available raw results.
Data license: Official code is MIT and the site requires FinanceBenchmark citation. Terms for complete live-result exports are unconfirmed; code licensing does not replace data licensing.
Results are published by FinanceBenchmark. Original scores and the NexGateHub consensus index use different scales and cannot be added directly.