BullshitBench v2
Peter Gostev / BullshitBench / Knowledge · Tests whether models identify false premises rather than continue fabricating an answer.
What it measures and how
Uses the default share of all attempts explicitly identifying false premises. Mean scores over answered responses and refusal rates are retained for audit. Neither raw metric receives current overall or category weight.
How this evidence is used
Reference results: collected automatically after aggregates, tasks, configurations, and run dates are checked against one public version. Anonymous models are hidden. Rolling endpoints lacking fixed-version evidence are not representative scores. Excluded from overall and category scoring.
Evaluation limits & attribution
All questions have false premises, so results do not measure knowledge breadth. Model judges and refusal handling affect scores. Rolling aliases and anonymous models need identity verification first.
Data license: MIT · Copyright (c) 2026 Peter Gostev. Official results ship with the repository, without a separate data-license exception.
Results are published by Peter Gostev / BullshitBench. Original scores and the NexGateHub consensus index use different scales and cannot be added directly.