How the database was built
Public audit reports, collected from each publisher, parsed into findings and classified with reproducible scripts. This page gives the measured accuracy of each step, coverage per publisher, the rights that apply and how to report an error.
Read the full methods, including LLM reference annotation, cross-model agreement and evaluation limits.
From report to finding
Choose the publishers
Established audit firms, audit collectives and contest platforms that publish their reports.
Collect the reports
Reports come from each publisher's GitHub repository or website, pinned to a permanent link where possible and fingerprinted with SHA-256. Collectors respect robots.txt and rate limits, and never pass a login or paywall.
Identify engagements
Revisions, translations and PDF or Markdown copies of one audit merge into a single engagement. The audit year is the stated completion date, or the report's issue date; publication dates form a separate series, and file or crawl dates never stand in for either. Engagements without an eligible date stay in an unknown bucket.
Parse the findings
Layout-specific parsers read each publisher's report formats and extract every formal finding: the publisher's id, title, severity, status and full text with page numbers. Severity and status labels map to common scales through checked-in tables, and the original labels are kept.
Classify the findings
A model classifier assigns one primary vulnerability class per finding from the title and the start of the text, following written decision rules, and abstains when the evidence is too thin.
Release
Release scripts write every file, this site and the checksums from the final data, record hashes of their inputs, and produce identical output when rerun.
Measured accuracy
Loading measurements…
Vulnerability taxonomy
Loading the taxonomy…
Coverage per publisher
Loading coverage…
Known limits
- The database holds audits that were published. Firms choose which private reviews to publish and some remove old reports, so older years are the least complete.
- Finding counts follow each publisher's reporting habits: some list every gas and style note, others only security issues.
- Severity scales differ between publishers; the mapping keeps the original label next to the normalised one.
- Fix status is what the report recorded at publication. It covers the reviewed code and can predate later fixes or deployment changes.
- Text extraction from PDFs can break code listings and tables; every finding links to its page in the original report.
- The latest year is year to date.
Rights and corrections
Original reports belong to their publishers. The database links to each one, pinned where possible and fingerprinted with SHA-256, so anyone can confirm they are reading the same file.
Derived metadata, such as severities, classes, dates and counts, is open under the data licence. Quoted finding text keeps its publisher's rights. Where a publisher's notice asks for it, the text is withheld; where a publisher's site terms do not allow automated copying, its reports are listed with links only and their text is not reproduced. The rights policy lists each case.
Found an error, or want a report removed or attributed differently? Open an issue with the finding or report link. Corrections land in the next release.