The first full pass over all 60,039 reference URLs: 1,566 answer 404 or
410, which removes 2,663 links from 2,419 entries and 2,445 lines from
references.txt. Anything that timed out, refused a HEAD or answered 403
kept its link.
The weekly job takes it from here on a five-week cycle, so this only had to
be paid once.
A timeout kept the link either way, but the checker still followed a failed
HEAD with a GET to the same dead host, so every unreachable URL cost two
full timeouts. Return on the first network failure and drop the timeout to
eight seconds: a measured 1,500 references went from roughly three URLs a
second to fifteen.
That pass removed 98 links across 86 inventory lines, and the weekly budget
rises to 12,000 references, which sweeps the corpus every five weeks.
Thirty of thirty sampled returned 404: the service is gone, so all 942
links across 911 entries are permanently dead. 576 entries held nothing
else and drop out of the search index rather than offering a reader a
guaranteed miss.
The audit only ever asked GitHub about repositories, leaving 60,112
reference URLs unchecked since the corpus was inherited. Sampling put hard
404s at 4%, and one host accounts for a fifth of that on its own: CIS
retired the OVAL Repository, so all 942 links to it are permanently gone
and are removed in the next commit.
The rest is diffuse, so it needs sweeping rather than a rule. Check an
eighth of the URLs each week alongside the repositories, ordered
round-robin by host so a slice does not arrive at one server as a burst.
Only a 404 or 410 removes a link: a timeout, a refused HEAD or a 403 keeps
it, and update_cves already applies the same test to references it
discovers, so this holds inherited links to the standard new ones meet.
An advisory copied from a GitHub Security Advisory opens with "### Impact"
or "### Summary". The section parser treated any heading as a boundary, so
those descriptions ended themselves and reached the site empty. Only the
four headings an entry actually uses end a section now, and leading heading
markers are stripped when the text is rendered.
A trailing slash made exploit-db.com/exploits/41034 and .../41034/ two
different links, which put 53 duplicate rows in front of readers.
github.com also serves attachments and advisories, and 103 of those were
being parsed as owner/repo -- rendering "user-attachments / files" with an
empty star column instead of the PoC archive they point to.
CISA publishes a catalogue of vulnerabilities observed being exploited in
the wild, and 1,111 of its 1,685 entries are CVEs this index already holds
PoCs for. That is the one fact a reader most wants before opening an
exploit, and it was nowhere on the page.
The weekly audit stores the catalogue, the build trims it to the CVEs the
index carries, and a result row shows a KEV chip -- with the ransomware
flag where CISA records one, which covers 352 of them. 37 KB served.
A failed fetch keeps the stored catalogue rather than dropping the badge
off every entry. The chip reuses the header pill's treatment: the palette
still spends green on things you can click and amber on star counts only.
Switching the age column to the last commit left the job still sorting on
updated_at, which moves when a repository merely gains a star. The table
kept its "most recently touched" ordering while displaying commit dates,
so the front page and the README filled up with entries last worked on
122, 900 and 923 days ago.
Filter the search to repositories pushed in the last 90 days and re-sort
the pool by that same date. The API can only sort by updated_at, but a
push bumps it too, so the newest commits are always near the top of that
order and a pool of 50 per year is enough to find them.
The 2026 table now opens on a PoC pushed two days ago instead of four
months, and no row anywhere is older than the window.
A pass over 104,043 entries costs 25-125ms, which is fast enough to feel
instant once but not nine times in a row while a word is typed. Coalesce a
burst at 160ms, skip single-character queries that match most of the index
without narrowing anything, and total the PoC count during the scan rather
than walking the matches again to display it.
The corpus was assembled by a workflow that, in its own words, output every
search result without trying to filter them. Re-checking every link whose
repository does not name its CVE against the current classifier found
27,026 of 32,675 failing it: Shodan cheat sheets under Heartbleed, FTC
robotics team repositories under the jQuery prototype pollution CVE, Docker
images matched through their dependency manifests, and a repository called
"test".
16,613 links whose repository names the CVE were never examined, a
repository that could not be fetched kept all of its links, and one that
calls itself a PoC and names the CVE is kept even where the classifier
says otherwise — deleting a real PoC costs more than leaving a doubtful
one in place.
7,223 entries lighter, and repo_meta.json drops to the 26,755
repositories still referenced.
The PoC vocabulary was English-only, so a repository describing itself as
"漏洞利用" — an exploit — read as evidence that it was not one. Chinese is
the second language of this corpus by a wide margin, and the same gap
covered Spanish, Portuguese, French, Russian, Japanese and Korean.
NON_POC_RE also vetoed "prueba", which is the proof in "prueba de
concepto", inverting the signal for Spanish repositories entirely.
On the 600-repository discovery sample this admits two more genuine PoCs
and nothing else.
Sorting a CVE's links on stars alone handed the top of Log4Shell to
ThreatMapper, Viper, afrog and Awesome-OSINT-For-Everything: large tools
that mention the CVE somewhere. Rank repositories named after the CVE
first and only order those by stars, which puts CVE-2025-32463_chwoot
above a 2.5k-star general sudo toolkit.
A term no advisory text uses leaves every match tied at the weakest tier,
where newest-first is meaningless; searching log4shell led with text4shell
and then a CVE holding a single incidental link. Break those ties on how
many of the linked PoCs carry the term instead, which is only consulted
when nothing matched a description, so product searches are unchanged.
Also drop two controls that could not do anything: an expander on a
description already short enough to fit, and a footer offering to expand
a list of one.
Space Grotesk for prose, IBM Plex Mono for every identifier, and the flat
hairline-bordered surfaces from the handoff. Green marks what you can act
on and amber only ever means a star count; nothing else is coloured.
Three changes to how the same data reads. Descriptions clamp to three lines
behind an expander, so a kernel CVE no longer pushes the links people came
for below the fold. Raw URLs become owner / repo rows carrying stars and
last-commit age, ranked best first, five at a time. And an empty search box
now keeps the trending PoCs on screen with a TRENDING / RECENT toggle
rather than showing an empty page.
The search grammar, ranking and negation are unchanged.
The audit already asks GitHub about every linked repository, so the same
query can carry back a star count and a last-push date for free. Results go
to repo_meta.json, which lets a search result say which of two hundred
linked PoCs is worth opening.
2.1 MB for 45,008 repositories, refreshed weekly.
The trending table sorted and displayed updated_at, which also moves when a
repository merely gains a star, so a four-month-old exploit was claiming it
had been updated three hours ago. Record pushed_at and created_at and let
the page decide: last commit for the age it shows, repository age for the
trending rank.
The job now writes trending.json alongside the README table, so the site
reads the data instead of regex-parsing markdown the same job generated.
The audit found 8,404 of the 47,210 linked repositories gone from GitHub,
so 10,065 links across 5,495 entries pointed at 404s. 1,940 entries held
nothing but dead links and now carry no PoCs at all, which drops them from
the search index rather than offering a reader broken results.
The 4,733 entries whose advisory lists no affected version still rendered
an empty shields.io chip; they now read n/a like the product and
vulnerability badges beside them.
Ten modules totalling 2,206 lines, reachable from no workflow and no live
script, alongside 18 MB of github-all.txt and references-all.txt that
nothing reads and 400 KB of stale api/v1 output that shipped in every
Pages artifact. scripts/README.md documented this as the build process.
utils.py existed for one caller and duplicated functions build_site.py
already had, so it folds into it. Nothing imports a third-party package
any more, and requirements.txt goes with it.
Sampling the corpus put 17% of the repositories it links to behind a 404,
so readers were following links to repositories that no longer exist. The
previous audit script was 630 lines, never wired to anything, and kept its
state under a gitignored directory.
Replace it with a check that batches repository lookups through GraphQL
and removes a link only when GitHub answers NOT_FOUND for it, so a
throttle or a network error can never drop one. A quarter of the corpus
runs each week, which stays inside the token's hourly budget and sweeps
everything within the month.
Typing before the payload arrives searched an empty dataset and reported
no results, then the loader hid the table and left a blank page, so the
query had to be typed a second time. Show that data is loading and run
the pending query once it lands.
Nineteen links across sixteen entries, each pointing at a repository for
an advisory whose number merely starts with the entry's own — CVE-2019-1144
holding CVE-2019-11447-POC, CVE-2023-3883 holding the WinRAR CVE-2023-38831
generator. Every one was re-checked against the classifier and none of
them survives it.
A README caption in maniakh/CVE-2026-31431---Copy-Fail-PoC reads
"Copy Fail (CVE-2026-3143)", one digit short. That still parses as a
valid identifier, and the guard meant to catch it was satisfied by the
repo's generic exploit.py, so the PoC was filed against an unrelated
WordPress advisory. CVE-2026-8222 (Open5GS) collected a GiveWP lab the
same way. Drop an id from the same year that a longer mentioned id
extends, unless the repository name deliberately carries both.
Underscores are word characters, so the id regex matched neither half of
a name like CVE-2022-4174_CVE-2022-41742; read the separator-normalised
name too. Over 600 live candidates that recovers two real PoCs the gap
was hiding, Keycloak_CVE-2026-18963_PoC and CVE-2026-42945_NginxRift.
Unquoted words were scored independently, so "active directory" matched a
description holding "active session" and "working directory" as strongly
as a real AD advisory, and the newest-first tie-break then put the wrong
one on top. Reward the words appearing together.
Plain substring matching also meant "rce" hit "open source" and "sudo"
hit "pseudo"; require a term to start a word, while still allowing it to
match into one so "word" finds "wordpress". Weight matches near the start
of a description, in coarse buckets so a few characters never outweigh
recency: "log4j rce" now leads with Log4j2 rather than GeoTools.
The two scheduled jobs write disjoint paths but pin their checkout when
they are queued, so a commit landing in between throws away a completed
run. Rebase onto the current main and retry instead, deepening the clone
only on the rare occasion it is needed.
Scores were boosted by how many PoC links matched, which promoted old
CVEs with large link piles above recent findings; searching WordPress led
with 2015-2018. Rank on match quality alone and break ties by CVE year
and number, newest first.
Only the id and description are needed for every comparison, so PoC text
is derived on demand: preparing 108934 entries drops from 2.9s to 0.6s.
Dropping no-store lets the browser revalidate the 53 MB payload instead
of refetching it on every visit; the 5 KB trending file stays uncached.
Build result rows directly, since String.replace reads $& in the
replacement as a pattern, and escape link text like every other cell.
The search API was called anonymously on a 10 requests/minute limit while
the workflow already had a token; send it as an Authorization header,
never in the URL, and never print it. Dropping the token from the origin
remote also stops it being written into .git/config on the runner, since
actions/checkout already provides push credentials.
Rewrite on urllib so the job installs nothing, retry rate limits and 5xx,
and fail loudly instead of silently publishing a README with a year
missing. Repository text is flattened so a pipe cannot break the table.
Pages builds from the workflow artifact, so the committed CVE_list.json
was never served while adding a fresh 53 MB blob to history every day.
Generate it during the Pages build instead and drop the duplicate
generator; scripts/build_site.py is now the only thing that writes it.
The jinja2 templates re-rendered a trending table that logic.js replaces
on load, so the page shell goes back to being a plain static file and the
Pages build no longer installs anything. Shallow-clone the sync job too:
it never reads git history, and the three-day lookback already recovers
a skipped day.
A repository dedicated to one CVE that calls itself a PoC and ships code
was still vetoed when its description happened to contain a generic word
like writeup, patched or detector. Check the description alongside the
name for PoC intent; pure scanners that never claim to be a PoC stay out.
Measured on 600 live candidates: 15 repositories gained, none lost.
Fall back to n/a for missing versions, matching the product and
vulnerability badges instead of rendering an empty shields.io chip.
0xMarcio