The daily sweep found new repositories within 2 to 9 hours where their
name carried the CVE, and up to 30 hours where only the README did. The
hourly run now searches repositories pushed in the last day for this
year's and last year's CVEs, reads their READMEs and applies the same
classifier, so a PoC published this morning is on the site within the
hour. The daily run keeps covering every year and the CVE record
references; the hourly run skips those, and skipping leaves the delta
checkpoint alone.
A sample of two hundred indexed repositories held six with nothing in
them: an empty README or none, a placeholder, a link to a paywall. A
repository with no directory, no code, no file named for the CVE or as a
PoC, and a README with no code block, command, request or real write-up
is bare, and neither discovery nor the daily audit keeps it. The audit
now reads each repository's README and root to judge that, and sweeps
4,000 repositories a day instead of 1,750, since one link in eight
pointed at a deleted repository under the fortnightly cycle.
The upstream PoC-in-GitHub collection is judged daily rather than
weekly: fourteen of the last thirty-nine additions came from it, which
means it finds repositories our own search does not.
Twenty-eight year folders sat above the README, so anyone opening the
repository scrolled past 1999 through 2026 before reaching a word about what
it is. The entries move to cves/ and the files the pipeline reads and writes
move to index/, which leaves the root at README, LICENSE and five
directories.
Nothing about the data changes. Every script takes its two roots from the
same place, and all five syncs report zero writes against the moved tree,
which is what a correct repoint looks like.
KEV reached the site three days ago but nothing in the markdown said a
vulnerability was under active attack, and the repository is what most
people actually read. 1,595 entries now carry a red CISA KEV shield beside
their product and vulnerability badges, marked for ransomware use where
CISA records it.
update_cves writes the badge on entries it creates, and the weekly audit
reconciles the whole set against the refreshed catalogue, so an entry CISA
adds gains the badge and one it withdraws loses it. Re-running the sync is
a no-op.
Every colour is now a Primer dark_default token. The header band, its four
capability pills and the heading above the trending table are gone, so the
search field is the first thing on the page and the switcher labels its own
table. Rows tighten to 10px, descriptions clamp to two lines, and the PoC
list loses its box for a hairline rule.
Blue is the only thing you can click. Amber means 500 stars or more and red
means known-exploited, and nothing else carries a hue: the star glyph is
dimmed at every value so the column reads as numbers rather than symbols.
The handoff's placeholder wanted an em dash. It reads "Search CVE, vendor,
product or keyword, exclude with -windows" with the middot the count line
already uses, and the six em dashes I had left in code comments are gone
too.
The first full pass over all 60,039 reference URLs: 1,566 answer 404 or
410, which removes 2,663 links from 2,419 entries and 2,445 lines from
references.txt. Anything that timed out, refused a HEAD or answered 403
kept its link.
The weekly job takes it from here on a five-week cycle, so this only had to
be paid once.
A timeout kept the link either way, but the checker still followed a failed
HEAD with a GET to the same dead host, so every unreachable URL cost two
full timeouts. Return on the first network failure and drop the timeout to
eight seconds: a measured 1,500 references went from roughly three URLs a
second to fifteen.
That pass removed 98 links across 86 inventory lines, and the weekly budget
rises to 12,000 references, which sweeps the corpus every five weeks.
The audit only ever asked GitHub about repositories, leaving 60,112
reference URLs unchecked since the corpus was inherited. Sampling put hard
404s at 4%, and one host accounts for a fifth of that on its own: CIS
retired the OVAL Repository, so all 942 links to it are permanently gone
and are removed in the next commit.
The rest is diffuse, so it needs sweeping rather than a rule. Check an
eighth of the URLs each week alongside the repositories, ordered
round-robin by host so a slice does not arrive at one server as a burst.
Only a 404 or 410 removes a link: a timeout, a refused HEAD or a 403 keeps
it, and update_cves already applies the same test to references it
discovers, so this holds inherited links to the standard new ones meet.
CISA publishes a catalogue of vulnerabilities observed being exploited in
the wild, and 1,111 of its 1,685 entries are CVEs this index already holds
PoCs for. That is the one fact a reader most wants before opening an
exploit, and it was nowhere on the page.
The weekly audit stores the catalogue, the build trims it to the CVEs the
index carries, and a result row shows a KEV chip -- with the ransomware
flag where CISA records one, which covers 352 of them. 37 KB served.
A failed fetch keeps the stored catalogue rather than dropping the badge
off every entry. The chip reuses the header pill's treatment: the palette
still spends green on things you can click and amber on star counts only.
The audit already asks GitHub about every linked repository, so the same
query can carry back a star count and a last-push date for free. Results go
to repo_meta.json, which lets a search result say which of two hundred
linked PoCs is worth opening.
2.1 MB for 45,008 repositories, refreshed weekly.
Sampling the corpus put 17% of the repositories it links to behind a 404,
so readers were following links to repositories that no longer exist. The
previous audit script was 630 lines, never wired to anything, and kept its
state under a gitignored directory.
Replace it with a check that batches repository lookups through GraphQL
and removes a link only when GitHub answers NOT_FOUND for it, so a
throttle or a network error can never drop one. A quarter of the corpus
runs each week, which stays inside the token's hourly budget and sweeps
everything within the month.
0xMarcio