Commit Graph
20 Commits
Author SHA1 Message Date
0xMarcio 6cbe907084 Track meaningful page changes for search crawlers 2026-10-06 22:06:54 +02:00
0xMarcio 02e8fadcd6 Improve crawlable SEO and search reliability 2026-10-06 21:51:21 +02:00
0xMarcio 89fbf68e21 Fix workflow queues and index consistency 2026-10-06 21:09:05 +02:00
0xMarcio aae0c789a3 Meet the query where it is written: slang, names, typos
Security slang and NVD prose rarely share a word. "sqli" found 843 CVEs
where "sql injection" found 12,719, "lpe" 80 against 1,311 for "privilege
escalation". A term now expands to its phrases and an entry matches on any
of them, with the best-scoring form counted.

A vulnerability searched by the name it is known by leads with the record
it belongs to, whatever the prose and the repository names say: ghostcat
put a different record first, moveit a newer one. Forty names are mapped.

A term with no word in the index at all is nearly always a typo. The
closest word one edit away stands in, weighed by how many records use it,
and the status line says which: citrx finds citrix, fortinte fortinet,
l0g4j log4j.

CVE pages carry BreadcrumbList markup for the home, year and block trail,
and the site publishes an OpenSearch descriptor so a browser can add it
as a search engine from the address bar.
2026-09-04 02:59:24 +02:00
0xMarcio 46b5acf052 Tighten the writers: timeouts, a shorter audit, fewer bytes in git
Every writer now has a timeout, so a hung download cannot hold the
repository-updates lock for six hours and cancel the runs queued behind
it. The link audit moves from one 12,000-and-12,000 batch a week, which
measured an hour under that lock, to a seventh of it every day.

The social card PNG is drawn at deploy time and no longer committed: it
changed about four times a day at 68 KB a copy. nuclei.json stops
mirroring EPSS scores; every indexed CVE already has one in epss.json,
the site never fell back to the copy, and the daily recomputation
rewrote 3,400 lines of it each night.

The metadata sync retries GitHub advisory calls on 403, 429 and 5xx
instead of failing on the first one, and refuses to publish a year
shard that an NVD feed would shrink by a tenth or delete outright.
2026-09-04 00:36:16 +02:00
0xMarcio 0e5e8d7e8c Fix correctness, security and consistency bugs across the generated site
The per-CVE pages read as a different site and disagreed with search about
their own contents. Both traced to the page generator having grown its own
copy of rules the browser already had.

Security:
- JSON-LD was serialised with json.dumps alone, which leaves < > and & raw. A
  description containing "</script>" ended the element and turned the rest into
  live markup. 78 descriptions in the corpus already carry that sequence; only
  the 155-character meta trim kept them out of the payload. Escape to \u00XX.

Correctness:
- The PoC count summed only the repository column, so 19,940 pages announced
  "0 public PoC exploits" directly above the ExploitDB and Metasploit entries
  they were listing, in the title and meta description too. Count every source
  the page renders, matching POC_FIELDS in logic.js.
- The severity chip took the highest score while search took the first, so
  18,434 CVEs showed one severity in the results and another on the page. Both
  now take the primary assessment, with the same Nuclei fallback search uses.
- Identical CVSS rows printed two and three times on 13,340 pages. Dedupe on
  (version, score, vector) and label each survivor with its assessor from
  row[4]; row[5] is the assessment type and left 129 pages showing two rows
  both marked "Secondary".
- repo_from_url accepted any host containing "github" and kept .git suffixes,
  so a clone URL and its plain form counted as two sources on the page and one
  in search. 273 CVEs disagreed. Match repoFromUrl in logic.js exactly.
- 41 CVEs nobody has scored were unreachable by search, including by exact ID:
  the severity filter tested membership of a list naming only the four scored
  bands, so ALL excluded them. ALL now means all.
- EPSS percentiles printed a hard-coded "th": 51th, 62th, 93th on 21,100 pages.
- Search reported "1 PoCs", and the empty-results branch returned before
  updating the live region, leaving it reading "loading index..." forever.

Deploy safety:
- A failed EPSS fetch returned {} and published it, silently erasing every
  score from 82,000 pages. It now raises, as does a feed covering under 90%
  of the index.

Presentation:
- CVE pages never loaded the webfonts the homepage loads, so all 82,468 fell
  back to system fonts. This is why they looked unlike the rest of the site.
- 404.html referenced four CSS classes that no longer existed and rendered
  unstyled, and its meta refresh threw a reader out of a search after two
  seconds. It is now generated from the same constants and header as every
  other page, and offers the search box instead.
- The homepage and the generated footer credited different sources. Both now
  come from brand.SOURCES_LINE, and the build refuses to publish a homepage
  that has drifted from it.
- Three aria-labels sat on div elements with no role, where they are ignored.

SEO:
- 61,936 pages filled "Related" with numerically adjacent CVEs, which are not
  related to anything. Keep only shared-author relations; 20,532 pages have one.
- Sitemap lastmod described the CVE record, so a CVE that gained forty PoCs
  still advertised a date from 2021. It now reflects the page, and each shard
  carries its newest page's date rather than today's.
- TechArticle is not a type Google draws a rich result from, and its dates
  claimed the CVE record's publication as the page's. Use WebPage with the
  vulnerability as mainEntity.
- Curated sources and the CVE and NVD records no longer carry nofollow; the
  crowd-sourced repository list still does.
2026-09-02 17:17:28 +02:00
0xMarcio 9b9957ea7f Give every CVE its own page at the site root
82,000 CVEs sat behind one URL. Each of them is a query somebody types, and the
pages answering those queries belonged to other people: searching
"CVE-2026-72898 poc" returns SentinelOne, Horizon3, Wiz and BishopFox, all of
them serving one page per CVE.

Pages are flat files at /CVE-2026-72898. No /cve/ prefix, no extension, no
trailing slash. GitHub Pages resolves an extensionless request to the .html
beside it, checked against the live site rather than assumed.

Each page carries what the index already knows and nothing invented: every CVSS
assessment with its vector, EPSS with percentile, KEV date and ransomware flag,
Nuclei severity and CWE, and the PoC links grouped by source with stars and last
push. 94% of them carry three or more of those signals.

Discovery works two ways. sitemap.xml is now an index over one shard per year,
which fits the 50,000 URL cap on the split the data already has and lets a
crawler refetch only the year that moved. Search results link to the CVE page
instead of straight out to NVD, which moves to a chip beside CVE.ORG, and each
page links six related CVEs picked by shared PoC author.

Rendering all of them takes 20s and adds nothing to the repository: the payloads
they read are already gitignored and built in CI, so the pages go into the Pages
artifact and git never sees them.
2026-09-02 04:00:28 +02:00
0xMarcio 31da22381c Rebrand to PoC Index and move to pocindex.io
The site called itself three different things: CVE PoC Hub in the title, CVE
Radar in the h1 and the social metadata, and cve on GitHub. Google had no
consistent site name to settle on, so none of the mentions compounded. CVE
Radar was taken anyway, by SOCRadar's free tool and by cveradar.com.

Every user-facing name and URL now comes from scripts/brand.py, so the next
move is one edit rather than a sweep through the tree.

Also repairs the parts of the site a crawler could not use:

- robots.txt and sitemap.xml were both 404. scripts/build_seo.py writes them,
  wired into the Pages job. trending_poc.json, kev.json and stats.json stay
  crawlable because the first paint needs them; the five bulk payloads are
  disallowed, which is 13.2 MB per crawl spent on data no crawler queries.
- Search state lives in ?q= now. Results were unlinkable, so nobody could cite
  one and there was no route for a SearchAction to declare. replaceState
  rather than pushState, so typing does not bury the back button.
- A WebSite JSON-LD block carries the name Google prints as the site name.
- The footer carries the MITRE trademark notice.
2026-09-02 03:36:22 +02:00
0xMarcio 81017c9871 Expand CVE PoC coverage and metadata 2026-09-01 19:01:07 +02:00
0xMarcio 84da5a5fc6 Index ExploitDB and Metasploit, score everything with EPSS, and stop hiding new PoCs
Four gaps, measured against the index before any of this was written.

ExploitDB and Metasploit are folded in the way nuclei templates were. ExploitDB
covers 25,066 CVEs and adds 30,327 links across 17,237 entries; Metasploit
covers 3,178 and adds 3,520 across 2,550. Together with the templates the index
goes from 101,809 CVEs carrying a PoC to 103,945. Neither archive skews recent,
which is why nuclei went first, but both are the difference between an old CVE
having nothing and having something that runs. Each keeps its own section and
its own field, and each leads the list ahead of the repositories: a module, an
entry and a template all run as they stand, without reading somebody's code.

EPSS now covers the index rather than the 4,274 CVEs a template happened to
rate. FIRST's daily feed scores 103,783 of 103,945, so a result can say how
likely a flaw is to be exploited in the next thirty days, not only that code
exists for it. It is fetched during the site build and never tracked here: a
daily rescore of 366,000 rows would be nothing but churn.

The trending tables also stop hiding what they exist to show. The star floor
excluded 1,059 of the 1,312 2026 PoC repositories pushed this quarter, and a
repository published this morning has no stars, so the newest row on the front
page was six days old. A separate lane drops the floor entirely and pays for it
with a ten day window, a cap of ten rows, and the same paperwork check the rest
of the table uses. The newest row is now hours old.
2026-09-01 05:24:39 +02:00
0xMarcio 35cb1e6962 Index nuclei templates as PoCs and take their ratings with them
A nuclei template is a runnable check for one CVE, which makes it a proof of
concept whether or not anyone published a repository for it. Measured against
the index: 3,952 CVEs gain a template link and 858 of those had no PoC at all,
584 of the additions falling in 2024 or later. That last figure is the point.
ExploitDB carries more CVEs overall and Metasploit is better weaponised, but
both skew historical; nuclei is where coverage of a recent CVE actually shows
up first.

Templates also carry the ratings the entries never had. Severity, CVSS with
its vector, EPSS with its percentile and CWE now sit in nuclei.json for the
site to read: 4,193 of 4,418 templates state a CVSS score and 4,274 an EPSS
score. A result can finally say how bad a flaw is and how likely it is to be
used, rather than only that somebody wrote code for it.

Templates are kept in their own section and their own field. They are not
somebody's repository and should not be ranked, starred or credited as one,
but they do lead the PoC list, being the one link that runs as it stands.

The index files are also revalidated rather than reused from cache. They are
rebuilt hourly and a returning visitor was holding whatever it downloaded
first, which cost a stale index here until the copy was thrown away by hand.
2026-09-01 05:11:31 +02:00
0xMarcio a6be0d9794 Ignore assistant tooling
None of it is part of the project and a stray config committed once tends to
stay. The repository already had no such files tracked; this keeps it that way.
2026-09-01 04:51:07 +02:00
0xMarcio 1aa51fa852 Mark the CVEs attackers are actually using
CISA publishes a catalogue of vulnerabilities observed being exploited in
the wild, and 1,111 of its 1,685 entries are CVEs this index already holds
PoCs for. That is the one fact a reader most wants before opening an
exploit, and it was nowhere on the page.

The weekly audit stores the catalogue, the build trims it to the CVEs the
index carries, and a result row shows a KEV chip -- with the ransomware
flag where CISA records one, which covers 352 of them. 37 KB served.

A failed fetch keeps the stored catalogue rather than dropping the badge
off every entry. The chip reuses the header pill's treatment: the palette
still spends green on things you can click and amber on star counts only.
2026-08-31 00:13:01 +02:00
0xMarcio 066ddf4400 Rebuild the frontend to the refit design
Space Grotesk for prose, IBM Plex Mono for every identifier, and the flat
hairline-bordered surfaces from the handoff. Green marks what you can act
on and amber only ever means a star count; nothing else is coloured.

Three changes to how the same data reads. Descriptions clamp to three lines
behind an expander, so a kernel CVE no longer pushes the links people came
for below the fold. Raw URLs become owner / repo rows carrying stars and
last-commit age, ranked best first, five at a time. And an empty search box
now keeps the trending PoCs on screen with a TRENDING / RECENT toggle
rather than showing an empty page.

The search grammar, ranking and negation are unchanged.
2026-08-30 21:53:31 +02:00
0xMarcio f9fc5ae28d Stop tracking generated site data
Pages builds from the workflow artifact, so the committed CVE_list.json
was never served while adding a fresh 53 MB blob to history every day.
Generate it during the Pages build instead and drop the duplicate
generator; scripts/build_site.py is now the only thing that writes it.

The jinja2 templates re-rendered a trending table that logic.js replaces
on load, so the page shell goes back to being a plain static file and the
Pages build no longer installs anything. Shallow-clone the sync job too:
it never reads git history, and the three-day lookback already recovers
a skipped day.
2026-08-30 16:59:31 +02:00
0xMarcio c8fbbc40b2 Clean and verify CVE PoC sources 2026-08-29 04:29:03 +02:00
0xMarcio a62cda8ec8 Automate CVE PoC ingestion 2026-08-29 00:21:06 +02:00
0xMarcio 1ae3d7e711 Fix search layout and add dynamic CVE pages 2025-12-17 17:38:30 +01:00
0xMarcio aac323de9b Add KEV/EPSS static site generator 2025-12-17 13:59:41 +01:00
0xMarcio 0c3e517733 Automate CVE sync and daily PoC refresh 2025-09-19 00:48:01 +02:00