Files
cve/.github/workflows/sync_exploits.yml
T
0xMarcio 46b5acf052 Tighten the writers: timeouts, a shorter audit, fewer bytes in git
Every writer now has a timeout, so a hung download cannot hold the
repository-updates lock for six hours and cancel the runs queued behind
it. The link audit moves from one 12,000-and-12,000 batch a week, which
measured an hour under that lock, to a seventh of it every day.

The social card PNG is drawn at deploy time and no longer committed: it
changed about four times a day at 68 KB a copy. nuclei.json stops
mirroring EPSS scores; every indexed CVE already has one in epss.json,
the site never fell back to the copy, and the daily recomputation
rewrote 3,400 lines of it each night.

The metadata sync retries GitHub advisory calls on 403, 429 and 5xx
instead of failing on the first one, and refuses to publish a year
shard that an NVD feed would shrink by a tenth or delete outright.
2026-09-04 00:36:16 +02:00

82 lines
2.3 KiB
YAML

name: Sync Exploit Archives
on:
workflow_dispatch:
schedule:
- cron: '50 4 * * *'
permissions:
contents: write
actions: write
concurrency:
group: repository-updates
cancel-in-progress: false
jobs:
ScheduledRun:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.13'
- name: Pull ExploitDB, Metasploit and Vulhub into the index
run: python3 scripts/sync_exploits.py
- name: Verify the generated index
run: |
python3 -m unittest discover -s tests
python3 scripts/build_site.py
- name: Refresh published counts
run: |
python3 .github/hero.py
python3 .github/getTrending.py --counts-only
- name: Check for changes and commit
id: commit
run: |
git config user.name 'CVE-BOT'
git config user.email '[email protected]'
git add --all -- \
README.md \
docs/hero.svg \
docs/social-card.svg \
docs/stats.json \
docs/kev.svg \
index/cve_dates.json \
index/cve_record_state.json \
cves/
if git diff --cached --quiet; then
echo "No changes to commit"
exit 0
fi
git commit -m "Sync exploit archives $(date -u +'%Y-%m-%d') :robot:"
for attempt in 1 2 3; do
if git push origin HEAD:main; then
echo "pushed=true" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "Push rejected; replaying onto the current main (attempt $attempt)"
git fetch --unshallow origin main 2>/dev/null || git fetch origin main
git rebase FETCH_HEAD || { git rebase --abort; exit 1; }
done
exit 1
# Commits pushed with the workflow token raise no push event, so site.yml
# would otherwise wait for its daily schedule. Dispatching is exempt.
- name: Rebuild and deploy the site
if: steps.commit.outputs.pushed == 'true'
env:
GH_TOKEN: ${{ github.token }}
run: gh workflow run site.yml --ref main