feat: add GitHub Actions report artifact workflow

This commit is contained in:
ubuntu-master committed 2026-06-07 03:27:02 +08:00
1 parent 526b634da2
commit 06b231be5e
12 files changed
+590 -2

No files matched your search

+89
View File
@@ -0,0 +1,89 @@
name: Generate Security Report
on:
workflow_dispatch:
inputs:
max_items:
description: Maximum candidates per section
required: false
default: "20"
keyword_limit:
description: Maximum default keywords to query
required: false
default: "5"
keywords:
description: Optional comma-separated keyword override
required: false
default: ""
include_cve:
description: Include CVE candidate collection
required: false
type: boolean
default: true
include_repositories:
description: Include repository candidate collection
required: false
type: boolean
default: true
schedule:
- cron: "0 1 * * *"
permissions:
contents: read
jobs:
generate-report:
runs-on: ubuntu-latest
timeout-minutes: 30
env:
GITHUB_TOKEN: ${{ secrets.REPORT_GITHUB_TOKEN || github.token }}
GITHUB_TOKENS: ${{ secrets.REPORT_GITHUB_TOKENS || '' }}
REPORT_OUTPUT: artifacts/github-security-report.md
REPORT_MAX_ITEMS: ${{ github.event.inputs.max_items || '20' }}
REPORT_KEYWORD_LIMIT: ${{ github.event.inputs.keyword_limit || '5' }}
REPORT_KEYWORDS: ${{ github.event.inputs.keywords || '' }}
REPORT_INCLUDE_CVE: ${{ github.event.inputs.include_cve || 'true' }}
REPORT_INCLUDE_REPOSITORIES: ${{ github.event.inputs.include_repositories || 'true' }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Generate report
run: |
mkdir -p artifacts
args=(
--output "$REPORT_OUTPUT"
--max-items "$REPORT_MAX_ITEMS"
--keyword-limit "$REPORT_KEYWORD_LIMIT"
--database-path github-actions-report.db
)
if [ -n "$REPORT_KEYWORDS" ]; then
args+=(--keywords "$REPORT_KEYWORDS")
fi
if [ "$REPORT_INCLUDE_CVE" != "true" ]; then
args+=(--skip-cve)
fi
if [ "$REPORT_INCLUDE_REPOSITORIES" != "true" ]; then
args+=(--skip-repositories)
fi
python scripts/generate_report.py "${args[@]}"
- name: Upload report artifact
uses: actions/upload-artifact@v4
with:
name: cybersentinel-ai-report-${{ github.run_number }}
path: artifacts/*.md
if-no-files-found: error
retention-days: 14
+6
View File
@@ -31,6 +31,8 @@ logs/
results/
data/
reports/daily/
artifacts/
github-actions-report.db
# Build/package artifacts
build/
@@ -49,3 +51,7 @@ dist/
*.bak
*.tmp
*.log
# Local agent planning artifacts
.codex-tasks/
docs/superpowers/
+5 -2
View File
@@ -1,4 +1,4 @@
.PHONY: install test run clean
.PHONY: install test run report clean
install:
python3 -m venv .venv
@@ -10,5 +10,8 @@ test:
run:
python main.py
report:
python scripts/generate_report.py
clean:
rm -rf .pytest_cache __pycache__ */__pycache__ .coverage htmlcov build dist *.egg-info
rm -rf .pytest_cache __pycache__ */__pycache__ .coverage htmlcov build dist *.egg-info artifacts github-actions-report.db
+7
View File
@@ -13,6 +13,7 @@ CyberSentinel AI is an automated security monitoring system for tracking CVE-rel
- Stores monitor state in SQLite.
- Generates local Markdown and HTML reports.
- Optionally publishes reports to a blog API when `BLOG_TOKEN` is configured.
- Generates GitHub Actions Markdown report artifacts without committing runtime data.
## Quick start
@@ -110,6 +111,12 @@ main.py runtime entrypoint
config.py environment-backed configuration
```
## More documentation
- [GitHub Actions reports](docs/GITHUB_ACTIONS_REPORTS.md)
- [Configuration guide](docs/CONFIGURATION.md)
- [Deployment notes](docs/DEPLOYMENT.md)
## License
MIT. See [LICENSE](LICENSE).
+2
View File
@@ -11,6 +11,7 @@ CyberSentinel AI is an automated security monitoring and AI analysis system. It
- Store monitor state in SQLite.
- Generate Markdown and HTML reports.
- Publish reports to a blog API only when `BLOG_TOKEN` is configured.
- Generate GitHub Actions Markdown report artifacts without committing runtime data.
## Quick start
@@ -96,6 +97,7 @@ config.py environment-backed configuration
More documentation:
- [GitHub Actions reports](docs/GITHUB_ACTIONS_REPORTS.md)
- [Configuration guide](docs/CONFIGURATION.md)
- [Deployment notes](docs/DEPLOYMENT.md)
+2
View File
@@ -11,6 +11,7 @@ CyberSentinel AI 是一个自动化安全监控与 AI 分析系统,用于跟
- 使用 SQLite 保存监控状态和分析记录。
- 生成 Markdown/HTML 报告。
- 配置 `BLOG_TOKEN` 后可自动同步到博客;未配置时自动跳过博客发布。
- 支持 GitHub Actions 自动生成 Markdown 报告并作为 Artifact 保存,不写回仓库。
## 快速开始
@@ -101,6 +102,7 @@ config.py 基于环境变量的配置入口
更多说明:
- [GitHub Actions 自动报告](docs/GITHUB_ACTIONS_REPORTS.md)
- [中文快速开始](docs/QUICKSTART_ZH.md)
- [配置说明](docs/CONFIGURATION.md)
- [部署说明](docs/DEPLOYMENT.md)
+64
View File
@@ -0,0 +1,64 @@
# GitHub Actions Reports
CyberSentinel AI can generate a one-shot Markdown report in GitHub Actions and store it as a workflow artifact. The workflow does not commit generated reports back to the repository.
## Workflow
The workflow file is:
```text
.github/workflows/generate-report.yml
```
It supports:
- daily scheduled runs at `01:00 UTC`;
- manual runs through **Actions -> Generate Security Report -> Run workflow**;
- artifact upload with 14-day retention.
## Secrets
The workflow can run with the built-in `GITHUB_TOKEN`, but a personal token usually has better API limits for GitHub search.
Optional repository secrets:
| Secret | Purpose |
| --- | --- |
| `REPORT_GITHUB_TOKEN` | Single GitHub token used by the report workflow. |
| `REPORT_GITHUB_TOKENS` | JSON array of GitHub tokens, for example `["token1","token2"]`. |
Do not store tokens in workflow files or commit them to the repository.
## Manual inputs
| Input | Default | Description |
| --- | --- | --- |
| `max_items` | `20` | Maximum candidates per report section. |
| `keyword_limit` | `5` | Number of default keywords to query when no override is supplied. |
| `keywords` | empty | Optional comma-separated keyword override, for example `rce,poc,cloud security`. |
| `include_cve` | `true` | Include CVE candidate collection. |
| `include_repositories` | `true` | Include repository candidate collection. |
## Local dry run
```bash
python scripts/generate_report.py \
--output artifacts/github-security-report.md \
--max-items 20 \
--keyword-limit 5
```
The generated report is ignored by git:
```text
artifacts/
github-actions-report.db
```
## Artifact download
After the workflow finishes:
1. Open the workflow run in GitHub Actions.
2. Download the `cybersentinel-ai-report-<run-number>` artifact.
3. Open `github-security-report.md`.
+229
View File
@@ -0,0 +1,229 @@
from __future__ import annotations
from dataclasses import dataclass, field
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from clients.github_client import GithubClient
from config import GITHUB_KEYWORDS, WATCHED_REPOSITORIES, settings
from database.init_db import init_database
from database.session import SessionFactory, get_engine
from monitors.cve_monitor import CVEMonitor
from monitors.github_monitor import GithubMonitor
from utils.blacklist_manager import BlacklistManager
@dataclass
class ReportData:
generated_at: datetime
keywords: list[str] = field(default_factory=list)
cve_candidates: list[dict[str, Any]] = field(default_factory=list)
repository_candidates: list[dict[str, Any]] = field(default_factory=list)
errors: list[str] = field(default_factory=list)
def parse_keywords(raw_keywords: str | None, *, defaults: list[str], limit: int) -> list[str]:
"""Parse comma/newline separated keywords while keeping multi-word phrases."""
source = raw_keywords or ""
candidates = source.replace("\n", ",").split(",") if source.strip() else defaults
normalized: list[str] = []
seen: set[str] = set()
for item in candidates:
keyword = str(item).strip()
if not keyword:
continue
key = keyword.lower()
if key in seen:
continue
seen.add(key)
normalized.append(keyword)
return normalized[: max(limit, 0)]
def trim_items(items: list[dict[str, Any]], limit: int) -> list[dict[str, Any]]:
return list(items[: max(limit, 0)])
def _format_reasons(value: Any) -> str:
if isinstance(value, list):
reasons = [str(item) for item in value if str(item).strip()]
else:
reasons = [str(value)] if str(value).strip() else []
return ", ".join(reasons) if reasons else "-"
def _format_score(value: Any) -> str:
try:
return f"{float(value):.2f}"
except (TypeError, ValueError):
return "-"
def _format_stars(value: Any) -> str:
try:
return str(int(value))
except (TypeError, ValueError):
return "-"
def build_report_markdown(
*,
generated_at: datetime,
cve_candidates: list[dict[str, Any]],
repository_candidates: list[dict[str, Any]],
keywords: list[str],
errors: list[str] | None = None,
) -> str:
generated = generated_at.astimezone(timezone.utc).isoformat()
lines = [
"# CyberSentinel AI GitHub Actions Report",
"",
f"- Generated at: `{generated}`",
"- Mode: `artifact-only`",
"- This report is generated by GitHub Actions and is not committed back to the repository.",
"",
"## Search configuration",
"",
f"- Keywords: {', '.join(keywords) if keywords else '-'}",
f"- CVE candidates: {len(cve_candidates)}",
f"- Repository candidates: {len(repository_candidates)}",
"",
]
if errors:
lines.extend(["## Warnings", ""])
for error in errors:
lines.append(f"- {error}")
lines.append("")
lines.extend(
[
"## CVE candidates",
"",
"| CVE | Repository | Stars | Quality | Reasons |",
"| --- | --- | ---: | ---: | --- |",
]
)
if cve_candidates:
for item in cve_candidates:
cve_id = str(item.get("cve_id") or "Unknown")
repo_url = str(item.get("repository_url") or item.get("url") or "")
cve_cell = f"[{cve_id}]({repo_url})" if repo_url else cve_id
lines.append(
"| {cve} | {repo} | {stars} | {score} | {reasons} |".format(
cve=cve_cell,
repo=repo_url or "-",
stars=_format_stars(item.get("stars")),
score=_format_score(item.get("quality_score")),
reasons=_format_reasons(item.get("quality_reasons")),
)
)
else:
lines.append("| - | - | - | - | No CVE candidates collected |")
lines.extend(
[
"",
"## Repository candidates",
"",
"| Repository | Keyword | Stars | Quality | Reasons |",
"| --- | --- | ---: | ---: | --- |",
]
)
if repository_candidates:
for item in repository_candidates:
name = str(item.get("name") or item.get("repository_name") or "Unknown")
url = str(item.get("url") or item.get("repository_url") or "")
name_cell = f"[{name}]({url})" if url else name
lines.append(
"| {name} | {keyword} | {stars} | {score} | {reasons} |".format(
name=name_cell,
keyword=str(item.get("search_keyword") or "-"),
stars=_format_stars(item.get("stars")),
score=_format_score(item.get("quality_score")),
reasons=_format_reasons(item.get("quality_reasons")),
)
)
else:
lines.append("| - | - | - | - | No repository candidates collected |")
lines.extend(
[
"",
"## Next steps",
"",
"- Download this Markdown file from the workflow artifact.",
"- Review candidates manually before acting on them.",
"",
]
)
return "\n".join(lines)
def collect_report_data(
*,
max_items: int,
keyword_limit: int,
keywords: str | None,
include_cve: bool,
include_repositories: bool,
database_path: str | None,
) -> ReportData:
selected_keywords = parse_keywords(keywords, defaults=GITHUB_KEYWORDS, limit=keyword_limit)
data = ReportData(generated_at=datetime.now(timezone.utc), keywords=selected_keywords)
if not include_cve and not include_repositories:
return data
db_path = database_path or settings.database_path
init_database(db_path)
engine = get_engine(db_path)
session_factory = SessionFactory(engine)
github_client = GithubClient()
blacklist_manager = BlacklistManager()
if include_cve:
try:
cve_monitor = CVEMonitor(
engine,
github_client=github_client,
blacklist_manager=blacklist_manager,
session_factory=session_factory,
)
fetched = cve_monitor.fetch_latest_cves()
data.cve_candidates = trim_items(cve_monitor.process_cve_data(fetched), max_items)
except Exception as exc: # pragma: no cover - network/runtime dependent
data.errors.append(f"CVE collection failed: {exc}")
if include_repositories:
try:
github_monitor = GithubMonitor(
engine,
github_client=github_client,
blacklist_manager=blacklist_manager,
session_factory=session_factory,
keywords=selected_keywords,
watched_repositories=WATCHED_REPOSITORIES,
)
data.repository_candidates = trim_items(github_monitor.run(interval=1), max_items)
except Exception as exc: # pragma: no cover - network/runtime dependent
data.errors.append(f"Repository collection failed: {exc}")
return data
def write_report(output_path: str | Path, report_data: ReportData) -> Path:
path = Path(output_path)
path.parent.mkdir(parents=True, exist_ok=True)
markdown = build_report_markdown(
generated_at=report_data.generated_at,
cve_candidates=report_data.cve_candidates,
repository_candidates=report_data.repository_candidates,
keywords=report_data.keywords,
errors=report_data.errors,
)
path.write_text(markdown, encoding="utf-8")
return path
+1
View File
@@ -0,0 +1 @@
"""Helper scripts for CyberSentinel AI."""
+77
View File
@@ -0,0 +1,77 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
from pathlib import Path
import sys
PROJECT_ROOT = Path(__file__).resolve().parents[1]
if str(PROJECT_ROOT) not in sys.path:
sys.path.insert(0, str(PROJECT_ROOT))
from reports.action_report import collect_report_data, write_report # noqa: E402
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="generate-report",
description="Generate a one-shot CyberSentinel AI report for GitHub Actions artifacts.",
)
parser.add_argument(
"--output",
default="artifacts/github-security-report.md",
help="Markdown output path. Default: artifacts/github-security-report.md",
)
parser.add_argument(
"--max-items",
type=int,
default=20,
help="Maximum CVE and repository candidates to include per section. Default: 20",
)
parser.add_argument(
"--keyword-limit",
type=int,
default=5,
help="Maximum search keywords to use when REPORT_KEYWORDS is not supplied. Default: 5",
)
parser.add_argument(
"--keywords",
default=None,
help="Comma or newline separated keyword list. Defaults to project security keywords.",
)
parser.add_argument(
"--database-path",
default="github-actions-report.db",
help="SQLite database path for this one-shot run. Default: github-actions-report.db",
)
parser.add_argument(
"--skip-cve",
action="store_true",
help="Skip CVE candidate collection.",
)
parser.add_argument(
"--skip-repositories",
action="store_true",
help="Skip repository candidate collection.",
)
return parser
def main(argv: list[str] | None = None) -> int:
args = build_parser().parse_args(argv)
report_data = collect_report_data(
max_items=args.max_items,
keyword_limit=args.keyword_limit,
keywords=args.keywords,
include_cve=not args.skip_cve,
include_repositories=not args.skip_repositories,
database_path=args.database_path,
)
path = write_report(args.output, report_data)
print(f"Report written to {path}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+61
View File
@@ -0,0 +1,61 @@
from __future__ import annotations
from datetime import datetime, timezone
from reports.action_report import build_report_markdown, parse_keywords, trim_items
def test_parse_keywords_uses_user_values_with_limit():
keywords = parse_keywords("rce, poc\ncloud security", defaults=["fallback"], limit=2)
assert keywords == ["rce", "poc"]
def test_parse_keywords_falls_back_to_defaults():
keywords = parse_keywords("", defaults=["cve", "security tool", "rce"], limit=2)
assert keywords == ["cve", "security tool"]
def test_trim_items_handles_invalid_limit():
items = [{"name": "a"}, {"name": "b"}]
assert trim_items(items, 0) == []
assert trim_items(items, -1) == []
assert trim_items(items, 1) == [{"name": "a"}]
def test_build_report_markdown_redacts_operational_details():
markdown = build_report_markdown(
generated_at=datetime(2026, 6, 7, 1, 0, tzinfo=timezone.utc),
cve_candidates=[
{
"cve_id": "CVE-2026-1111",
"repository_url": "https://github.com/owner/cve-repo",
"stars": 42,
"quality_score": 0.8,
"quality_reasons": ["readme", "poc"],
}
],
repository_candidates=[
{
"name": "owner/sec-repo",
"url": "https://github.com/owner/sec-repo",
"stars": 100,
"quality_score": 0.7,
"quality_reasons": ["security_context"],
"search_keyword": "security tool",
}
],
keywords=["security tool"],
errors=["GitHub API rate limited"],
)
assert "# CyberSentinel AI GitHub Actions Report" in markdown
assert "2026-06-07T01:00:00+00:00" in markdown
assert "[CVE-2026-1111](https://github.com/owner/cve-repo)" in markdown
assert "[owner/sec-repo](https://github.com/owner/sec-repo)" in markdown
assert "security tool" in markdown
assert "GitHub API rate limited" in markdown
assert "github_pat_" not in markdown
assert "ghp_" not in markdown
+47
View File
@@ -0,0 +1,47 @@
from __future__ import annotations
from datetime import datetime, timezone
from reports.action_report import ReportData
from scripts import generate_report
def test_generate_report_script_writes_artifact(monkeypatch, tmp_path):
output = tmp_path / "artifact" / "report.md"
def fake_collect_report_data(*, max_items, keyword_limit, keywords, include_cve, include_repositories, database_path):
assert max_items == 3
assert keyword_limit == 2
assert keywords == "rce,poc"
assert include_cve is True
assert include_repositories is False
assert database_path == str(tmp_path / "state.db")
return ReportData(
generated_at=datetime(2026, 6, 7, 1, 0, tzinfo=timezone.utc),
keywords=["rce", "poc"],
cve_candidates=[{"cve_id": "CVE-2026-2222", "repository_url": "https://github.com/o/r"}],
repository_candidates=[],
errors=[],
)
monkeypatch.setattr(generate_report, "collect_report_data", fake_collect_report_data)
exit_code = generate_report.main(
[
"--output",
str(output),
"--max-items",
"3",
"--keyword-limit",
"2",
"--keywords",
"rce,poc",
"--skip-repositories",
"--database-path",
str(tmp_path / "state.db"),
]
)
assert exit_code == 0
assert output.exists()
assert "CVE-2026-2222" in output.read_text(encoding="utf-8")