admin/rea
1
0 Fork 0
mirror of https://github.com/morluto/rea.git synced 2026-10-29 12:00:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor!: simplify runtime tools and remove replay engines (#572)

Browse Source 
* fix(hopper): report unavailable procedure body coverage

* refactor!: simplify runtime tools and remove replay engines

Keep direct inspection, capture, and comparison tools. Bind handlers to named contracts and let agents compose experiments with commands and local fixture servers.

BREAKING CHANGE: Remove controlled replay, Node characterization, finite replay-machine, managed runtime planning, and redundant string-tracing tools. Retire process replay, shims, reactive scenarios, custom checkpoints, and active browser replay and origin-scope inputs.
This commit is contained in:
morluto authored and GitHub committed 2026-10-05 21:15:04 +08:00
1 parent c6d3bdf597
commit e07b81649e
258 files changed
+1193 -22444
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/real-javascript-replay.yml
-40
View File
@@ -1,40 +0,0 @@
name: Real JavaScript replay verification
on:
pull_request:
paths:
- ".github/workflows/real-javascript-replay.yml"
- "scripts/verify-real-javascript-replay.mjs"
- "src/replay/**"
- "src/application/JavaScriptReplay*"
- "src/domain/javascriptReplay.ts"
- "tests/fixtures/replay/**"
- "package.json"
- "package-lock.json"
workflow_dispatch:
concurrency:
group: real-replay-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
HUSKY: "0"
permissions:
contents: read
jobs:
real-replay:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: .nvmrc
cache: npm
- run: npm ci
- run: sudo apt-get update && sudo apt-get install -y bubblewrap
- run: npm run verify:replay
env:
REA_REPLAY_ALLOW_UNAVAILABLE: "true"
AGENTS.md
+7 -3
View File
@@ -20,7 +20,7 @@ REA is a local-only tool. Preserve caller-selected inputs, captured output, URLs
REA is a layered ESM TypeScript application. Dependencies flow inward from pure domain logic through contracts, providers, application workflows, and CLI/MCP adapters. See [docs/architecture.mermaid](docs/architecture.mermaid) for the component map.
- `src/domain/` owns pure provider-neutral semantics; `src/contracts/` owns caller-visible schemas and the canonical tool inventory.
- `src/hopper/`, `src/ghidra/`, `src/browser/`, `src/replay/`, `src/native/`, `src/artifacts/`, and `src/dotnet/` own provider-specific boundaries. Keep provider protocols out of domain and application code.
- `src/hopper/`, `src/ghidra/`, `src/browser/`, `src/native/`, `src/artifacts/`, and `src/dotnet/` own provider-specific boundaries. Keep provider protocols out of domain and application code.
- `src/application/` composes shared CLI/MCP workflows; `src/server/` translates MCP requests; `src/cli.ts` and `src/main.ts` are the CLI and MCP entry points.
- `src/process/` owns shared process lifecycle primitives, not provider wire protocols. `bridge/` contains provider-side adapters.
- `tests/` contains unit, composition, boundary, acceptance, and conformance tests. `scripts/verify-*` contains real-toolchain checks.
@@ -58,10 +58,14 @@ Leave meaningful target, action, capture, and output choices to the agent. A sel
Verify the representation consumed at the next boundary. Internal validator success does not establish JSON Schema validity or client compatibility: validate advertised input and output schemas against their declared dialect after SDK conversion, and keep generated contracts aligned. Use representative producer data and regressions that exercise the failed behavior. Platform and provider support claims require the corresponding real workflow; package startup, capability probes, and mock transport tests establish only their narrower claims. Report unverified coverage explicitly.
Bind tool handlers to named contracts, preserving their exact input and output types. Catalog ordering is presentation metadata; changing that order must not select a different schema or operation for an existing handler.
## Designing MCP Tools
Start from the analyst question and desired result, not a provider API. Before adding a tool, inspect the existing contract and its nearest alternative.
Let agents compose experiments with ordinary commands, scripts, and local fixture servers. REA tools should perform the requested inspection or capture and return evidence. A custom orchestration language, replay engine, or separate prepare/execute plan needs an observed requirement that those primitives cannot satisfy. A plan-only tool without an executor does not establish runtime behavior; expose the useful inspection directly.
- Prefer reusable, composable primitives: inspect one explicit object or
relationship and return evidence that can support different analyst
questions. Compose primitives into a workflow only for a recurring outcome
@@ -69,7 +73,7 @@ Start from the analyst question and desired result, not a provider API. Before a
business-domain interpretations out of general tool contracts.
- Use **inspect/search** tools for facts about a target or candidate set; use **trace** tools for relationships; use **compare** tools for explicitly paired inputs.
- Add a **workflow** when observed agent use shows a repeated sequence that REA can compose without losing analyst control. Return useful results inline so callers can choose their next action.
- Use **observe/capture/replay** only when runtime activity is required, and declare authority and lifecycle effects in the contract.
- Use **observe/capture** only when runtime activity is required, and declare authority and lifecycle effects in the contract.
- Extend an existing tool when intent and result contract are unchanged. Add a tool for a distinct analyst outcome or materially different authority.
- Keep caller-facing names and results provider-neutral. Put engine-specific behavior in provider adapters and report each provider's exact coverage.
- Keep results complete by default. Add a limit only when it follows from a real format, protocol, authority, or resource-safety constraint; explain truncation and unsupported facets. Keep observed, derived, inferred, and unknown results distinct.
@@ -80,7 +84,7 @@ See [docs/tool-design.md](docs/tool-design.md) for the design checklist. When us
## Testing Guidelines
Name tests `*.test.ts`. Use Vitest and production seams (`tests/fixtures/`) rather than module mocks. Domain tests assert pure behavior; adapter tests use fake launcher/socket seams; MCP tests connect with the client SDK version pinned in `package.json`. Preserve the canonical tool inventory defined by `TOOL_CONTRACTS` and verified through `CATALOG_IDENTITY` and generated product metadata. Cover malformed input, cancellation, lifecycle cleanup, and actual format, protocol, host-permission, and target-identity boundaries. Do not add tests that merely freeze arbitrary caps or prescribed call sequences. Real Hopper, Ghidra, browser, JavaScript replay, managed conformance, and any real managed-tool claims cannot be replaced by mocks; use the corresponding `verify:*` command.
Name tests `*.test.ts`. Use Vitest and production seams (`tests/fixtures/`) rather than module mocks. Domain tests assert pure behavior; adapter tests use fake launcher/socket seams; MCP tests connect with the client SDK version pinned in `package.json`. Preserve the canonical tool inventory defined by `TOOL_CONTRACTS` and verified through `CATALOG_IDENTITY` and generated product metadata. Cover malformed input, cancellation, lifecycle cleanup, and actual format, protocol, host-permission, and target-identity boundaries. Do not add tests that merely freeze arbitrary caps or prescribed call sequences. Real Hopper, Ghidra, browser, managed conformance, and any real managed-tool claims cannot be replaced by mocks; use the corresponding `verify:*` command.
Keep tool catalogs complete and self-describing; prefer capability- and session-scoped availability over schema truncation. Serialized bytes alone do not measure agent usability or model context cost.
README.md
+21 -23
View File
@@ -261,10 +261,10 @@ rea analyze /absolute/path/to/app --snapshot /absolute/path/to/analysis/app.json
rea analyze /absolute/path/to/app --snapshot /absolute/path/to/analysis/app.json
```
Exact CLI evidence replays happen before any provider process starts. In MCP sessions,
Exact CLI cached-evidence reads happen before any provider process starts. In MCP sessions,
pass `snapshot_path` to `open_binary` to import a snapshot atomically while
opening its matching target; MCP providers may still start before a cached call
is replayed. Pass `snapshot_path` and, when required, `overwrite: true` to
result is returned. Pass `snapshot_path` and, when required, `overwrite: true` to
`close_binary` to save atomically before Hopper resources are released. If the
save fails, REA deliberately leaves the session open.
@@ -306,17 +306,17 @@ See [native investigation](docs/native-investigation.md) for keyed archives, ins
| Tool family | Count | Examples |
| ------------------------- | ----: | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Native inspection | 39 | functions, pseudocode, assembly, strings, symbols, calls, references, annotations, byte reads, and file offsets |
| Investigation workflows | 15 | app overviews, function dossiers, native APIs and dispatch, batch decompilation, feature traces, call paths, call graphs, Swift and Objective-C discovery |
| Investigation workflows | 14 | app overviews, function dossiers, native APIs and dispatch, batch decompilation, feature traces, call paths, call graphs, Swift and Objective-C discovery |
| Native macOS utilities | 7 | Mach-O metadata, code signatures, plists, architectures, and Swift demangling without launching Hopper |
| Artifact graph | 5 | directory and package inventories, compiled Interface Builder files, Apple asset catalogs, and extraction |
| Managed PE/CLI | 8 | .NET identity, metadata, CIL instructions, native dependencies, reconstruction imports, runtime-correlation plans, and build comparisons |
| Browser observation | 9 | page structure, network metadata, scripts, source maps, WebMCP discovery, screenshots, capture comparisons, and browser scenarios |
| Electron analysis | 5 | renderer observation, static app mapping, static/runtime reconciliation, and click/wait scenarios |
| Managed PE/CLI | 7 | .NET identity, metadata, CIL instructions, native dependencies, reconstruction imports, and build comparisons |
| Browser observation | 9 | page structure, network metadata, scripts, source maps, WebMCP discovery, screenshots, and capture comparisons |
| Electron analysis | 5 | renderer observation, static app mapping, and static/runtime reconciliation |
| JavaScript runtime | 2 | Node/Electron Inspector target discovery, script locations, and execution-context events |
| Application workflows | 10 | cross-layer feature traces, build comparisons, historical source mapping, return-shape comparison, Linux replay, and reconstruction checks |
| Workspace and observation | 22 | sessions, evidence bundles, navigation context, finite replay machines, process/artifact/function comparisons, and open-question tracking |
| Application workflows | 7 | cross-layer feature traces, build comparisons, historical source mapping, static return-shape comparison, and reconstruction checks |
| Workspace and observation | 21 | sessions, evidence bundles, navigation context, process/artifact/function comparisons, and open-question tracking |
The public interface describes what the agent is trying to learn. Providers decide how to answer. macOS utilities handle common semantic inspection without launching Hopper; Hopper handles deeper native analysis; the process harness implements controlled behavioral capture.
The public interface describes what the agent is trying to learn. Providers decide how to answer. macOS utilities handle common semantic inspection without launching Hopper; Hopper handles deeper native analysis; the process harness records direct behavioral captures.
## Current status
@@ -327,9 +327,8 @@ REA supports native application, JavaScript, Electron, .NET, and browser investi
- **JavaScript and Electron:** Map modules, imports, source maps, routes, IPC channels, storage, and native add-ons without running the app. Compare builds and trace a feature across the recovered graph. Dynamic and ambiguous relationships remain unresolved. See [JavaScript application workflows](docs/javascript-application-workflows.md).
- **Websites:** Inspect a selected page in an existing Chrome-family browser. Capture page structure, network metadata, script evidence, and screenshots requested by the call. Passive observation does not navigate or execute page JavaScript. See [browser observation](docs/browser-observation.md).
- **Electron and Node runtime observation:** Inspect selected Electron pages or attach to a Node/Electron V8 Inspector target. Inspector observation records script locations and execution-context events; it does not infer imports, IPC activity, or which modules executed. See [runtime observation](docs/javascript-runtime-observation.md).
- **.NET assemblies:** Inspect metadata and CIL instructions, compare builds, and check declared native dependencies without loading or running the assembly. Imported decompiler output is labeled as analyst inference. Runtime-correlation plans are supported, but an executor is not implemented. See [managed-code analysis](docs/managed-code-analysis.md).
- **.NET assemblies:** Inspect metadata and CIL instructions, compare builds, and check declared native dependencies without loading or running the assembly. Imported decompiler output is labeled as analyst inference. See [managed-code analysis](docs/managed-code-analysis.md).
- **Controlled behavior capture:** Run process, browser, or Electron scenarios with the target, actions, and lifecycle declared in each request, then compare the resulting evidence. Missing observations cannot establish that two runs behaved the same way.
- **JavaScript replay:** Run selected extracted modules in the Linux sandbox. Each execution request is bound to the exact plan digest; replay also requires the isolation controls described in [controlled JavaScript replay](docs/controlled-javascript-replay.md).
- **Evidence and comparison:** Save results with artifact identity, provider, locations, confidence, and limitations. Export or import bundles, compare artifacts and functions, and connect static findings to runtime observations without claiming causality from correlation.
- **Open questions:** Track unresolved findings, contradictions, and follow-up probes. Reconstruction checks report pass, fail, or unknown rather than treating missing evidence as a pass.
- **Guided workflows:** Start six [MCP investigation workflows](docs/mcp-prompts.md) with suggestions based on your current session.
@@ -396,10 +395,9 @@ The [current status](#current-status) section describes shipped capabilities. Th
### Next
1. **Test replay isolation:** add hostile fixtures and test the Linux replay sandbox across more kernels.
2. **Connect more application layers:** add static extractors and runtime observations to feature traces.
3. **Extend .NET analysis:** improve comparisons of obfuscated assemblies and connect managed findings to verified native analysis. See the [managed-code guide](docs/managed-code-analysis.md).
4. **Compare more runtime behavior:** expand process, protocol, filesystem, reconnect, and version-comparison coverage.
1. **Connect more application layers:** add static extractors and runtime observations to feature traces.
2. **Extend .NET analysis:** improve comparisons of obfuscated assemblies and connect managed findings to verified native analysis. See the [managed-code guide](docs/managed-code-analysis.md).
3. **Compare more runtime behavior:** expand process, protocol, filesystem, reconnect, and version-comparison coverage.
### Later
@@ -537,7 +535,7 @@ The session also reports active work and cleanup status. If a caller times out,
`setup` returns `1` for `planned`, `needs_confirmation`, or `needs_human`
because configuration is not ready; rerun it after approval or remediation.
`doctor` returns `1` when required checks for its readiness scope are unhealthy.
Unavailable optional replay prerequisites remain visible as informational
Unavailable optional provider prerequisites remain visible as informational
diagnostics and do not block setup or unrelated providers. Output format, full envelopes,
filters, and token controls never change the operation status.
@@ -562,9 +560,9 @@ Closing a session shuts down REA's bridge and removes its temporary socket direc
## Process capture
Process capture runs the exact executable and scenario declared in the request,
with the requested working directory, filesystem roots, environment, and
network behavior. The process runs with your user permissions; Process Capture
records behavior and is not a security sandbox.
with the requested working directory and environment. Filesystem observation
paths select what to snapshot. The process runs with your user permissions;
Process Capture records behavior and is not a security sandbox.
Capture a scenario or compare two saved Process Capture Evidence records:
@@ -575,9 +573,9 @@ rea compare-process-captures authority.json reconstruction.json
```
The comparison reports each observed dimension separately and identifies the
first terminal, interaction, exit, filesystem, protocol, process, or shim
divergence. See [Process Capture](docs/process-capture.md) for scenario
fields, command-shim replay, checkpoint triggers, limits, and safety behavior.
first terminal, interaction, exit, filesystem, or process divergence.
See [Process Capture](docs/process-capture.md) for scenario fields, limits, and
evidence boundaries.
REA installs a prebuilt PTY backend for supported macOS, Linux, and Windows
architectures. If the capability check reports that the backend is unavailable,
@@ -596,7 +594,7 @@ verified or absent.
Analysis runs locally. REA communicates with Hopper and Ghidra through authenticated private local sockets. Your agent or model provider has its own data policy.
Runtime requests act on the declared target and lifecycle. Analysis tools and launched targets run with your user permissions, and native UI capture still depends on macOS Accessibility and Screen Recording access. Extracted JavaScript replay runs in a separate Linux sandbox and requires the isolation controls described in [controlled JavaScript replay](docs/controlled-javascript-replay.md).
Runtime requests act on the declared target and lifecycle. Analysis tools and launched targets run with your user permissions, and native UI capture still depends on macOS Accessibility and Screen Recording access. Static JavaScript analysis does not execute extracted modules; use direct browser, Electron, or process capture when runtime behavior is needed.
Windows Ghidra operations are blocked until REA implements the required process ownership, private-directory permissions, and safe-path checks. Report vulnerabilities through the private process in [SECURITY.md](SECURITY.md).
README_ar.md
+12 -12
View File
@@ -200,18 +200,18 @@ rea setup
## منصة أدوات التحقيق
| عائلة الأدوات | العدد | الاستخدام |
| --------------------- | ----: | ------------------------------------------------------------------------------------------------- |
| فحص الملفات التنفيذية | 39 | الدوال والشيفرة شبه المصدرية والتعليمات والسلاسل والرموز والمراجع والتعليقات |
| التحليل المركب | 15 | النظرة العامة وتحليل الدوال وفك الترجمة الدفعي ومخطط الاستدعاء وفحص Swift وObjC |
| أدوات macOS الأصلية | 7 | بيانات Mach-O والتوقيعات وplist والمعماريات واستعادة أسماء Swift |
| الملفات والحزم | 5 | فحص الدلائل والحزم وInterface Builder وموارد Apple والاستخراج |
| .NET PE/CLI | 8 | هوية التجميع والبيانات الوصفية وCIL والاستدعاءات الأصلية والمقارنة واستيراد إعادة البناء |
| مراقبة المتصفح | 9 | الصفحات والبرامج النصية وخرائط المصدر وWebMCP والصور ومقارنة الالتقاط |
| تحليل Electron | 5 | مراقبة الصفحات وبنية التطبيق وربط النتائج الثابتة بنتائج التشغيل والتفاعل المعتمد |
| وقت تشغيل JavaScript | 2 | الاتصال بهدف Node/Electron Inspector موجود لمراقبة البرامج وسياقات التنفيذ |
| سير عمل التطبيقات | 10 | تتبع الميزات ومقارنة الإصدارات وبنية قيم الإرجاع وإعادة التشغيل المعزولة والتحقق من إعادة الإنشاء |
| جلسة الملف التنفيذي | 22 | تبديل الأهداف وحفظ الأدلة ومقارنة العمليات والدوال وتسجيل الأسئلة غير المحسومة |
| عائلة الأدوات | العدد | الاستخدام |
| --------------------- | ----: | ---------------------------------------------------------------------------------------- |
| فحص الملفات التنفيذية | 39 | الدوال والشيفرة شبه المصدرية والتعليمات والسلاسل والرموز والمراجع والتعليقات |
| التحليل المركب | 14 | النظرة العامة وتحليل الدوال وفك الترجمة الدفعي ومخطط الاستدعاء وفحص Swift وObjC |
| أدوات macOS الأصلية | 7 | بيانات Mach-O والتوقيعات وplist والمعماريات واستعادة أسماء Swift |
| الملفات والحزم | 5 | فحص الدلائل والحزم وInterface Builder وموارد Apple والاستخراج |
| .NET PE/CLI | 7 | هوية التجميع والبيانات الوصفية وCIL والاستدعاءات الأصلية والمقارنة واستيراد إعادة البناء |
| مراقبة المتصفح | 9 | الصفحات والبرامج النصية وخرائط المصدر وWebMCP والصور ومقارنة الالتقاط |
| تحليل Electron | 5 | مراقبة الصفحات وبنية التطبيق وربط النتائج الثابتة بنتائج التشغيل |
| وقت تشغيل JavaScript | 2 | الاتصال بهدف Node/Electron Inspector موجود لمراقبة البرامج وسياقات التنفيذ |
| سير عمل التطبيقات | 7 | تتبع الميزات ومقارنة الإصدارات وبنية قيم الإرجاع والتحقق من إعادة الإنشاء |
| جلسة الملف التنفيذي | 21 | تبديل الأهداف وحفظ الأدلة ومقارنة العمليات والدوال وتسجيل الأسئلة غير المحسومة |
## خطة العمل
README_ja.md
+5 -5
View File
@@ -192,15 +192,15 @@ REA は手順 1〜5 のバイナリ解析を処理し、手順 6 はエージェ
| ツール分類 | 数 | 用途 |
| -------------------- | --: | --------------------------------------------------------------------------- |
| バイナリ検査 | 39 | 関数、疑似コード、アセンブリ、文字列、シンボル、参照、注釈 |
| 組み合わせた解析 | 15 | 概要、関数解析、一括逆コンパイル、呼び出しグラフ、Swift と ObjC |
| 組み合わせた解析 | 14 | 概要、関数解析、一括逆コンパイル、呼び出しグラフ、Swift と ObjC |
| macOS ネイティブ | 7 | Mach-O メタデータ、署名、plist、アーキテクチャ、Swift 名の復元 |
| ファイルとパッケージ | 5 | ディレクトリとパッケージ、Interface Builder、Apple リソース、抽出 |
| .NET PE/CLI | 8 | アセンブリ識別、メタデータ、CIL、ネイティブ呼び出し、比較、再構築の取り込み |
| .NET PE/CLI | 7 | アセンブリ識別、メタデータ、CIL、ネイティブ呼び出し、比較、再構築の取り込み |
| ブラウザー観察 | 9 | ページ、スクリプト、ソースマップ、WebMCP、スクリーンショット、比較 |
| Electron 解析 | 5 | ページ観察、アプリ構造、静的・実行時結果の対応付け、承認済み操作 |
| Electron 解析 | 5 | ページ観察、アプリ構造、静的・実行時結果の対応付け |
| JavaScript 実行時 | 2 | 既存の Node/Electron Inspector に接続し、スクリプトと実行コンテキストを観察 |
| アプリのワークフロー | 10 | 機能追跡、バージョン比較、戻り値構造の比較、隔離実行、再実装の検証 |
| バイナリセッション | 22 | 対象切り替え、根拠の保存、プロセス・関数比較、未解決事項の記録 |
| アプリのワークフロー | 7 | 機能追跡、バージョン比較、戻り値構造の比較、再実装の検証 |
| バイナリセッション | 21 | 対象切り替え、根拠の保存、プロセス・関数比較、未解決事項の記録 |
## ロードマップ
README_ko.md
+5 -5
View File
@@ -193,15 +193,15 @@ REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이
| 도구 분류 | 수 | 용도 |
| ----------------- | --: | -------------------------------------------------------------------- |
| 바이너리 검사 | 39 | 함수, 의사 코드, 어셈블리, 문자열, 심볼, 참조, 주석 |
| 결합 분석 | 15 | 개요, 함수 분석, 일괄 디컴파일, 호출 그래프, Swift와 ObjC 검사 |
| 결합 분석 | 14 | 개요, 함수 분석, 일괄 디컴파일, 호출 그래프, Swift와 ObjC 검사 |
| macOS 네이티브 | 7 | Mach-O 메타데이터, 서명, plist, 아키텍처, Swift 이름 복원 |
| 파일과 패키지 | 5 | 디렉터리와 패키지 검사, Interface Builder, Apple 리소스, 추출 |
| .NET PE/CLI | 8 | 어셈블리 식별, 메타데이터, CIL, 네이티브 호출, 비교, 재구성 가져오기 |
| .NET PE/CLI | 7 | 어셈블리 식별, 메타데이터, CIL, 네이티브 호출, 비교, 재구성 가져오기 |
| 브라우저 관찰 | 9 | 페이지, 스크립트, 소스 맵, WebMCP, 스크린샷, 캡처 비교 |
| Electron 분석 | 5 | 페이지 관찰, 앱 구조, 정적·런타임 결과 연결, 승인된 상호작용 |
| Electron 분석 | 5 | 페이지 관찰, 앱 구조, 정적·런타임 결과 연결 |
| JavaScript 런타임 | 2 | 기존 Node/Electron Inspector에 연결해 스크립트와 실행 컨텍스트 관찰 |
| 앱 워크플로 | 10 | 기능 추적, 버전 비교, 반환 구조 비교, 격리 실행, 재구현 검증 |
| 바이너리 세션 | 22 | 대상 전환, 근거 저장, 프로세스·함수 비교, 미해결 항목 기록 |
| 앱 워크플로 | 7 | 기능 추적, 버전 비교, 반환 구조 비교, 재구현 검증 |
| 바이너리 세션 | 21 | 대상 전환, 근거 저장, 프로세스·함수 비교, 미해결 항목 기록 |
## 로드맵
README_zh.md
+5 -5
View File
@@ -192,15 +192,15 @@ REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常
| 工具类别 | 数量 | 用途 |
| ----------------- | ---: | --------------------------------------------------------- |
| 二进制检查 | 39 | 函数、伪代码、汇编、字符串、符号、段、引用与注释 |
| 组合分析 | 15 | 概览、函数分析、批量反编译、调用图、Swift 与 ObjC 检查 |
| 组合分析 | 14 | 概览、函数分析、批量反编译、调用图、Swift 与 ObjC 检查 |
| macOS 原生工具 | 7 | Mach-O 元数据、代码签名、plist、架构与 Swift 符号还原 |
| 制品检查 | 5 | 目录与软件包检查、Interface Builder、Apple 资源目录与提取 |
| .NET PE/CLI | 8 | 程序集身份、元数据、CIL、原生调用声明、成员比较与重建导入 |
| .NET PE/CLI | 7 | 程序集身份、元数据、CIL、原生调用声明、成员比较与重建导入 |
| 浏览器观察 | 9 | 页面、脚本、来源映射、WebMCP 工具、截图与捕获比较 |
| Electron 分析 | 5 | 页面观察、应用结构、静态与运行时结果关联、批准的交互场景 |
| Electron 分析 | 5 | 页面观察、应用结构与静态/运行时结果关联 |
| JavaScript 运行时 | 2 | 连接现有 Node/Electron Inspector，观察脚本与执行上下文 |
| 应用工作流 | 10 | 功能追踪、版本比较、返回结构比较、隔离重放与重建验证 |
| 二进制会话 | 22 | 目标切换、证据包、进程与函数比较、待解决问题记录 |
| 应用工作流 | 7 | 功能追踪、版本比较、返回结构比较与重建验证 |
| 二进制会话 | 21 | 目标切换、证据包、进程与函数比较、待解决问题记录 |
## 路线图
bridge/hopper_bridge.py
+19 -2
View File
@@ -241,6 +241,10 @@ def _procedure_identity(procedure):
"address": _hex(procedure.getEntryPoint()),
"name": _procedure_name(procedure),
"classification": None,
"body": {
"available": False,
"reason": "Hopper's public Python API does not expose complete function body ranges",
},
}
@@ -559,7 +563,11 @@ def _analyze_function(document, params):
referenced_strings.sort(key=lambda item: (int(item["address"], 16), int(item["source_address"], 16)))
referenced_names.sort(key=lambda item: (int(item["address"], 16), int(item["source_address"], 16)))
return {
"procedure": {"address": _hex(procedure.getEntryPoint()), "name": _procedure_name(procedure), "classification": None, "signature": procedure.signatureString(), "locals": _procedure_locals(procedure)},
"procedure": {
**_procedure_identity(procedure),
"signature": procedure.signatureString(),
"locals": _procedure_locals(procedure),
},
"pseudocode": pseudo,
"assembly": assembly_lines,
"comments": comments,
@@ -770,7 +778,16 @@ def _dispatch(method, params):
if method == "procedure_info":
blocks = list(procedure.basicBlockIterator())
length = sum(max(0, block.getEndingAddress() - block.getStartingAddress()) for block in blocks)
return {"name": _procedure_name(procedure), "entrypoint": _hex(procedure.getEntryPoint()), "basicblock_count": procedure.getBasicBlockCount(), "length": length, "signature": procedure.signatureString(), "locals": _procedure_locals(procedure), "classification": None}
return {
"name": _procedure_name(procedure),
"entrypoint": _hex(procedure.getEntryPoint()),
"basicblock_count": procedure.getBasicBlockCount(),
"length": length,
"signature": procedure.signatureString(),
"locals": _procedure_locals(procedure),
"classification": None,
"body": _procedure_identity(procedure)["body"],
}
if method == "set_address_name":
address = _address(document, params.get("address"))
try:
docs/adr/0002-controlled-replay-authority-and-sandbox.md
+7 -8
View File
@@ -1,15 +1,14 @@
# ADR-0002: Controlled JavaScript replay authority and sandbox policy
- Status: Accepted
- Status: Superseded; the controlled JavaScript replay tool was removed
- Date: 2026-07-16
- Implementation status: Implemented by `run_controlled_replay` with a
Linux-x86_64 Bubblewrap, seccomp, and delegated systemd user-cgroup backend.
Hosts that cannot establish every required boundary fail closed.
- Implementation status: The implementation was removed. The design below is
retained as historical context and does not describe a supported REA tool.
> Historical design note: REA's separate permission-policy, grant, and
> per-call approval workflow was later removed. Current tool requests directly
> name their target and lifecycle. The sandbox and exact plan-digest behavior
> described here remain part of controlled replay.
> Historical design note: The tool, sandbox backend, and plan/execute workflow
> described here are no longer part of REA. For runtime behavior, run probes
> directly against the application and use the available browser, Electron, or
> process capture workflows.
## Context
docs/adr/0003-managed-code-evidence-and-provider-boundary.md
+14 -29
View File
@@ -1,11 +1,10 @@
# ADR-0003: Managed-code evidence and provider boundary
> Historical design note: The admission record described below remains a
> plan-only output. The separate administrator and per-call approval workflow
> was removed; current tool requests directly declare their target and
> lifecycle.
> Historical design record: The non-executing runtime-correlation planning
> tool described in earlier revisions was removed. This ADR's current
> implementation status below covers only the static managed tools that ship.
- Status: Accepted
- Status: Superseded for runtime planning; static evidence boundary remains in force
- Date: 2026-07-16
- Implementation status: Read-only PE/CLI triage and exact identity are shipped
through `inspect_managed_artifact` / `rea inspect-managed-artifact`. Bounded
@@ -24,11 +23,8 @@ inspect-managed-members` provides CLI parity. Static member comparison and build
addresses. Decompiler reconstruction import is
shipped through `import_managed_reconstruction` /
`rea import-managed-reconstruction`; it records user-supplied C#/IL/pseudocode
as analyst inference only after exact static member locks match. Non-executing
runtime-correlation planning is shipped through
`plan_managed_runtime_correlation` / `rea plan-managed-runtime-correlation`;
it does not probe a runtime, establish confinement, attach, load, debug,
reflect, instrument, invoke, or execute target code. Managed static graph
as analyst inference only after exact static member locks match. Managed
inspection does not execute assemblies or observe CLR internals. Managed static graph
projection is shipped through
`project_managed_application_graph` /
`rea project-managed-application-graph`; it projects authenticated managed
@@ -56,7 +52,7 @@ makes names especially weak while tokens remain coordinates in only one build.
REA already separates one selected deep native provider from disjoint auxiliary
providers under [ADR-0001](0001-provider-selection-and-analysis-profiles.md).
It also separates static evidence, passive observation, process capture, and
controlled replay authorities. Managed analysis must fit those boundaries
interactive observation authorities. Managed analysis must fit those boundaries
without requiring a CLR to inspect an artifact, loading target code into the
REA process, or treating a managed token as a native address.
@@ -362,22 +358,13 @@ emit full IL, decompiled source, runtime logs, user data, credentials, or
service/account material. No proprietary binary, hash-locked application
manifest, or derived dump is committed to REA.
### 8. Keep runtime correlation planning distinct from execution
### 8. Keep runtime behavior separate from static managed analysis
Runtime-correlation planning records requested effects and bounds without
probing the runtime or establishing confinement. It binds to the exact artifact
SHA-256, MVID, method signature, and CIL/body shape observed by the static
path, and records that no target code ran. A future executor requires its own
provider and threat model before any effect can occur.
Runtime admission must additionally validate host OS, CLR family, architecture,
and supported build/tool versions; bound time, threads, outputs, UI and network
effects; avoid real services and accounts; and own all created processes and
artifacts through complete cleanup. Reflection-only loading is still a CLR load
and is not a substitute for the static byte parser.
Runtime execution is intentionally outside the current managed-code track and
requires its own threat model and admission decision.
No managed runtime planning or execution tool ships. Managed inspection does
not execute assemblies or establish which methods ran. Where supported, direct
process capture can record target-level inputs and outputs, but it does not
prove CLR method execution or resolution. Internal managed behavior remains
unknown until independently observed with suitable runtime evidence.
## Public contract consequences
@@ -444,9 +431,7 @@ or development-only oracles.
5. Add source-built, pinned real-tool, package, CLI, and MCP conformance.
Source-owned conformance is shipped; pinned external real-tool checks remain
planned.
6. Add non-executing runtime-correlation planning. Shipped; runtime execution
and confinement remain planned.
7. Project authenticated managed static Evidence into the application graph.
6. Project authenticated managed static Evidence into the application graph.
Shipped.
Every gate must keep unsupported ReadyToRun, NativeAOT, IL2CPP, mixed-mode, or
docs/adr/README.md
+5 -5
View File
@@ -4,8 +4,8 @@ Accepted decisions describe the architecture that implementation PRs must
follow. Acceptance does not by itself mean the behavior is shipped; each record
states its implementation status separately.
| ADR | Status | Implementation |
| ------------------------------------------------------------------------------------------------------------------ | -------- | ----------------------------------------------------------------------------------------------------------- |
| [0001: Provider selection and analysis profiles](0001-provider-selection-and-analysis-profiles.md) | Accepted | Provider selection and 22 Ghidra operations shipped on Linux/macOS; Windows analysis blocked |
| [0002: Controlled JavaScript replay authority and sandbox policy](0002-controlled-replay-authority-and-sandbox.md) | Accepted | Linux x86_64 extracted-module replay shipped |
| [0003: Managed-code evidence and provider boundary](0003-managed-code-evidence-and-provider-boundary.md) | Accepted | Eight static-analysis and runtime-planning tools shipped; native-body mapping and runtime execution planned |
| ADR | Status | Implementation |
| ------------------------------------------------------------------------------------------------------------------ | -------------------- | -------------------------------------------------------------------------------------------- |
| [0001: Provider selection and analysis profiles](0001-provider-selection-and-analysis-profiles.md) | Accepted | Provider selection and 22 Ghidra operations shipped on Linux/macOS; Windows analysis blocked |
| [0002: Controlled JavaScript replay authority and sandbox policy](0002-controlled-replay-authority-and-sandbox.md) | Superseded | Historical design; the controlled JavaScript replay tool was removed |
| [0003: Managed-code evidence and provider boundary](0003-managed-code-evidence-and-provider-boundary.md) | Partially superseded | Seven static managed tools remain; runtime planning removed |
docs/architecture.mermaid
-1
View File
@@ -24,7 +24,6 @@ graph TD
GHIDRA["src/ghidra/<br/>Ghidra discovery, profile & function analysis<br/>snapshot, Unix/Windows transport, bounded lifecycle"]
AUX["src/artifacts/ + src/native/<br/>disjoint artifact and native capabilities"]
BROWSER["src/browser/<br/>Loopback CDP transport<br/>exact-origin passive observation"]
REPLAY["src/replay/<br/>Disposable JavaScript worker<br/>Bubblewrap, seccomp, cgroup limits"]
end
subgraph "Provider Process Foundation"
docs/browser-scenario-contract.md
+20 -23
View File
@@ -3,22 +3,21 @@
`capture_browser_scenario` accepts the provider-neutral
`browserScenarioSchema` through both MCP and the
`capture-browser-scenario INPUT_JSON` CLI command. The request is declarative:
it admits a fixed action vocabulary, exact HTTP(S) origins, deterministic
browser settings, explicit storage and request replay, and provider-owned
it admits a fixed action vocabulary, explicit HTTP(S) targets, deterministic
browser settings, explicit storage seeds, and provider-owned
liveness deadlines. `INPUT_JSON` may be inline JSON or a JSON file path.
The minimal request contains only browser launch/connect selection, `start_url`,
and at least one `actions` entry. Launch `headless` defaults to `true` and can
be set to `false` where the host supports a visible browser. `allowed_origins`
defaults to the origin of
`start_url`; add exact origins only when navigation, storage, or replay needs
another origin. Environment settings (including service workers blocked by
default), empty storage, disabled request replay, and a final sanitized URL
capture are supplied by default. Set `environment.service_workers` to `allow`
when the application requires them. Add secret declarations, storage seeds,
replay routes, and additional artifact or event capture only when the
investigation needs them. Action, secret, storage,
and replay-route counts are not capped. Duration, action, and navigation
be set to `false` where the host supports a visible browser. Navigations follow
the destinations named by the scenario and the page's ordinary browser requests;
there is no separate origin allowlist or request-interception layer. Environment
settings (including service workers blocked by default), empty storage, and a
final sanitized URL capture are supplied by default. Set
`environment.service_workers` to `allow` when the application requires them.
Add secret declarations, storage seeds, and additional artifact or event
capture only when needed. Action, secret, and storage counts are not capped.
Duration, action, and navigation
timeouts remain fixed provider-owned liveness limits; the request does not
accept a caller-controlled `limits` object.
@@ -32,23 +31,21 @@ confirms that an attached external browser remains alive.
HTTP(S) URLs may include ordinary query values and fragments directly. Use
ordered structured query entries when a value needs to reference a declared
environment-backed secret. Form, storage, cookie, and replay values are either
literal strings or declared secret references. Raw URL userinfo credentials
and provider-owned replay headers are rejected. Captured credential-header
environment-backed secret. Form, storage, and cookie values are either literal
strings or declared secret references. Raw URL userinfo credentials are rejected.
Captured credential-header
values are not retained, and declared secret values are redacted automatically.
A secret may be declared solely to
redact matching observed content; every secret reference in an action, URL,
storage value, or replay route still needs a declaration. Ordinary query values
storage value still needs a declaration. Ordinary query values
and fragments remain intact. Durable results replace resolved secret values
with their secret references.
Every start, navigation, storage, replay, and redirect origin must be in the
effective origin set: the explicit `allowed_origins`, or the `start_url` origin
when the list is omitted or empty. Unsupported action tags, unknown fields, duplicate step or
route IDs, undeclared secret references, and provider-owned replay headers fail
validation. HTTP and WebSocket routing enforces the same declared scope. Exact
replay may abort unmatched requests or pass through only origins included in
`allowed_origins`.
The exact `start_url`, navigation destinations, and storage-seed origins define
the inputs to a scenario; ordinary page requests are handled by the browser.
Storage cookies are scoped to the page URL at each capture. Unsupported action
tags, unknown fields, duplicate step IDs, and undeclared secret references fail
validation.
The result is Evidence with an initial state followed by one record per
declared action. Each step reports action status, elapsed time, sanitized URLs,
docs/controlled-javascript-replay.md
-97
View File
@@ -1,97 +0,0 @@
# Controlled JavaScript replay
`run_controlled_replay` executes only the extracted modules named in the request. It
does not launch or drive the original application. Static inference, passive
browser/Electron observation, isolated replay, and real application behavior
remain different authorities.
## Host prerequisites
Replay requires the current Node.js runtime,
`/usr/bin/bwrap`, `/usr/bin/systemd-run`, `/usr/bin/systemctl`, and
`/usr/bin/bash`. Override executable paths only with the corresponding
`REA_JAVASCRIPT_REPLAY_*_PATH` variable. Configuration creates the
runtime configuration; REA setup never installs these host components. Check
availability with `rea doctor`, which reports exact sandbox-probe failures
without running module code.
## Plan, review, execute
Call the same CLI command or MCP tool twice. The first request uses
`mode: "plan"`. Its response commits module paths and digests, closed dependency
aliases, cases, deterministic providers, runtime/backend identities, resource
limits, no-network policy, private filesystems, and a `plan_digest`.
Runtime commitments include the exact worker, seccomp filter, Node executable,
ELF loader, and shared-library source/destination paths and SHA-256 digests.
The execute request repeats the same manifest and includes the exact digest
returned by the plan request:
```json
{
"mode": "execute",
"plan_digest": "<exact digest returned by plan>"
}
```
The execute request directly selects that content-bound operation. There is no
separate REA grant or approval field.
REA rebuilds the plan. Any changed module, stub, case, runtime, backend, limit,
or export commitment returns `plan_stale` before worker admission.
Modules use `esm` or `commonjs-factory`. The latter accepts extracted Rspack
factory syntax such as `123(module, exports, require) { ... }` and implements
the bounded `require.d`, `require.r`, `require.n`, and `require.nmd` helpers.
Every import or require must map to a declared alias. There is no ambient
package resolution.
Explicit cases can be combined with deterministic `parser-boundaries`,
`sanitizer-boundaries`, or `clipboard-boundaries` generation. Supplying a
`right` manifest runs both sides with fresh realms per case and reports
`equal`, `changed`, or `unknown` comparisons.
## Boundary and Evidence
The worker receives module bytes over bounded stdin and runs under a fresh
Bubblewrap user/PID/network/IPC/UTS namespace, an architecture-checked seccomp
filter, an empty mount root with a descriptor-backed read-only Node closure,
private tmpfs, and a transient systemd user cgroup. Host and external network,
host writes, process spawning, workers, native addons, inspector access, and
ambient environment are unavailable. Wall time, memory, swap, tasks, CPU,
case input, worker-protocol input, output, stderr, depth, and node counts are
independently bounded. The parent also requires an exact worker response shape
and authenticates every returned case ID, order, and input digest. Values are
projected through own data descriptors only; Proxy values and accessors are
rejected without invoking their traps or getters.
Returns, exceptions, serialization failures, denials, timeouts, OOMs, crashes,
protocol failures, and cleanup state are observations with provider
`rea-javascript-replay` and authority `controlled-replay`. An observation does
not prove that the original renderer, preload, main process, browser, or remote
service behaved identically.
Each left/right run is retained as its own `observed` source Evidence. A
differential envelope is `derived` and links those source Evidence IDs, so the
comparison never erases the underlying observations.
Optional `reproducer_export` is committed by the plan and writes to its
caller-supplied path after the sandbox has stopped.
The owner-only manifest is written only after complete sandbox cleanup. Source
bytes are excluded unless `include_sources: true` was selected in the request.
An export failure is retained in the result and does not erase a completed
replay observation.
## Local real-artifact verification
`npm run verify:replay` uses source-owned parser and hostile fixtures. An
operator can additionally supply a local ESM or Rspack/CommonJS-factory
manifest without copying module source into the repository:
```sh
REA_REPLAY_INPUT_PATH=/absolute/replay-manifest.json npm run verify:replay
```
The verifier canonicalizes the manifest's module paths and prints only plan,
module, and Evidence digests, case/comparison summaries, and cleanup state.
This is the intended seam for local Notion-like parser/sanitizer benchmarks.
docs/error-contract.schema.json
-34
View File
@@ -566,40 +566,6 @@
},
"required": ["code", "category", "message", "retryable", "remediation"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"code": { "type": "string", "const": "plan_stale" },
"category": {
"type": "string",
"enum": [
"invalid_input",
"unsupported_provider",
"integrity_mismatch",
"truncated",
"cancelled",
"timeout",
"unavailable",
"execution_failure"
]
},
"message": { "type": "string", "minLength": 1 },
"retryable": { "type": "boolean" },
"remediation": {
"type": "object",
"properties": { "action": { "type": "string", "minLength": 1 } },
"required": ["action"],
"additionalProperties": false
},
"details": {
"type": "object",
"propertyNames": { "type": "string" },
"additionalProperties": { "$ref": "#/$defs/__schema0" }
}
},
"required": ["code", "category", "message", "retryable", "remediation"],
"additionalProperties": false
}
],
"$defs": {
docs/javascript-application-graph.md
+1 -1
View File
@@ -97,7 +97,7 @@ correspondence is a separately identified cross-layer inference.
Every observation and edge carries all of the following:
- an authority, such as artifact bytes, AST analysis, static relationship
inference, passive CDP runtime, cross-layer reconciliation, controlled replay,
inference, passive CDP runtime, and cross-layer reconciliation,
native analysis, historical reference, or user assertion;
- an epistemic state: `observed`, `inferred`, `unknown`, or `unavailable`;
- confidence independently bounded as exact, high, medium, low, or unknown;
docs/javascript-application-workflows.md
+10 -20
View File
@@ -132,8 +132,10 @@ unique occurrence on each side and pairing is reciprocal. Changes use JSON
Pointer paths with `added`, `removed`, `changed`, or `unknown` status. A missing
field is added or removed only when the relevant parent-property coverage is
complete on both shapes. The output includes exact selector candidates,
omissions, Evidence links, coverage, limitations, and a separate controlled
replay recommendation; it does not execute JavaScript.
omissions, Evidence links, coverage, and limitations; it does not execute
JavaScript. When runtime semantics matter, run behavioral probes directly
against the relevant application versions and capture them through the
available browser, Electron, or process workflows.
## CLI and verification
@@ -162,22 +164,10 @@ Because source Evidence and derived Evidence are retained by the normal session
ledger, evidence bundles and analysis snapshots can carry these records without
another persistence format.
## Controlled replay boundary
## Runtime evidence
Static graph workflows never execute a graph node or recovered module.
`run_controlled_replay` is the separate extracted-module boundary. A plan
request and execute request share the same manifest; execution must include the
exact plan digest, and the worker runs inside the mandatory Linux OS sandbox.
The sandbox is an execution boundary; it does not make replay a real application
runtime observation.
Replay observations retain `controlled-replay` authority. They cannot
promote static inference into passive runtime observation or prove that the
original application, renderer, preload, main process, or remote service
behaved identically.
Module, explicit case, and preset-generated case lists have no fixed
item-count ceiling. Replay plans commit the caller's resource budgets, and the
worker protocol checks the complete request against that declared byte budget
before probing executables or reading modules. Aggregate module-source and
case-input bytes are likewise checked against the caller's declared budgets.
Static graph workflows do not execute recovered code. When runtime semantics
matter, run behavioral probes directly against the relevant application
versions and capture them through the available browser, Electron, or process
workflows. Those observations do not prove behavior in an unobserved app
version or environment.
docs/javascript-runtime-observation.md
+1 -1
View File
@@ -9,7 +9,7 @@ deterministic Evidence.
Each request supplies the loopback Inspector endpoint and, for observation, the
target ID. This surface is separate from browser CDP, Electron file-page
inspection, Process Capture, and controlled JavaScript replay:
inspection, and Process Capture:
```bash
rea list-javascript-runtime-targets http://127.0.0.1:9229 \
docs/managed-code-analysis.md
+22 -36
View File
@@ -6,7 +6,7 @@ builds, and connect declared native calls to supplied native-analysis evidence.
CIL is Common Intermediate Language, the instruction format used by managed
assemblies.
The eight shipped tools are:
The seven shipped tools are:
| Task | MCP tool |
| --------------------------------------------------------------------------- | ----------------------------------- |
@@ -16,12 +16,12 @@ The eight shipped tools are:
| Compare members across builds and map build-local tokens | `compare_managed_members` |
| Check native call declarations against supplied export or function evidence | `verify_managed_native_boundaries` |
| Import decompiled code against verified static member identities | `import_managed_reconstruction` |
| Plan a runtime experiment without executing it | `plan_managed_runtime_correlation` |
| Add managed findings to an application graph | `project_managed_application_graph` |
Each has a matching CLI command: replace underscores with hyphens and prefix
the name with `rea`, for example `rea inspect-managed-artifact`.
Native-body bridge mapping and runtime execution remain planned.
Native-body bridge mapping remains planned; managed runtime execution is not
part of the current tool set.
This guide describes the implementation and verification of
[ADR-0003](adr/0003-managed-code-evidence-and-provider-boundary.md). The canonical
@@ -37,8 +37,8 @@ collapsing them:
3. What source-like or behavioral structure can be reconstructed or inferred?
4. Which findings survive an exact-build check or a cross-build structural
comparison?
5. Which remaining questions require native analysis or a future runtime
experiment?
5. Which remaining questions require native analysis or runtime evidence that
static inspection cannot supply?
The ordinary workflow ends at question four. Static analysis never loads or
executes the target.
@@ -236,16 +236,16 @@ Tokens are always remapped through observed structure. A caller cannot carry
## Managed/native boundary rules
| Boundary | Managed observation | Native observation | Permitted link |
| ----------------------- | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| P/Invoke | Module, entry point, charset/calling-convention flags, declaring method | Import/export/symbol/function evidence from the selected deep provider | Exact declared-name/module link or qualified resolution inference |
| COM | Interop attributes, GUIDs, imported interfaces, method signatures | Native registration/vtable evidence when independently available | Identifier/signature inference with explicit environment limitations |
| C++/CLI | Managed declaration and implementation flags | Native body/function evidence | Only a provider-supported bridge observation; never token-as-address |
| ReadyToRun | Component/header and per-method CIL/native availability | Native section/function evidence | Authenticated image mapping with format/profile version |
| Unmanaged export | Export metadata/attribute when present | PE export and native thunk/function | Exact export identity plus provider-qualified address |
| Single-file host | Bundle entry/component identity | Host and native component evidence | Outer bundle plus component digest/extent commitment |
| Unity IL2CPP | Supported metadata entity with authenticated pairing | Generated native function/type evidence | Versioned IL2CPP mapping only; no invented CIL |
| Runtime-resolved native | API/constant/data-flow candidate | Loaded-module/symbol observation only under separate runtime execution authority | Static candidate remains inference until separately observed |
| Boundary | Managed observation | Native observation | Permitted link |
| ----------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| P/Invoke | Module, entry point, charset/calling-convention flags, declaring method | Import/export/symbol/function evidence from the selected deep provider | Exact declared-name/module link or qualified resolution inference |
| COM | Interop attributes, GUIDs, imported interfaces, method signatures | Native registration/vtable evidence when independently available | Identifier/signature inference with explicit environment limitations |
| C++/CLI | Managed declaration and implementation flags | Native body/function evidence | Only a provider-supported bridge observation; never token-as-address |
| ReadyToRun | Component/header and per-method CIL/native availability | Native section/function evidence | Authenticated image mapping with format/profile version |
| Unmanaged export | Export metadata/attribute when present | PE export and native thunk/function | Exact export identity plus provider-qualified address |
| Single-file host | Bundle entry/component identity | Host and native component evidence | Outer bundle plus component digest/extent commitment |
| Unity IL2CPP | Supported metadata entity with authenticated pairing | Generated native function/type evidence | Versioned IL2CPP mapping only; no invented CIL |
| Runtime-resolved native | API/constant/data-flow candidate | Loaded-module/symbol observation from an independently collected runtime source | Static candidate remains inference until separately observed |
Declared import inventory supports a bounded positive claim. Its absence does
not exclude dynamic resolution, generated code, protected code, native helpers,
@@ -402,25 +402,13 @@ The benchmark may prove that REA reproduces selected facts for that exact local
build. It cannot establish general support on its own; source-built conformance
remains the admission requirement.
## Runtime admission boundary
## Runtime behavior boundary
The later runtime track is not an extension flag on a static operation. Its
design must expose whether it attaches to an existing process, launches a new
process, loads an assembly, uses reflection, installs a debugger/profiler, or
instruments code. Each effect must be declared and checked against the exact
build.
At minimum, admission requires target SHA-256, MVID, normalized signature and
body/CIL commitment, CLR family, OS, architecture, tool version, scenario
limits, output policy, and cleanup ownership to match. Any mismatch fails before
the experiment. It must not contact real services/accounts or claim that an
instrumented path represents an ordinary launch unless that proposition is
independently tested.
The shipped `plan_managed_runtime_correlation` path returns an experiment plan
and records that no target code was executed. There is no runtime executor.
Runtime behavior questions therefore remain explicit unknowns with suggested
probes; static support does not perform them.
Managed inspection does not execute assemblies or observe CLR internals.
Questions about actual behavior remain unknown until supported runtime
evidence is collected. A direct process capture can retain declared inputs,
outputs, filesystem and protocol activity for a target run; it does not prove
which managed methods executed or how the CLR resolved them.
## Delivery sequence
@@ -442,9 +430,7 @@ The managed-code track advances as reviewable pull requests:
PE/CLI corpus shipped through `npm run verify:managed`; optional BYO
`ilspycmd` real-tool oracle shipped through `REA_ILSPY_CMD_PATH`; dnSpy and
pinned Windows checks remain planned); and
8. non-executing runtime-correlation planning (shipped; no runtime prerequisite
probe, execution, or confinement claim);
9. managed static Evidence projection into the application graph (shipped).
8. managed static Evidence projection into the application graph (shipped).
Each implementation PR updates generated product facts only for behavior it
actually ships and states which real-tool checks were performed.
docs/mcp-contracts.md
+4 -4
View File
@@ -82,10 +82,10 @@ REA runs the declared request directly and does not infer a broader target or
action from it.
Host requirements remain in force. macOS may deny Accessibility,
Screen Recording, or native mounting; Linux replay requires its sandbox
prerequisites; provider tools require their selected analysis runtime. These
failures are reported at the operation that needs them. A configured provider
or an endpoint alone does not establish that a target is supported.
Screen Recording, or native mounting; provider tools require their selected
analysis runtime. These failures are reported at the operation that needs
them. A configured provider or an endpoint alone does not establish that a
target is supported.
Evidence bundles, snapshots, and extraction use the paths and output behavior
declared by their tools. Artifact extraction materializes the selected regular
docs/process-capture.md
+62 -216
View File
@@ -1,54 +1,33 @@
# Process capture
Process Capture records one caller-selected command as Evidence. It is
intended for authority-versus-reconstruction checks where terminal behavior,
child lifetime, filesystem state, or dependency timing matters.
Process Capture runs one caller-selected command and records bounded evidence
about its terminal behavior, scheduled interactions, process lifetime, and
selected filesystem state. Use it to compare two direct runs of the same
scenario, such as an authority build and a reconstruction. It does not replay a
previous execution or emulate dependencies.
The harness records both raw PTY chunks and terminal states rendered by xterm.
It also records scripted input, resize and signal delivery, sampled process
metadata, named filesystem checkpoints, command-shim invocations, loopback
HTTP/WebSocket exchanges, and process exit ownership. Missing or bounded
observations remain explicit; a truncated capture is never treated as
equivalent to another capture.
The command runs with the current user's permissions and inherits the host
environment before applying the scenario's explicit overrides. This is not a
sandbox. Filesystem observation paths select what REA snapshots; they do not
restrict what the process can read or write. The inherited environment is not
recorded, so its influence may remain unknown.
Every capture includes a run manifest with canonical SHA-256 commitments for
the caller-selected scenario projection, comparison contract, executable,
normalization rules, command-shim plan, and replay plan. The manifest also
records the REA/provider versions, platform, architecture, PTY backend, and UTC
start/completion timestamps. Capture creation, Evidence import, and comparison
recompute the self-contained commitments and reject invalid lifecycle data.
## Capture a command
## Request and host behavior
Write a JSON scenario and pass its path to the CLI:
The scenario names the command and arguments, with optional working-directory,
environment overrides, and filesystem observation paths. An omitted working
directory uses the invocation's current directory. The process inherits the
host environment, then applies any declared overrides. Filesystem observation
paths select what REA records; they do not constrain what the child process can
read or write. Process Capture is not a security sandbox: the target runs with
the current user's permissions.
Local output, arguments, environment overrides, URL query values, shim results,
and replay bodies are preserved without credential-name or text-pattern
redaction. Inputs explicitly marked `sensitive` use placeholders in Evidence.
The inherited environment is not copied into the manifest; its unrecorded
influence remains an explicit limitation.
## Capture a scenario
`rea capture-process` reads a JSON scenario and writes process capture
Evidence to stdout:
```bash
```sh
rea capture-process ./scenario.json > capture.json
```
A representative scenario is:
For example:
```json
{
"executable": "node",
"arguments": ["./signup.mjs"],
"working_directory": ".",
"environment": { "APP_MODE": "test" },
"filesystem_observation_paths": ["./state"],
"terminal": { "columns": 80, "rows": 24, "scrollback": 1000 },
"events": [
@@ -58,196 +37,63 @@ A representative scenario is:
"data": "[email protected]\r",
"sensitive": true
},
{ "type": "resize", "at_ms": 500, "columns": 100, "rows": 30 }
{ "type": "resize", "at_ms": 500, "columns": 100, "rows": 30 },
{ "type": "signal", "at_ms": 1000, "signal": "SIGINT" }
],
"checkpoints": [
{
"name": "signup_ready",
"trigger": { "type": "terminal_literal", "value": "All set!" }
},
{ "name": "root_exited", "trigger": { "type": "root_exit" } }
],
"command_shims": [
{
"name": "codex",
"routes": [
{
"arguments": ["--version"],
"outputs": [
{ "at_ms": 0, "stream": "stdout", "data": "codex 1.2.3\n" }
],
"termination": { "type": "exit", "code": 0 },
"max_calls": 1
}
]
}
]
"timeout_ms": 30000,
"idle_timeout_ms": 30000,
"settle_ms": 100
}
```
Input events marked `sensitive` are dispatched to the PTY but stored only as a
byte-count placeholder. The command-shim directory is placed first in the
captured process `PATH`; command lookup otherwise uses the inherited or
scenario-overridden `PATH`.
`executable` is required. `arguments`, `working_directory`, `environment`,
`filesystem_observation_paths`, terminal settings, timed `events`, timeouts,
resource limits, and normalization settings have defaults; see
`processScenarioSchema` for the exact contract. Event times are milliseconds
from launch, must be ordered, and must fall within `timeout_ms`. Inputs marked
`sensitive` are sent to the process but persisted as a byte-count placeholder.
Environment overrides are recorded in the scenario commitment; inherited
values are not copied into Evidence.
Scenario arguments, environment names, timed interactions, shim routes and
output chunks, filesystem checkpoints, and static HTTP/WebSocket scripts are
accepted without fixed item-count ceilings. The run still obeys its timeout
and the configured output-byte, process, filesystem, protocol-event, body-byte,
and connection budgets; each filesystem checkpoint uses those same
bounded snapshot limits.
The capture contains raw PTY output chunks and rendered terminal states,
interaction dispatch outcomes, exit reason, sampled process-tree observations,
and settlement status. When filesystem paths are selected, REA records
`files_before` and `files_after`, then classifies observed entries as created,
deleted, modified, or unchanged. These are bounded snapshots, not a syscall
trace; short-lived changes between snapshots may be missed. With no selected
paths, filesystem effects remain unknown.
## Reactive process scenarios
Output, file count, file size, process sampling, filesystem depth, total
runtime, idle time, and post-exit settlement are bounded by the scenario's
limits. The result marks truncated observations and residual unknowns rather
than treating missing data as proof of equivalence. Cancellation and timeout
run the same owned-process cleanup path. Settlement reports whether the
sampled process group quiesced or whether cleanup was needed or unverifiable;
sampling cannot prove that every short-lived or detached descendant was seen.
Set `reactive` to a process reactive scenario declaration when interaction
must follow observed output instead of guessed delays. The graph declares an
initial state, absolute scenario and state deadlines, explicitly prioritized
transitions, caller-chosen use and visit counts, trigger predicates, ordered
actions, and either a next state or `passed` outcome.
## Compare two captures
The admitted runtime slice matches decoded UTF-8 terminal literals and exact
`terminal_raw`, `interaction`, `process`, `filesystem`, `http`, `websocket`, or
`shim` journal events. Process events are sampled observations, not an
event-complete lifecycle. Filesystem events identify named checkpoints; shim
events identify a command and route. Live predicates receive the same normalized
and redacted payload later persisted in Evidence. Predicates may compose with
`all`, `any`, `sequence`, and `repeat` nodes and may select a frontier at
scenario start, state entry, a prior checkpoint, or an exact event ID. Actions
can write PTY input, resize it, signal the root process, or capture a named
filesystem checkpoint. `terminal_rendered`, `lifecycle`, and `replay_transition`
event triggers and non-root signal selectors fail during preflight before the
target launches.
Compare saved capture Evidence with:
Reactive results are recorded in `reactive_run`, including the terminal outcome,
active state, matched event IDs, action evidence IDs, and deterministic
transition journal. The reducer retains and evaluates every admitted
observation through the scenario deadline; it has no fixed state, transition,
predicate, action, repeat, or history-count ceilings. Recursive input depth has
a stack safety guard. Terminal controls also commit the last admitted
observation order, so later output cannot be moved ahead of a timeout,
cancellation, or target loss during validation. If the process exits before the graph finishes,
the outcome is `target_lost`; state/scenario deadlines remain distinct
`predicate_timeout`/`scenario_deadline` outcomes. Timed `events` remain
available for fixed schedules, but they are not silently translated into the
reactive graph.
## Checkpoints and deterministic shims
Checkpoint triggers may use an elapsed time, a terminal literal and occurrence
count, root exit, or settlement. REA always includes `before` and
`after_settlement` snapshots. Snapshot effects are relative to the preceding
checkpoint, so transient files remain visible even when the final tree matches
the initial tree.
Each command shim matches an exact argument array. Routes can emit timed stdout
or stderr chunks and then exit or receive `SIGINT`, `SIGTERM`, or `SIGKILL`.
Unmatched and exhausted calls are recorded. Shim observations are included in
the capture without a separate fixed count ceiling. Other configured
observation and byte budgets remain explicit in the capture result when they
affect completeness.
## Run a replay machine directly
Use `rea run-replay-machine ./run.json` to validate a finite replay machine
against an ordered `events` array without launching a process or opening a
socket. MCP clients use `run_replay_machine` with the same `{ machine, events }`
input. The result retains one decision per offered event, a transition journal
with capture alias metadata, and one redacted action-table entry per used
transition. It never returns request bodies, request headers, or captured
values. It also reports the initial and final states, whether the final state is
terminal, configured limits, and committed usage.
The direct evaluator has no fixed state, transition, event, guard, capture,
action, or action-byte count ceiling. It indexes transitions by protocol and
path, evaluates events in order, and returns a decision for each offered event
plus actions for used transitions. Caller-declared `max_uses`, `max_visits`,
`max_transitions`, and protocol budgets determine the finite run; there is no
upper schema cap on those values. Result size follows the input and is not
truncated by an arbitrary item quota.
Refused events remain data rather than aborting the run. Outcomes distinguish
unmatched events, invalid states, failed guards, exhausted transitions, invalid
captures, unexpected reconnects, and exhausted limits. This direct runner
evaluates the same domain runtime used by loopback Process Capture, but performs
no network or target execution.
## Compare captures
Compare two saved Evidence records with:
```bash
```sh
rea compare-process-captures authority.json reconstruction.json
```
Pass an optional third path, `rea compare-process-captures authority.json
reconstruction.json trace.json`, when valid executions may differ in declared
concurrent ordering. MCP callers pass the same object as `trace_spec` to
`compare_process_captures`. A trace specification declares exact named events
and chooses one language:
The comparison checks terminal, interaction, exit, filesystem, and
process observations under a shared comparison contract. It reports observed
differences with their locations, while residual unknowns or truncated
observations prevent a complete-equivalence claim. Matching scenario
commitments do not reveal whether redacted sensitive inputs were equal.
- `partial_order` requires every event pair to be related by a
`happens_before` edge (including transitive edges) or an explicit
`unordered_groups` declaration. `not_before` expresses negative ordering
constraints without inferring causality. Optional `prefix` and `suffix`
arrays remain exact.
- `finite_traces` lists the complete accepted variants. An `unordered` token
accepts permutations of exactly its declared event multiset; it does not
drop or broadly sort events.
For scenarios where independent scheduling can change event order, the CLI
also accepts an optional trace-specification JSON file as the third argument.
That specification must state the exact events and ordering constraints to
accept; it does not discard unmatched observations or make timestamps into
causal evidence. Trace comparison returns `unknown` when a capture lacks the
required complete event journal or contains relevant residual unknowns.
Each event uses one source family (`terminal_raw`, `terminal_rendered`,
`interaction`, `lifecycle`, `process`, `filesystem`, `http`, `websocket`,
`shim`, or `replay_transition`), an exact JSON payload, and a discriminated
cardinality: `required`, `optional`, `exact`, or `range`. Specifications with
duplicate or overlapping predicates, unknown references, cycles, implicit
ordering gaps, ordered/unordered conflicts, or finite variants that violate
cardinality are rejected before comparison.
Event payloads are exact by default. To compare schedules whose recorder
metadata necessarily changes, an event may explicitly list top-level
`ignore_fields` from the bounded set `sequence`, `at_ms`, `scheduled_at_ms`,
`dispatched_at_ms`, and `elapsed_ms`; its `exact` object must omit those fields.
All predicates for one source use the same ignore set, so relaxed metadata
cannot create declaration-order matching ambiguity. Other payload fields are
never dropped or normalized by trace comparison.
Process Capture records a monotonic `event_journal` at observation time. Trace
comparison uses only this capture order for cross-source causality; timestamps
are retained as event data but never promoted into happens-before evidence.
Older captures without a complete journal, truncated captures, and relevant
residual unknowns return a trace verdict of `unknown`. A passing result retains
each side's raw journal locations and identifies either the matched finite
variant or the complete satisfied constraint set. A failure returns the first
declaration-ordered predicate, cardinality, edge, prefix, suffix, or language
violation with its relevant event locations.
If both captures have the same invalid declared trace, the nested verdict is
`nonconforming` while the top-level capture comparison remains unchanged. Side
statuses and the diagnostic distinguish conformance from pairwise difference.
The result classifies terminal, interaction, exit, filesystem, protocol,
process, and shim behavior. `first_divergence` points to the earliest observed
difference. Residual unknowns prevent a claim of complete equivalence, but they
do not hide an observed difference in another dimension.
Captures must have equal comparison-contract commitments, but may identify
different scenarios and executables. MCP callers may set `max_capture_age_ms`;
the application clock then rejects stale captures before comparison.
## Limits and lifecycle behavior
Scenarios bound output bytes, frames, files, file bytes, process observations,
protocol/shim events, connections, filesystem depth, total runtime, idle time,
and settlement time. Process trees are sampled rather than syscall-traced, and
short-lived descendants may be missed. Filesystem observations are snapshots,
not filesystem event streams.
REA tags launched processes with a private run identifier. During cleanup it
revalidates that identifier before signaling sampled process groups, including
groups detached from the original parent. Temporary HOME, replay, shim,
terminal, checkpoint, and sampling resources are released on success, failure,
timeout, and cancellation.
After root exit, REA checks token-owned process groups every 50 ms. Two
consecutive empty observations establish `quiesced`; the settlement deadline
otherwise produces `alive_at_deadline`, while inspection or identity failures
produce `unverifiable`. Sampling can still miss short-lived or early-detached
descendants, which remains a residual unknown.
Each capture carries commitments for the selected scenario, executable,
comparison contract, and normalization rules. Comparison rejects captures
whose comparison contracts differ. Captures are local Evidence files; keep
their source artifacts and invocation context available when interpreting a
difference.
docs/product-catalog.json
+11 -26
View File
@@ -10,7 +10,7 @@
"skill_version": "23"
},
"tools": {
"total": 122,
"total": 116,
"families": [
{
"id": "direct",
@@ -61,13 +61,12 @@
{
"id": "enhanced",
"surface": "enhanced",
"count": 15,
"count": 14,
"tools": [
"analyze_function",
"analyze_swift_types",
"batch_decompile",
"binary_overview",
"find_code_for_string",
"find_xrefs_to_name",
"get_call_graph",
"get_objc_classes",
@@ -109,14 +108,13 @@
{
"id": "managed",
"surface": "managed",
"count": 8,
"count": 7,
"tools": [
"compare_managed_members",
"import_managed_reconstruction",
"inspect_managed_artifact",
"inspect_managed_members",
"inspect_managed_native_boundaries",
"plan_managed_runtime_correlation",
"project_managed_application_graph",
"verify_managed_native_boundaries"
]
@@ -161,16 +159,13 @@
{
"id": "application",
"surface": "application-workflow",
"count": 10,
"count": 7,
"tools": [
"build_reconstruction_obligation_ledger",
"compare_application_versions",
"compare_javascript_export_shapes",
"compare_source_to_bundle",
"evaluate_reconstruction_coverage",
"execute_node_characterization",
"prepare_node_characterization",
"run_controlled_replay",
"trace_application_feature",
"trace_javascript_semantics"
]
@@ -178,7 +173,7 @@
{
"id": "session",
"surface": "session",
"count": 22,
"count": 21,
"tools": [
"binary_session",
"build_call_path",
@@ -198,7 +193,6 @@
"list_unknowns",
"open_binary",
"record_unknown",
"run_replay_machine",
"update_unknown",
"verify_reconstruction",
"verify_unknown_resolution"
@@ -344,7 +338,6 @@
"capabilities": [
"compare_managed_members",
"import_managed_reconstruction",
"plan_managed_runtime_correlation",
"project_managed_application_graph",
"verify_managed_native_boundaries"
]
@@ -365,9 +358,6 @@
"compare_javascript_export_shapes",
"compare_source_to_bundle",
"evaluate_reconstruction_coverage",
"execute_node_characterization",
"prepare_node_characterization",
"run_controlled_replay",
"trace_application_feature",
"trace_javascript_semantics"
]
@@ -445,7 +435,7 @@
}
],
"cli": {
"primary_count": 74,
"primary_count": 69,
"commands": [
"analyze",
"analyze-javascript-application",
@@ -473,7 +463,6 @@
"evaluate-reconstruction-coverage",
"evidence-export",
"evidence-import",
"execute-node-characterization",
"extract-artifact",
"function",
"import-managed-reconstruction",
@@ -502,14 +491,10 @@
"observe-javascript-runtime",
"observe-native-ui",
"observe-web-session",
"plan-managed-runtime-correlation",
"prepare-node-characterization",
"project-managed-application-graph",
"providers",
"reconcile-javascript-runtime",
"resolve-native-call-targets",
"run-controlled-replay",
"run-replay-machine",
"search",
"setup",
"trace",
@@ -531,15 +516,15 @@
},
"runtime_catalog": {
"counts": {
"cli_commands": 74,
"mcp_tools": 122,
"cli_commands": 69,
"mcp_tools": 116,
"mcp_prompts": 6
},
"digests": {
"tools_sha256": "9f2b8a77ca7d972dedddf33294f5addb5fc7e00ce43ae5de21be989e0777871a",
"tools_sha256": "ff29fddb875d335c400902e846377990fa4276f04e4494a255dd3986a0fd7896",
"prompts_sha256": "4762db348821fec787a5b8c6423f2c48a71f6f0ec2afee97b16ad69030ec504b",
"combined_sha256": "3a0e6d66df95b913797ce2e4548e97d409d6e8768023f8a7a23d8d85ae9d766f",
"providers_sha256": "c5a88a49990fafc8244972ee8ff543fc19995eb79db960da55f6a311e15340d1"
"combined_sha256": "73a4cdf4fc83b1728426931ff89e2113f1c8beb1b3bf337c179f2c349ce3c0f2",
"providers_sha256": "2ae8b85b20be338739e500549d275e78e875aaa10235d2ca283f0ade7787da90"
}
}
}
docs/roadmap.md
+2 -2
View File
@@ -37,8 +37,8 @@ related-tool change; setup must never install Java.
Before enabling Windows Ghidra operations, add current-user-only DACL creation and
readback, handle-based reparse-point-safe path authority, a DACL-protected IPC
backend, and Job Object assignment before provider execution. Process capture,
controlled replay, Hopper, and broad filesystem-sensitive workflows remain
separate Windows projects rather than implied parity.
Hopper, and broad filesystem-sensitive workflows remain separate Windows
projects rather than implied parity.
## Capability-selective setup
docs/testing.md
+7 -4
View File
@@ -45,10 +45,13 @@ The process-global Vitest configuration contract rejects new direct temporary-ro
creation outside the workspace seam and its narrowly documented boundary/package
exceptions.
Real Hopper, Ghidra, browser, package, managed-code, and controlled-replay
claims belong to their explicit `npm run verify:*` lanes. They are not inferred
from mocks or folded into the deterministic local gate. Real model trials are
manual; Vitest covers only deterministic evaluator logic.
Real Hopper, Ghidra, browser, package, and managed-code claims belong to their
explicit `npm run verify:*` lanes. The reconstruction-readiness lane also
checks deterministic rerun, tamper, and stale-input handling; those checks do
not execute extracted JavaScript modules. When application runtime behavior is
needed, exercise the actual target through browser, Electron, or process
capture. Real model trials are manual; Vitest covers deterministic evaluator
logic.
## End-to-end, integration and golden evidence
docs/tool-design.md
+8 -8
View File
@@ -8,14 +8,14 @@ protocol, authority, and resource-safety constraints, and report their effects.
## Choose the tool shape
| Shape | Use it for | Contract should return |
| -------------------------------- | -------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `inspect` | Facts about one explicit target, address, object, or resource | Relevant fields, source locations, and facet-level availability |
| `search` / `list` | Finding candidate targets or entities | Stable ordering and useful result context; expose pagination when the result format or caller's query needs it |
| `trace` | Relationships across code, metadata, UI resources, or observations | Typed edges, supporting evidence, and unresolved paths; identify any genuine traversal boundary |
| `compare` | Two explicitly identified artifacts, versions, or evidence sets | Paired identity, comparable coverage, and deltas with evidence |
| `workflow` | A distinct analyst outcome that benefits from composition inside REA | A useful inline result, contributing evidence, and partial/unavailable facets |
| `observe` / `capture` / `replay` | A question that requires runtime behavior | Required authority, launch/attach behavior, real operational constraints, lifecycle, and cleanup status |
| Shape | Use it for | Contract should return |
| --------------------- | -------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `inspect` | Facts about one explicit target, address, object, or resource | Relevant fields, source locations, and facet-level availability |
| `search` / `list` | Finding candidate targets or entities | Stable ordering and useful result context; expose pagination when the result format or caller's query needs it |
| `trace` | Relationships across code, metadata, UI resources, or observations | Typed edges, supporting evidence, and unresolved paths; identify any genuine traversal boundary |
| `compare` | Two explicitly identified artifacts, versions, or evidence sets | Paired identity, comparable coverage, and deltas with evidence |
| `workflow` | A distinct analyst outcome that benefits from composition inside REA | A useful inline result, contributing evidence, and partial/unavailable facets |
| `observe` / `capture` | A question that requires runtime behavior | Required authority, launch/attach behavior, real operational constraints, lifecycle, and cleanup status |
These are task shapes, not required prefixes. Name a tool for the action and
object agents reason about; keep the name distinct from nearby alternatives.
docs/verification/managed-conformance-ledger.json
+3 -10
View File
@@ -1,5 +1,5 @@
{
"ledger_id": "ecl_b01c0b28362b48128765d176113471a5a82637fca0acfcd3182971155942e0d4",
"ledger_id": "ecl_132e857a8a93f0b366cd72f9cfa1c338ffd4100460a7834ef69abff936c84472",
"records": [
{
"claim_id": "managed.application-graph-projection",
@@ -78,13 +78,6 @@
"ev_f974f4b9d039cda841894c6173cae86427ecef48f4e7a4d6c73edf7b23af2b2a"
]
},
{
"claim_id": "managed.runtime-correlation-admission-plan",
"status": "pass",
"evidence_ids": [
"ev_3e546c20493faf9696502171a5cfe9504569cffb22f0561ee3000f0027b9bbd8"
]
},
{
"claim_id": "managed.unicode-obfuscated-identifiers",
"status": "pass",
@@ -101,8 +94,8 @@
}
],
"summary": {
"total": 14,
"pass": 12,
"total": 13,
"pass": 11,
"fail": 0,
"unsupported": 2,
"truncated": 0,
docs/verification/managed-conformance-manifest.json
+2 -16
View File
@@ -1,5 +1,5 @@
{
"manifest_id": "ecm_d8e34c1218d199b3639830b184f02f87642c42aa9e8e8e2474f8daa60e1645ac",
"manifest_id": "ecm_c47e33ff69b099583ff6a3b19cb06b602040ecc735351cbac21b897501c4cffe",
"verifier": {
"id": "rea-managed-conformance",
"version": "1"
@@ -13,7 +13,7 @@
"skill_digests": [
{
"skill_id": "reverse-engineer-anything",
"sha256": "7d767925a840318504e9f1712a6ecc4a985cd7a08f00bffe36105b75481a54c2"
"sha256": "54490266ef46ed8e57354dbca8cc1ad74ab6b7f21d38116bebfc87a04bfb55a5"
}
],
"claims": [
@@ -169,20 +169,6 @@
"version": "1"
}
},
{
"claim_id": "managed.runtime-correlation-admission-plan",
"scenario": {
"id": "runtime-correlation-admission-plan",
"version": 1
},
"artifact_sha256s": [
"cead4cc99733e4e56a5fd0cec8a0813bbc24e249aa72a10eca4c47ee3b0e129d"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.unicode-obfuscated-identifiers",
"scenario": {
docs/windows-ghidra-p0.md
+3 -3
View File
@@ -21,9 +21,9 @@ Once the required native controls are implemented, the intended P0 is limited to
- an explicit native, non-managed, non-DLL x86-64 PE application; and
- the 22 read-only Ghidra inventory and function-analysis operations.
Hopper, Ghidra GUI state, mutation, controlled JavaScript replay, process
capture, browser/Electron observation, artifact extraction, and general
Windows feature parity are not implied by this boundary. Managed PE/CLI
Hopper, Ghidra GUI state, mutation, process capture, browser/Electron
observation, artifact extraction, and general Windows feature parity are not
implied by this boundary. Managed PE/CLI
inspection remains a separate execution-free provider and is not routed
through Windows Ghidra P0.
package.json
-1
View File
@@ -132,7 +132,6 @@
"verify:javascript-runtime": "npm run build:cached && node scripts/verify-local-javascript-runtime.mjs",
"verify:application-workflows": "npm run build:cached && node scripts/verify-local-application-workflows.mjs",
"verify:managed": "npm run build:cached && node scripts/verify-managed-conformance.mjs",
"verify:replay": "npm run build:cached && node scripts/verify-real-javascript-replay.mjs",
"verify:agent": "npm run build:cached && node scripts/verify-agent-experience.mjs",
"verify:hopper:linux": "npm run verify:fixtures && npm run build:cached && node scripts/verify-real-hopper-linux.mjs",
"verify:setup:linux": "npm run build:cached && node scripts/verify-linux-setup-e2e.mjs",
scripts/lib/browser-scenario-verifier.mjs
-2
View File
@@ -53,7 +53,6 @@ export function browserScenario(browser, origin) {
},
],
},
allowed_origins: [origin],
environment: {
viewport: { width: 1_280, height: 720, device_scale_factor: 1.25 },
locale: "en-US",
@@ -83,7 +82,6 @@ export function browserScenario(browser, origin) {
},
],
storage: duplicateOriginStorage(origin),
request_replay: { mode: "disabled" },
secrets: [
{
secret_id: "verifier_url",
scripts/lib/browser-screenshot-e2e.mjs
-1
View File
@@ -77,7 +77,6 @@ export async function verifyLargeScreenshotE2e(endpoint, origin) {
name: "capture_web_screenshot",
arguments: {
cdp_endpoint: endpoint,
allowed_origins: [origin],
target_id: target.id,
},
},
scripts/lib/managed-completion-report.mjs
-5
View File
@@ -93,11 +93,6 @@ const sourceOwnedClaims = ({
scenarioId: "decompiler-reconstruction-import",
targets: [obfuscated.target],
}),
completionClaim({
claimId: "managed.runtime-correlation-admission-plan",
scenarioId: "runtime-correlation-admission-plan",
targets: [obfuscated.target],
}),
completionClaim({
claimId: "managed.mvid-and-token-drift",
scenarioId: "mvid-and-token-drift",
scripts/verify-agent-experience.mjs
+5 -5
View File
@@ -82,7 +82,7 @@ try {
"depth = 1",
],
["static", "inferred"],
["runtime", "replay"],
["runtime", "behavioral probes"],
],
prompt: `Without directly reading target files, compare the default export in parser.mjs between ${targets.javascriptShapeLeft} and ${targets.javascriptShapeRight}. Use REA to analyze each shipped artifact and then compare its exact static export return shapes. State the exact heading-shape change, cite produced Evidence, and distinguish static inference from runtime semantics.`,
},
@@ -283,13 +283,13 @@ async function createTargets(root, includeManaged) {
mkdir(javascriptShapeRight, { recursive: true }),
]);
await Promise.all([
cp(
join(repositoryRoot, "tests/fixtures/replay/parser.mjs"),
writeFile(
join(javascriptShapeLeft, "parser.mjs"),
'export default function parse() { return { heading: "Title" }; }\n',
),
cp(
join(repositoryRoot, "tests/fixtures/replay/parser-v2.mjs"),
writeFile(
join(javascriptShapeRight, "parser.mjs"),
'export default function parse() { return { heading: { text: "Title", level: 1 } }; }\n',
),
]);
scripts/verify-managed-conformance.mjs
+7 -50
View File
@@ -1,4 +1,4 @@
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import assert from "node:assert/strict";
@@ -9,7 +9,6 @@ import { inspectManagedNativeBoundariesBytes } from "../dist/dotnet/ManagedNativ
import { compareManagedMemberPaths } from "../dist/application/ManagedMemberComparisonService.js";
import { verifyManagedNativeBoundariesEvidence } from "../dist/application/ManagedNativeVerificationService.js";
import { importManagedReconstructionEvidence } from "../dist/application/ManagedReconstructionService.js";
import { planManagedRuntimeCorrelationEvidence } from "../dist/application/ManagedRuntimeCorrelationService.js";
import { projectManagedApplicationGraphEvidence } from "../dist/application/ManagedApplicationGraphService.js";
import { traceApplicationFeatureEvidence } from "../dist/application/JavaScriptApplicationWorkflowService.js";
import { MANAGED_STATIC_PROVIDER } from "../dist/application/InvestigationProviders.js";
@@ -249,9 +248,8 @@ try {
assert.equal(obfuscatedMembers.types[0]?.full_name, "Fixture.ꙮType");
assert.equal(obfuscatedMembers.methods[0]?.name, "λ⛧");
assert.equal(obfuscatedMembers.fields[0]?.name, "字段");
const runtimeExecutable = process.execPath;
const runtimeMethod = obfuscatedMembers.methods[0];
assert.ok(runtimeMethod);
const reconstructionMethod = obfuscatedMembers.methods[0];
assert.ok(reconstructionMethod);
const obfuscatedMembersEvidence = createEvidence(
undefined,
MANAGED_STATIC_PROVIDER,
@@ -274,9 +272,9 @@ try {
},
methods: [
{
token: runtimeMethod.token,
signature_sha256: runtimeMethod.signature.raw_sha256,
normalized_il_sha256: runtimeMethod.body.normalized_il_sha256,
token: reconstructionMethod.token,
signature_sha256: reconstructionMethod.signature.raw_sha256,
normalized_il_sha256: reconstructionMethod.body.normalized_il_sha256,
reconstruction: {
kind: "decompiled-csharp",
language: "csharp",
@@ -293,45 +291,6 @@ try {
false,
);
assert.equal(reconstructionImport.value.confidence, "inferred");
const runtimePlan = await planManagedRuntimeCorrelationEvidence(
{ configuration: () => ({ executablePath: runtimeExecutable }) },
{
static_members: obfuscatedMembersEvidence,
method: {
token: runtimeMethod.token,
signature_sha256: runtimeMethod.signature.raw_sha256,
normalized_il_sha256: runtimeMethod.body.normalized_il_sha256,
},
requested_effect: "debugger",
host: {
os: "linux",
clr_family: "dotnet",
architecture: "x86_64",
},
bounds: {
timeout_ms: 5_000,
max_threads: 32,
max_output_bytes: 65_536,
allow_network: false,
allow_ui: false,
},
},
);
assert.equal(runtimePlan.ok, true);
assert.equal(runtimePlan.value.normalized_result.executed, false);
assert.equal(
runtimePlan.value.normalized_result.unsupported_until_executor_exists,
true,
);
assert.equal(
runtimePlan.value.normalized_result.effect_taxonomy.uses_debugger,
true,
);
assert.equal(
runtimePlan.value.normalized_result.requested_runtime.confinement,
"not-established",
);
const left = await fixture("token-drift-left.exe", {
methodName: "StableSemanticSlice",
});
@@ -410,7 +369,7 @@ try {
process.stdout.write(
`${JSON.stringify({
verified:
12 +
11 +
(operatorManifest === null ? 0 : 1) +
(ilspyOracle === null ? 0 : 1),
managedSurfaces: [
@@ -420,7 +379,6 @@ try {
"compare_managed_members",
"verify_managed_native_boundaries",
"import_managed_reconstruction",
"plan_managed_runtime_correlation",
"project_managed_application_graph",
],
coverage: [
@@ -431,7 +389,6 @@ try {
"managed-application-graph-projection",
"unicode-obfuscated-identifiers",
"decompiler-reconstruction-import",
"runtime-correlation-admission-plan",
"mvid-and-token-drift",
"not-managed",
"malformed-metadata",
scripts/verify-package-managed.mjs
-70
View File
@@ -115,69 +115,6 @@ const verifyManagedReconstruction = async ({
throw new Error("packaged managed reconstruction import CLI failed");
};
const verifyManagedRuntimePlan = async ({
cli,
managedMembers,
managedMethod,
workspace,
environment,
}) => {
const managedRuntimePlanInput = {
static_members: managedMembers,
method: {
token: managedMethod.token,
signature_sha256: managedMethod.signature.raw_sha256,
normalized_il_sha256: managedMethod.body.normalized_il_sha256,
},
requested_effect: "attach",
host: {
os: "linux",
clr_family: "dotnet",
architecture: "x86_64",
},
bounds: {
timeout_ms: 5_000,
max_threads: 32,
max_output_bytes: 65_536,
allow_network: false,
allow_ui: false,
},
};
const managedRuntimePlan = json(
await run(
cli,
[
"plan-managed-runtime-correlation",
JSON.stringify(managedRuntimePlanInput),
"--json",
],
{
...environment,
REA_MANAGED_RUNTIME_EXECUTABLE_PATH: process.execPath,
},
),
);
if (
managedRuntimePlan.operation !== "plan_managed_runtime_correlation" ||
managedRuntimePlan.provider?.id !== "rea-dotnet-workflows" ||
managedRuntimePlan.normalized_result?.executed !== false ||
managedRuntimePlan.normalized_result?.unsupported_until_executor_exists !==
true ||
managedRuntimePlan.normalized_result?.effect_taxonomy?.attaches_process !==
true ||
managedRuntimePlan.normalized_result?.effect_taxonomy
?.invokes_target_code !== false
)
throw new Error("packaged managed runtime-correlation CLI failed");
if (
managedRuntimePlan.normalized_result?.requested_runtime?.confinement !==
"not-established"
)
throw new Error(
"packaged managed runtime plan claimed unverified confinement",
);
};
const verifyManagedBoundaries = async ({ cli, managedPath, environment }) => {
const managedBoundaries = json(
await run(
@@ -363,13 +300,6 @@ export async function verifyManaged({ cli, workspace, environment }) {
managedMethod,
environment,
});
await verifyManagedRuntimePlan({
cli,
managedMembers,
managedMethod,
workspace,
environment,
});
const managedBoundaries = await verifyManagedBoundaries({
cli,
managedPath,
scripts/verify-package.mjs
+1 -1
View File
@@ -117,7 +117,7 @@ try {
evidenceRoot,
});
process.stdout.write(
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), cli: true, analysisCli: true, artifactCli: true, managedCli: true, managedReconstructionCli: true, managedNativeVerificationCli: true, managedRuntimePlanCli: true, managedApplicationGraphCli: true, evidenceCli: true, incurMcpCommand: PRODUCT_IDENTITY.mcpCommand, lifecycleScriptsRequired: false, doctor: "platform-appropriate", setup: supportedSetupHost ? "planned-then-idempotent" : "unsupported-host-rejected", setupPlanReadOnly: supportedSetupHost, existingHopperPreserved: supportedSetupHost, clients: supportedSetupHost ? 3 : 0, backupReadback: supportedSetupHost, failureRecovery: supportedSetupHost, configSymlinkLifecycle: supportedSetupHost, skill: supportedSetupHost, skillReferences: supportedSetupHost, mcpTools: TOOL_CONTRACTS.length, mcpPrompts: prompts.names.length, promptCompletion: true, promptCompletionLifecycle: true, evidenceMcp: true, targetFree: true, targetLifecycle: true, boundedRegexBridge: true, mcpStartup, mcpModuleLoading })}\n`,
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), cli: true, analysisCli: true, artifactCli: true, managedCli: true, managedReconstructionCli: true, managedNativeVerificationCli: true, managedApplicationGraphCli: true, evidenceCli: true, incurMcpCommand: PRODUCT_IDENTITY.mcpCommand, lifecycleScriptsRequired: false, doctor: "platform-appropriate", setup: supportedSetupHost ? "planned-then-idempotent" : "unsupported-host-rejected", setupPlanReadOnly: supportedSetupHost, existingHopperPreserved: supportedSetupHost, clients: supportedSetupHost ? 3 : 0, backupReadback: supportedSetupHost, failureRecovery: supportedSetupHost, configSymlinkLifecycle: supportedSetupHost, skill: supportedSetupHost, skillReferences: supportedSetupHost, mcpTools: TOOL_CONTRACTS.length, mcpPrompts: prompts.names.length, promptCompletion: true, promptCompletionLifecycle: true, evidenceMcp: true, targetFree: true, targetLifecycle: true, boundedRegexBridge: true, mcpStartup, mcpModuleLoading })}\n`,
);
} finally {
if (tarball) await rm(join(root, tarball), { force: true });
scripts/verify-real-browser.mjs
+3 -4
View File
@@ -82,7 +82,7 @@ try {
});
const endpoint = `http://127.0.0.1:${String(port)}`;
const provider = new CdpBrowserProvider();
const target = await pageTarget(provider, endpoint, site.origin);
const target = await pageTarget(provider, endpoint);
const observed = await provider.inspectPage(
inspectWebPageInputSchema.parse({
cdp_endpoint: endpoint,
@@ -355,7 +355,7 @@ try {
}
async function verifyPageScopedTransport(provider, proxy, origin) {
const target = await pageTarget(provider, proxy.endpoint, origin);
const target = await pageTarget(provider, proxy.endpoint);
const input = inspectWebPageInputSchema.parse({
cdp_endpoint: proxy.endpoint,
allowed_origins: [origin],
@@ -432,12 +432,11 @@ async function browserExecutable() {
);
}
async function pageTarget(provider, endpoint, origin) {
async function pageTarget(provider, endpoint) {
for (let attempt = 0; attempt < 100; attempt += 1) {
const listed = await provider.listTargets(
listBrowserTargetsInputSchema.parse({
cdp_endpoint: endpoint,
allowed_origins: [origin],
}),
);
if (listed.ok && listed.value.targets[0] !== undefined)
scripts/verify-real-javascript-replay.mjs
-314
View File
@@ -1,314 +0,0 @@
#!/usr/bin/env node
import { readFile, realpath } from "node:fs/promises";
import { resolve } from "node:path";
import { runControlledReplay } from "../dist/application/JavaScriptReplayService.js";
import { LinuxJavaScriptReplayRunner } from "../dist/replay/LinuxJavaScriptReplayRunner.js";
import { SystemJavaScriptReplayHost } from "../dist/replay/SystemJavaScriptReplayHost.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const verifierRun = createVerifierRun();
const suppliedInputPath = process.env.REA_REPLAY_INPUT_PATH;
const suppliedInput =
suppliedInputPath === undefined
? undefined
: JSON.parse(await readFile(await realpath(suppliedInputPath), "utf8"));
const canonicalSide = async (side) => ({
...side,
modules: await Promise.all(
side.modules.map(async (module) => ({
...module,
path: await realpath(module.path),
})),
),
});
const leftPath = await realpath(
process.env.REA_REPLAY_LEFT_PATH ??
resolve("tests/fixtures/replay/parser.mjs"),
);
const rightPath = await realpath(
process.env.REA_REPLAY_RIGHT_PATH ??
resolve("tests/fixtures/replay/parser-v2.mjs"),
);
const suppliedLeft =
suppliedInput === undefined
? undefined
: await canonicalSide(suppliedInput.left);
const suppliedRight =
suppliedInput?.right === undefined
? undefined
: await canonicalSide(suppliedInput.right);
const configuration = {
nodePath: process.execPath,
bubblewrapPath:
process.env.REA_JAVASCRIPT_REPLAY_BWRAP_PATH ?? "/usr/bin/bwrap",
systemdRunPath:
process.env.REA_JAVASCRIPT_REPLAY_SYSTEMD_RUN_PATH ??
"/usr/bin/systemd-run",
systemctlPath:
process.env.REA_JAVASCRIPT_REPLAY_SYSTEMCTL_PATH ?? "/usr/bin/systemctl",
shellPath: process.env.REA_JAVASCRIPT_REPLAY_SHELL_PATH ?? "/usr/bin/bash",
};
const dependencies = {
configuration: () => configuration,
host: new SystemJavaScriptReplayHost(),
runner: new LinuxJavaScriptReplayRunner(),
};
const defaultInput = {
mode: "plan",
left: {
modules: [
{
alias: "parser",
path: leftPath,
format: "esm",
role: "module",
dependencies: {},
},
],
entry_alias: "parser",
entry_export: "default",
},
right: {
modules: [
{
alias: "parser",
path: rightPath,
format: "esm",
role: "module",
dependencies: {},
},
],
entry_alias: "parser",
entry_export: "default",
},
cases: [
{ case_id: "heading", arguments: ["# Title"] },
{ case_id: "plain", arguments: ["plain"] },
],
generator: { preset: "parser-boundaries", seed: 7, count: 4 },
};
const suppliedPlanInput =
suppliedInput === undefined
? undefined
: (({ approved: _approved, plan_digest: _planDigest, ...planInput }) =>
planInput)(suppliedInput);
const input =
suppliedPlanInput === undefined
? defaultInput
: {
...suppliedPlanInput,
mode: "plan",
left: suppliedLeft,
...(suppliedRight === undefined ? {} : { right: suppliedRight }),
reproducer_export: undefined,
};
const planned = await runControlledReplay(dependencies, input);
if (!planned.ok) {
if (process.env.REA_REPLAY_ALLOW_UNAVAILABLE === "true") {
process.stdout.write(
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), available: false, error_tag: planned.error._tag })}\n`,
);
process.exit(0);
}
throw planned.error;
}
const plan = planned.value.plan;
if (plan === null || typeof plan.plan_digest !== "string")
throw new Error("Real replay verifier did not receive a plan");
const executed = await runControlledReplay(dependencies, {
...input,
mode: "execute",
plan_digest: plan.plan_digest,
});
if (!executed.ok) throw executed.error;
const evidence = executed.value.evidence;
if (
evidence === null ||
evidence.authority !== "controlled-replay" ||
evidence.environment?.isolation !== "container" ||
evidence.normalized_result.termination !== "completed" ||
evidence.normalized_result.cleanup.state !== "complete" ||
evidence.normalized_result.comparison?.some(
({ status }) => status === "unknown",
) === true
)
throw new Error(
`Real replay verifier received incomplete Evidence: ${JSON.stringify({
authority: evidence?.authority ?? null,
isolation: evidence?.environment?.isolation ?? null,
termination: evidence?.normalized_result?.termination ?? null,
cleanup: evidence?.normalized_result?.cleanup?.state ?? null,
outcomes:
evidence?.normalized_result?.outcomes?.map(({ outcome }) => outcome) ??
[],
stderr: evidence?.normalized_result?.stderr ?? null,
comparison_statuses:
evidence?.normalized_result?.comparison?.map(({ status }) => status) ??
[],
})}`,
);
if (suppliedInput !== undefined) {
process.stdout.write(
`${JSON.stringify({
verifier_run: await completeVerifierRun(verifierRun),
available: true,
supplied_manifest: true,
plan_digest: plan.plan_digest,
evidence_id: evidence.evidence_id,
source_evidence_ids: executed.value.source_evidence.map(
({ evidence_id: id }) => id,
),
left_sha256: plan.left.modules.map(({ sha256 }) => sha256),
right_sha256: plan.right?.modules.map(({ sha256 }) => sha256) ?? [],
cases: plan.cases.length,
comparison:
evidence.normalized_result.comparison?.map(({ case_id, status }) => ({
case_id,
status,
})) ?? [],
cleanup: evidence.normalized_result.cleanup.state,
})}\n`,
);
process.exit(0);
}
const runSingle = async (path, limits = undefined) => {
const manifest = {
mode: "plan",
left: {
modules: [
{
alias: "entry",
path: await realpath(path),
format: "esm",
role: "module",
dependencies: {},
},
],
entry_alias: "entry",
entry_export: "default",
},
cases: [{ case_id: "probe", arguments: [] }],
...(limits === undefined ? {} : { limits }),
};
const proposed = await runControlledReplay(dependencies, manifest);
if (!proposed.ok || proposed.value.plan === null)
throw new Error("Replay conformance plan failed");
const observed = await runControlledReplay(dependencies, {
...manifest,
mode: "execute",
plan_digest: proposed.value.plan.plan_digest,
});
if (!observed.ok || observed.value.evidence === null)
throw new Error("Replay conformance execution failed");
return observed.value.evidence.normalized_result;
};
const sideEffects = await runSingle(
resolve("tests/fixtures/replay/side-effect-attempt.mjs"),
);
if (
sideEffects.outcomes[0]?.outcome !== "return" ||
Object.values(sideEffects.outcomes[0].value).some(
(value) => value !== "undefined",
)
)
throw new Error("Ambient side-effect surface was exposed to replay code");
const timeout = await runSingle(
resolve("tests/fixtures/replay/busy-loop.mjs"),
{
wall_time_ms: 100,
},
);
if (timeout.termination !== "timeout" || timeout.cleanup.state !== "complete")
throw new Error("Replay timeout did not kill and clean the complete sandbox");
const differentialTimeoutInput = {
mode: "plan",
left: {
modules: [
{
alias: "entry",
path: await realpath(resolve("tests/fixtures/replay/busy-loop.mjs")),
format: "esm",
role: "module",
dependencies: {},
},
],
entry_alias: "entry",
entry_export: "default",
},
right: {
modules: [
{
alias: "entry",
path: await realpath(resolve("tests/fixtures/replay/parser.mjs")),
format: "esm",
role: "module",
dependencies: {},
},
],
entry_alias: "entry",
entry_export: "default",
},
cases: [{ case_id: "probe", arguments: [] }],
limits: { wall_time_ms: 100 },
};
const differentialTimeoutPlan = await runControlledReplay(
dependencies,
differentialTimeoutInput,
);
if (!differentialTimeoutPlan.ok || differentialTimeoutPlan.value.plan === null)
throw new Error("Differential timeout plan failed");
const differentialTimeout = await runControlledReplay(dependencies, {
...differentialTimeoutInput,
mode: "execute",
plan_digest: differentialTimeoutPlan.value.plan.plan_digest,
});
if (
!differentialTimeout.ok ||
differentialTimeout.value.evidence === null ||
differentialTimeout.value.source_evidence.length !== 2 ||
differentialTimeout.value.evidence.normalized_result.comparison?.[0]
?.status !== "unknown" ||
differentialTimeout.value.source_evidence.some(
(item) => item.normalized_result.outcomes[0]?.outcome !== "timeout",
)
)
throw new Error("Differential termination did not retain both observations");
const memory = await runSingle(
resolve("tests/fixtures/replay/memory-pressure.mjs"),
{ wall_time_ms: 3000, memory_bytes: 32 * 1024 * 1024 },
);
if (memory.termination !== "oom" || memory.cleanup.state !== "complete")
throw new Error(
`Replay OOM did not report and clean the cgroup limit: ${JSON.stringify(memory)}`,
);
process.stdout.write(
`${JSON.stringify({
verifier_run: await completeVerifierRun(verifierRun),
available: true,
plan_digest: plan.plan_digest,
evidence_id: evidence.evidence_id,
left_sha256: plan.left.modules[0]?.sha256,
right_sha256: plan.right?.modules[0]?.sha256,
cases: plan.cases.length,
comparison: evidence.normalized_result.comparison?.map(
({ case_id, status }) => ({ case_id, status }),
),
cleanup: evidence.normalized_result.cleanup.state,
side_effect_surface: "unavailable",
timeout: timeout.termination,
differential_timeout:
differentialTimeout.value.evidence.normalized_result.comparison[0]
?.status,
memory: memory.termination,
})}\n`,
);
skills/reverse-engineer-anything/SKILL.md
+7 -7
View File
@@ -3,14 +3,14 @@ name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, and browser applications with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "23"
tool_count: 122
catalog_digest: "3a0e6d66df95b913797ce2e4548e97d409d6e8768023f8a7a23d8d85ae9d766f"
tool_count: 116
catalog_digest: "73a4cdf4fc83b1728426931ff89e2113f1c8beb1b3bf337c179f2c349ce3c0f2"
---
# REA
Use REA when a claim depends on a shipped binary or package, decompilation,
passive application runtime evidence, controlled replay, or comparison with
passive application runtime evidence, or comparison with
behavior not established by available source. For ordinary analysis of a
complete source repository, use normal repository tools and do not run REA
readiness or provider commands.
@@ -71,8 +71,10 @@ For requests that span multiple features or subsystems, use a staged workflow:
they affect the question. Use format-aware inventory and parsers; do not
infer behavior from filenames, strings, or layout alone.
5. Corroborate a conclusion with the evidence type it requires. Use runtime
observation or controlled replay only when static evidence cannot answer the
question and the required host runtime and OS access are available.
observation only when static evidence cannot answer the question and the
required host runtime and OS access are available. For JavaScript behavior,
run probes against the actual app through the available browser, Electron,
or process capture workflow.
6. Decompose work into independent questions. When parallel workers are
available and the questions do not depend on one another, assign distinct
scopes, point workers to existing evidence, and ask them to return sources,
@@ -96,8 +98,6 @@ as complete while required questions remain open.
[references/runtime-observation.md](references/runtime-observation.md)
- Evidence paging, comparisons, residual unknowns, and verification:
[references/evidence-workflows.md](references/evidence-workflows.md)
- Controlled JavaScript replay:
[references/controlled-replay.md](references/controlled-replay.md)
## Readiness and setup
skills/reverse-engineer-anything/references/controlled-replay.md
-12
View File
@@ -1,12 +0,0 @@
# Controlled JavaScript replay
Use `run_controlled_replay` only for operator-selected extracted modules when
the separate replay policy is enabled. Call `mode: plan` first. Review the exact
module, stub, runtime, sandbox, case, limits, and policy commitments. Execute
only with `approved: true` and that exact plan digest.
Treat return values, exceptions, denials, limits, and crashes as observations of
the isolated experiment with controlled-replay authority, not facts about the
real application. Never substitute browser/Electron permission, process
capture, or an in-process `vm` run. Reproducer export requires separate literal
approval and evidence-write authority after sandbox cleanup.
skills/reverse-engineer-anything/references/javascript-applications.md
+3 -2
View File
@@ -31,5 +31,6 @@ Include the complete Evidence records from both analysis calls. Accept variant
pairing only through the tool's unique exact literal discriminant. Cite the
comparison Evidence and report JSON Pointer changes; dynamic values, ambiguous
variants, and incomplete parent-property coverage stay unknown. This is static
inference, not runtime behavior. Use `run_controlled_replay` separately only
when the user needs approved runtime semantics.
inference, not runtime behavior. When runtime semantics are needed, run
behavioral probes against the relevant application versions and capture them
through the available browser, Electron, or process workflows.
src/application/BrowserScenarioEvidence.ts
+2 -3
View File
@@ -8,18 +8,17 @@ import {
type EvidenceObservation,
} from "../domain/evidence.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import { digestJson } from "./JavaScriptReplayPlanning.js";
import { digestCanonicalJson } from "../domain/browserScenarioNormalization.js";
const browserScenarioParameters = (
scenario: BrowserScenario,
): EvidenceObservation["parameters"] => ({
scenario_sha256: digestJson(scenario),
scenario_sha256: digestCanonicalJson(scenario),
browser_mode: scenario.browser.mode,
...(scenario.browser.mode === "launch"
? { browser_headless: scenario.browser.headless }
: {}),
start_url: sanitizeBrowserUrl(scenario.start_url.url),
allowed_origins: scenario.allowed_origins,
environment: scenario.environment,
actions: scenario.actions.map(({ step_id, action }) => ({
step_id,
src/application/CapabilityInventory.test.ts
-21
View File
@@ -4,7 +4,6 @@ import { buildCapabilityInventory } from "./CapabilityInventory.js";
const enabledPolicy: Parameters<typeof buildCapabilityInventory>[1] = {
processCaptureEnabled: true,
controlledReplayEnabled: true,
browserObservationEnabled: true,
browserScenarioEnabled: true,
electronObservationEnabled: true,
@@ -159,26 +158,6 @@ describe("capability inventory: provider status", () => {
available: false,
reason: "provider_missing",
});
expect(byName.get("run_controlled_replay")).toMatchObject({
available: true,
reason: "available",
});
expect(byName.get("plan_managed_runtime_correlation")).toMatchObject({
available: true,
reason: "available",
});
});
it("reports controlled replay as unsupported on other host targets", () => {
const replay = entry("run_controlled_replay", status(), {
...enabledPolicy,
controlledReplayEnabled: false,
});
expect(replay).toMatchObject({
available: false,
reason: "unsupported_host",
remediation: "Run controlled replay on a Linux x86_64 host.",
});
});
});
src/application/CapabilityInventory.ts
-12
View File
@@ -11,7 +11,6 @@ import {
import type { ToolKind } from "../contracts/toolContractTypes.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { GENERATED_MCP_TOOL_CATALOG } from "../generatedMcpToolCatalog.js";
import { isSupportedControlledReplayHost } from "./ControlledReplayHostSupport.js";
import {
clientRequirementsFor,
NO_CLIENT_FEATURES,
@@ -29,7 +28,6 @@ type ToolAvailabilityReason = "available" | ToolUnavailabilityReason;
type ProviderDescriptor = ProviderCapability;
export type AvailabilityPolicy = {
readonly processCaptureEnabled: boolean;
readonly controlledReplayEnabled?: boolean;
readonly browserObservationEnabled?: boolean;
readonly browserScenarioEnabled?: boolean;
readonly electronObservationEnabled?: boolean;
@@ -214,16 +212,6 @@ const workflowAvailabilityFor = ({
reason: "unsupported_host",
remediation: "Run process capture on a supported Linux or macOS host.",
};
if (name === "run_controlled_replay")
return (policy.controlledReplayEnabled ??
isSupportedControlledReplayHost(process.platform, process.arch))
? { reason: "available", remediation: null }
: {
reason: "unsupported_host",
remediation: "Run controlled replay on a Linux x86_64 host.",
};
if (name === "plan_managed_runtime_correlation")
return { reason: "available", remediation: null };
if (kind === "application") return { reason: "available", remediation: null };
const electron = electronProviderAvailability(name, kind, policy);
if (electron !== null) return electron;
src/application/CommandShimReplay.ts
-202
View File
@@ -1,202 +0,0 @@
import { createServer, type Server } from "node:http";
import { chmod, mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import type {
ProcessScenario,
RecordProcessCaptureEvent,
ShimEvent,
} from "../domain/processCapture.js";
interface Invocation {
readonly command: string;
readonly arguments: readonly string[];
readonly working_directory: string;
}
/** Owned command-shim replay resources for one process scenario. */
export interface CommandShimReplay {
readonly binPath: string;
readonly url: string;
readonly events: readonly ShimEvent[];
readonly truncated: boolean;
close(): Promise<void>;
}
const RUNNER = `const command = process.argv[2];
const maximumTimerDelay = 2147483647;
const delay = milliseconds => new Promise(resolve => {
let remaining = milliseconds;
const schedule = () => {
const interval = Math.min(remaining, maximumTimerDelay);
setTimeout(() => {
remaining -= interval;
if (remaining === 0) resolve();
else schedule();
}, interval);
};
schedule();
});
const response = await fetch(process.env.REA_SHIM_LEDGER_URL, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ command, arguments: process.argv.slice(3), working_directory: process.cwd() }),
});
if (!response.ok) process.exit(127);
const route = await response.json();
let elapsed = 0;
for (const output of route.outputs) {
const wait = Math.max(0, output.at_ms - elapsed);
if (wait > 0) await delay(wait);
elapsed = output.at_ms;
(output.stream === "stdout" ? process.stdout : process.stderr).write(output.data);
}
if (route.termination.type === "signal") process.kill(process.pid, route.termination.signal);
else process.exit(route.termination.code);
`;
const shellQuote = (value: string): string =>
`'${value.replaceAll("'", `'\\''`)}'`;
/**
* Start bounded executable replay wrappers and their private invocation ledger.
*
* Exact argument matching is deliberate: a fallback to the host executable
* would make an unmatched dependency call look authoritative. Unmatched and
* exhausted routes therefore fail visibly and remain part of the capture.
*/
export const startCommandShimReplay = async (
scenario: ProcessScenario,
temporaryRoot: string,
started: number,
recordEvent: RecordProcessCaptureEvent = () => undefined,
): Promise<CommandShimReplay> => {
const binPath = join(temporaryRoot, "shims");
await mkdir(binPath);
const runnerPath = join(temporaryRoot, "shim-runner.mjs");
await writeFile(runnerPath, RUNNER, { mode: 0o600, flag: "wx" });
for (const shim of scenario.command_shims) {
const path = join(binPath, shim.name);
const wrapper = `#!/bin/sh\nexec ${shellQuote(process.execPath)} ${shellQuote(runnerPath)} ${shellQuote(shim.name)} "$@"\n`;
await writeFile(path, wrapper, { mode: 0o700, flag: "wx" });
await chmod(path, 0o700);
}
const events: ShimEvent[] = [];
let truncated = false;
const calls = new Map<string, number>();
const server = createServer((request, response) => {
if (request.method !== "POST" || request.url !== "/invoke") {
response.writeHead(404).end();
return;
}
readInvocation(request)
.then((invocation) => {
const shim = scenario.command_shims.find(
({ name }) => name === invocation.command,
);
const routeIndex = shim?.routes.findIndex(
({ arguments: expected }) =>
JSON.stringify(expected) === JSON.stringify(invocation.arguments),
);
const key = `${invocation.command}:${String(routeIndex ?? -1)}`;
const used = calls.get(key) ?? 0;
const route =
shim !== undefined && routeIndex !== undefined && routeIndex >= 0
? shim.routes[routeIndex]
: undefined;
const outcome =
route === undefined
? "unmatched"
: used >= route.max_calls
? "exhausted"
: "matched";
if (events.length >= scenario.limits.protocol_events) truncated = true;
else {
const index = events.length;
events.push({
sequence: index,
at_ms: Math.max(0, Date.now() - started),
command: invocation.command,
route_index:
routeIndex !== undefined && routeIndex >= 0 ? routeIndex : null,
arguments: invocation.arguments,
working_directory: invocation.working_directory,
outcome,
});
recordEvent("shim_events", index);
}
if (outcome !== "matched" || route === undefined) {
response.writeHead(409).end();
return;
}
calls.set(key, used + 1);
response
.writeHead(200, { "content-type": "application/json" })
.end(JSON.stringify(route));
})
.catch(() => response.writeHead(400).end());
});
await listen(server);
const address = server.address();
if (address === null || typeof address === "string") {
await closeServer(server);
throw new Error("command shim ledger did not bind an IPv4 port");
}
return {
binPath,
url: `http://127.0.0.1:${String(address.port)}/invoke`,
events,
get truncated() {
return truncated;
},
close: () => closeServer(server),
};
};
const readInvocation = async (
request: import("node:http").IncomingMessage,
): Promise<Invocation> => {
const chunks: Buffer[] = [];
let bytes = 0;
for await (const chunk of request) {
const buffer = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
bytes += buffer.length;
if (bytes > 64 * 1024) throw new Error("shim invocation exceeds limit");
chunks.push(buffer);
}
const value: unknown = JSON.parse(Buffer.concat(chunks).toString("utf8"));
if (
typeof value !== "object" ||
value === null ||
!("command" in value) ||
typeof value.command !== "string" ||
!("arguments" in value) ||
!Array.isArray(value.arguments) ||
!value.arguments.every((item) => typeof item === "string") ||
!("working_directory" in value) ||
typeof value.working_directory !== "string"
)
throw new Error("invalid shim invocation");
return {
command: value.command,
arguments: value.arguments,
working_directory: value.working_directory,
};
};
const listen = (server: Server): Promise<void> =>
new Promise((resolveListen, rejectListen) => {
server.once("error", rejectListen);
server.listen(0, "127.0.0.1", () => {
server.off("error", rejectListen);
resolveListen();
});
});
const closeServer = (server: Server): Promise<void> =>
new Promise((resolveClose, rejectClose) =>
server.close((cause) =>
cause === undefined ? resolveClose() : rejectClose(cause),
),
);
src/application/ControlledReplayHostSupport.test.ts
-12
View File
@@ -1,12 +0,0 @@
import { describe, expect, it } from "vitest";
import { isSupportedControlledReplayHost } from "./ControlledReplayHostSupport.js";
describe("controlled replay host support", () => {
it("admits only Linux x86_64", () => {
expect(isSupportedControlledReplayHost("linux", "x64")).toBe(true);
expect(isSupportedControlledReplayHost("linux", "arm64")).toBe(false);
expect(isSupportedControlledReplayHost("darwin", "x64")).toBe(false);
expect(isSupportedControlledReplayHost("win32", "x64")).toBe(false);
});
});
src/application/ControlledReplayHostSupport.ts
-5
View File
@@ -1,5 +0,0 @@
/** Return whether this host has the platform boundary required by replay v1. */
export const isSupportedControlledReplayHost = (
platform: string,
architecture: string,
): boolean => platform === "linux" && architecture === "x64";
src/application/Doctor.runtimeExecutables.test.ts
-66
View File
@@ -1,66 +0,0 @@
import { describe, expect, it } from "vitest";
import { createDoctorHostFixture as host } from "./Doctor.fixture.js";
import { runDoctor } from "./Doctor.js";
describe("doctor runtime executable diagnostics", () => {
it("reports a broken shadowed Node candidate without hiding the healthy launcher", async () => {
const result = await runDoctor(
undefined,
host({
runtimeExecutables: () =>
Promise.resolve({
launcher_node: "/healthy/node",
candidates: [
{
tool: "node",
lexical_path: "/healthy/node",
canonical_path: "/healthy/node",
path_index: null,
selection: "rea-launcher",
version: "v24.18.0",
healthy: true,
failure: null,
},
{
tool: "node",
lexical_path: "/opt/homebrew/bin/node",
canonical_path: "/opt/homebrew/Cellar/node/25.2.1/bin/node",
path_index: 2,
selection: "path-shadowed",
version: null,
healthy: false,
failure: {
code: "runtime_dynamic_library_missing",
exit_code: 134,
signal: null,
dependency:
"/opt/homebrew/opt/simdjson/lib/libsimdjson.29.dylib",
stderr: "dyld: Library not loaded",
},
},
],
}),
}),
);
expect(result.healthy).toBe(false);
expect(result.identity?.runtime_executables).toMatchObject({
launcher_node: "/healthy/node",
candidates: [
{ healthy: true, selection: "rea-launcher" },
{
healthy: false,
failure: { code: "runtime_dynamic_library_missing" },
},
],
});
expect(
result.checks.find(({ name }) => name === "node-toolchains"),
).toMatchObject({
ok: false,
classification: "missing_dependency",
remediation: expect.stringContaining("Do not create"),
});
});
});
src/application/Doctor.test.ts
-27
View File
@@ -3,33 +3,6 @@ import type { ClientRegistrationStatus } from "./ClientRegistrationStatus.js";
import { createDoctorHostFixture as host } from "./Doctor.fixture.js";
import { runDoctor } from "./Doctor.js";
it("reports unavailable replay without blocking installation or Hopper readiness", async () => {
const replayCheck = {
name: "javascript-replay",
ok: false,
classification: "unsupported_host" as const,
detail: "Controlled replay requires Linux x86_64",
};
const doctorHost = host({
javascriptReplayCheck: () => Promise.resolve(replayCheck),
});
for (const scope of [undefined, { providers: ["hopper"] }]) {
const report = await runDoctor(undefined, doctorHost, scope);
expect(report.healthy).toBe(true);
expect(report.checks).toContainEqual(replayCheck);
expect(report.informational_checks).toContainEqual(replayCheck);
expect(report.scope_checks).not.toContainEqual(replayCheck);
}
const unhealthy = await runDoctor(
undefined,
host({
nodeVersion: "20.0.0",
javascriptReplayCheck: () => Promise.resolve(replayCheck),
}),
);
expect(unhealthy.healthy).toBe(false);
});
describe("doctor", () => {
it("returns exact recovery for every failed diagnostic", async () => {
const result = await runDoctor(
src/application/Doctor.ts
+3 -32
View File
@@ -20,14 +20,9 @@ import {
type ClientRegistrationStatus,
type UnhealthyClientRegistrationStatus,
} from "./ClientRegistrationStatus.js";
import {
inspectRuntimeExecutables,
type RuntimeExecutableInventory,
} from "./RuntimeExecutableDiagnostics.js";
import {
collectDoctorDiagnostics,
doctorHealthy,
isOptionalDoctorCheck,
} from "./DoctorDiagnostics.js";
import {
normalizeDoctorScope,
@@ -91,9 +86,7 @@ export interface DoctorHost {
installationPaths?(): Promise<readonly string[]>;
installedSkillIdentity?(): Promise<InstalledSkillIdentity | undefined>;
clientRegistrations?(): Promise<readonly ClientRegistrationStatus[]>;
javascriptReplayCheck?(): Promise<DoctorCheck>;
ilspyCmdVersion?(path: string): Promise<string | undefined>;
runtimeExecutables?(): Promise<RuntimeExecutableInventory>;
}
/** Parsed identity committed by an installed REA skill. */
@@ -145,7 +138,6 @@ interface DoctorIdentity {
readonly remediation: string | null;
};
readonly registrations: readonly ClientRegistrationStatus[];
readonly runtime_executables: RuntimeExecutableInventory | null;
}
/**
@@ -162,9 +154,8 @@ export const runDoctor = async (
checks: diagnosticChecks,
hopperPath,
providerInspections,
runtimeExecutables,
} = await collectDoctorDiagnostics(target, host);
const identity = await collectDoctorIdentity(host, runtimeExecutables);
const identity = await collectDoctorIdentity(host);
const checks = [...diagnosticChecks, ...identity.checks];
const providers = {
hopperAvailable:
@@ -188,14 +179,9 @@ export const runDoctor = async (
healthy: scope.mode === "audit-wide" ? environmentHealthy : scoped.healthy,
environment_healthy: environmentHealthy,
scope,
scope_checks:
scope.mode === "audit-wide"
? checks.filter(({ name }) => !isOptionalDoctorCheck(name))
: scoped.scopeChecks,
scope_checks: scope.mode === "audit-wide" ? checks : scoped.scopeChecks,
informational_checks:
scope.mode === "audit-wide"
? checks.filter(({ name }) => isOptionalDoctorCheck(name))
: scoped.informationalChecks,
scope.mode === "audit-wide" ? [] : scoped.informationalChecks,
...(hopperPath === undefined ? {} : { hopperPath }),
...(providerInspections === undefined ? {} : { providerInspections }),
checks,
@@ -205,7 +191,6 @@ export const runDoctor = async (
const collectDoctorIdentity = async (
host: DoctorHost,
runtimeExecutables: RuntimeExecutableInventory | undefined,
): Promise<{
readonly checks: readonly DoctorCheck[];
readonly value: DoctorIdentity;
@@ -253,7 +238,6 @@ const collectDoctorIdentity = async (
: "Run rea setup to update the installed REA skill.",
},
registrations,
runtime_executables: runtimeExecutables ?? null,
},
};
};
@@ -298,7 +282,6 @@ export interface SystemDoctorHostOptions {
readonly providerInspections?: () => Promise<
readonly DoctorProviderInspection[]
>;
readonly javascriptReplayCheck?: () => Promise<DoctorCheck>;
readonly linuxDemoRuntimeCheck?: () => Promise<DoctorCheck>;
}
@@ -344,9 +327,6 @@ export const systemDoctorHost = (
...(options.providerInspections === undefined
? {}
: { providerInspections: options.providerInspections }),
...(options.javascriptReplayCheck === undefined
? {}
: { javascriptReplayCheck: options.javascriptReplayCheck }),
async ilspyCmdVersion(path) {
try {
await access(path, constants.X_OK);
@@ -357,15 +337,6 @@ export const systemDoctorHost = (
return undefined;
}
},
runtimeExecutables: () =>
inspectRuntimeExecutables({
platform: process.platform,
path: process.env.PATH ?? "",
launcherNode: process.execPath,
...(process.env.PATHEXT === undefined
? {}
: { pathExtensions: process.env.PATHEXT.split(delimiter) }),
}),
async installationPaths() {
try {
const command = process.platform === "win32" ? "where" : "which";
src/application/DoctorDiagnostics.ts
+4 -40
View File
@@ -7,7 +7,6 @@ import type {
DoctorHost,
DoctorProviderInspection,
} from "./Doctor.js";
import type { RuntimeExecutableInventory } from "./RuntimeExecutableDiagnostics.js";
const DEFAULT_HOPPER =
"/Applications/Hopper Disassembler.app/Contents/MacOS/hopper";
@@ -18,7 +17,6 @@ export interface DoctorDiagnostics {
readonly checks: readonly DoctorCheck[];
readonly hopperPath?: string;
readonly providerInspections?: readonly DoctorProviderInspection[];
readonly runtimeExecutables?: RuntimeExecutableInventory;
}
/** Collect host, provider, and optional-target diagnostics without mutation. */
@@ -26,11 +24,7 @@ export const collectDoctorDiagnostics = async (
target: string | undefined,
host: DoctorHost,
): Promise<DoctorDiagnostics> => {
const runtimeExecutables = await host.runtimeExecutables?.();
const checks: DoctorCheck[] = [
nodeCheck(host),
...runtimeChecks(runtimeExecutables),
];
const checks: DoctorCheck[] = [nodeCheck(host)];
checks.push(await hostCheck(host));
const hopperPath = await findHopper(host);
@@ -42,8 +36,6 @@ export const collectDoctorDiagnostics = async (
if (providerInspections !== undefined)
checks.push(...providerInspections.flatMap(providerDoctorChecks));
const javascriptReplayCheck = await host.javascriptReplayCheck?.();
if (javascriptReplayCheck !== undefined) checks.push(javascriptReplayCheck);
const ilspyCheck = await optionalIlspyCheck(host);
if (ilspyCheck !== undefined) checks.push(ilspyCheck);
const hopperDemoCheck = await optionalHopperDemoCheck(host, hopperPath);
@@ -55,7 +47,6 @@ export const collectDoctorDiagnostics = async (
checks,
...(hopperPath === undefined ? {} : { hopperPath }),
...(providerInspections === undefined ? {} : { providerInspections }),
...(runtimeExecutables === undefined ? {} : { runtimeExecutables }),
};
};
@@ -70,19 +61,16 @@ export const doctorHealthy = (
| undefined;
},
): boolean => {
const requiredChecks = checks.filter(
({ name }) => !isOptionalDoctorCheck(name),
);
if (providers.providerInspections === undefined)
return requiredChecks.every(({ ok }) => ok);
const providerChecks = requiredChecks.filter(
return checks.every(({ ok }) => ok);
const providerChecks = checks.filter(
({ name }) =>
name.startsWith("hopper") ||
providers.providerInspections?.some((inspection) =>
providerCheckName(inspection.id, name),
) === true,
);
const coreHealthy = requiredChecks
const coreHealthy = checks
.filter((candidate) => !providerChecks.includes(candidate))
.every(({ ok }) => ok);
const configuredProvidersHealthy = providerChecks
@@ -102,36 +90,12 @@ export const doctorHealthy = (
);
};
/** Identify workflow diagnostics that do not gate core installation readiness. */
export const isOptionalDoctorCheck = (name: string): boolean =>
name === "javascript-replay";
const nodeCheck = (host: DoctorHost): DoctorCheck =>
check("node", supportsNodeVersion(host.nodeVersion), host.nodeVersion, {
remediation: "Install Node.js 22.19+ or 24.11+.",
classification: "missing_dependency",
});
const runtimeChecks = (
inventory: RuntimeExecutableInventory | undefined,
): readonly DoctorCheck[] => {
if (inventory === undefined) return [];
const broken = inventory.candidates.filter(({ healthy }) => !healthy);
const detail =
broken.length === 0
? `${String(inventory.candidates.length)} runtime executable candidates passed bounded version probes.`
: `${String(broken.length)} of ${String(inventory.candidates.length)} runtime executable candidates failed bounded version probes: ${broken
.map(({ lexical_path: path }) => path)
.join(", ")}`;
return [
check("node-toolchains", broken.length === 0, detail, {
remediation:
"Select a verified healthy Node toolchain or repair the failing installation, then rerun rea doctor. Do not create compatibility-library symlinks.",
classification: "missing_dependency",
}),
];
};
const hostCheck = async (host: DoctorHost): Promise<DoctorCheck> => {
const macosVersion =
host.platform === "darwin" ? await host.macosVersion() : undefined;
src/application/DoctorScope.ts
+1 -2
View File
@@ -7,7 +7,7 @@ import type {
DoctorProviderInspection,
DoctorScope,
} from "./Doctor.js";
import { doctorHealthy, isOptionalDoctorCheck } from "./DoctorDiagnostics.js";
import { doctorHealthy } from "./DoctorDiagnostics.js";
/** Normalized readiness boundary included in every doctor response. */
export interface DoctorScopeReport {
@@ -58,7 +58,6 @@ export const scopeDoctorChecks = (
const coreChecks = input.checks.filter(
({ name }) =>
!isProviderCheck(name, input.providers.providerInspections) &&
!isOptionalDoctorCheck(name) &&
name !== "skill:identity" &&
!name.startsWith("registration:"),
);
src/application/ElectronActiveEvidence.ts
+2 -2
View File
@@ -6,7 +6,7 @@ import type {
ElectronActiveObservationResult,
} from "../domain/electronActiveObservation.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import { digestJson } from "./JavaScriptReplayPlanning.js";
import { digestCanonicalJson } from "../domain/browserScenarioNormalization.js";
type CanonicalElectronActiveObservationInput =
ElectronActiveObservationInput & {
@@ -49,5 +49,5 @@ const parameters = (
input: CanonicalElectronActiveObservationInput,
): EvidenceObservation["parameters"] => ({
...scenarioProjection(input),
scenario_sha256: digestJson(scenarioProjection(input)),
scenario_sha256: digestCanonicalJson(scenarioProjection(input)),
});
src/application/EnhancedLiteralTracing.test.ts
-24
View File
@@ -16,9 +16,7 @@ describe("literal feature tracing", () => {
throw new Error(`Unexpected analysis call: ${name}`);
},
{ query: "launch feature", case_sensitive: false },
"feature",
);
expect(result).toMatchObject({
ok: true,
value: {
@@ -30,26 +28,4 @@ describe("literal feature tracing", () => {
},
});
});
it("keeps string-code tracing scoped to string labels", async () => {
const result = await traceLiteralFeature(
async (name) => {
if (name === "search_strings")
return ok([{ address: "0x1000", value: "launch feature" }]);
if (name === "xrefs") return ok([]);
throw new Error(`Unexpected analysis call: ${name}`);
},
{ query: "launch", case_sensitive: false },
"string",
);
expect(result).toMatchObject({
ok: true,
value: {
matches: [
{ type: "string", address: "0x1000", value: "launch feature" },
],
references: [],
},
});
});
});
src/application/EnhancedLiteralTracing.ts
+8 -9
View File
@@ -25,29 +25,28 @@ interface ReferenceTraceRequest {
readonly signal?: AbortSignal;
}
const FEATURE_SEARCH_SOURCES: readonly LiteralSearchSource[] = [
const LITERAL_SEARCH_SOURCES: readonly LiteralSearchSource[] = [
["search_strings", "string"],
["search_procedures", "procedure"],
];
const STRING_SEARCH_SOURCES: readonly LiteralSearchSource[] = [
["search_strings", "string"],
];
export interface LiteralTraceInput {
readonly query: string;
readonly case_sensitive: boolean;
}
/** Trace every matching feature or string literal through provider xrefs. */
/** Trace matching strings and procedures through provider xrefs. */
export const traceLiteralFeature = async (
call: AnalysisCall,
input: LiteralTraceInput,
scope: "feature" | "string",
signal?: AbortSignal,
): EnhancedResult => {
const sources =
scope === "feature" ? FEATURE_SEARCH_SOURCES : STRING_SEARCH_SOURCES;
const searched = await literalMatches(call, input, sources, signal);
const searched = await literalMatches(
call,
input,
LITERAL_SEARCH_SOURCES,
signal,
);
if (!searched.ok) return searched;
const traced = await traceReferences(call, {
matches: searched.value.matches,
src/application/EnhancedTools.ts
+3 -21
View File
@@ -56,11 +56,7 @@ export class EnhancedTools {
input: unknown,
signal?: AbortSignal,
): EnhancedResult {
if (
name === "trace_feature" ||
name === "find_code_for_string" ||
name === "trace_call_path"
)
if (name === "trace_feature" || name === "trace_call_path")
return this.#executeTracing(name, input, signal);
if (name === "trace_native_values") {
const parsed = enhancedInputSchemas.trace_native_values.safeParse(input);
@@ -163,15 +159,6 @@ export class EnhancedTools {
(name, arguments_, operationSignal) =>
this.#call(name, arguments_, operationSignal),
call.input,
"feature",
signal,
);
case "find_code_for_string":
return traceLiteralFeature(
(name, arguments_, operationSignal) =>
this.#call(name, arguments_, operationSignal),
call.input,
"string",
signal,
);
case "trace_call_path":
@@ -190,7 +177,7 @@ export class EnhancedTools {
}
#executeTracing(
name: "trace_feature" | "find_code_for_string" | "trace_call_path",
name: "trace_feature" | "trace_call_path",
input: unknown,
signal?: AbortSignal,
): EnhancedResult {
@@ -199,12 +186,7 @@ export class EnhancedTools {
if (!parsed.success) return invalidEnhancedInput(name, parsed.error);
return this.executeValidated({ name, input: parsed.data }, signal);
}
if (name === "trace_feature") {
const parsed = enhancedInputSchemas.trace_feature.safeParse(input);
if (!parsed.success) return invalidEnhancedInput(name, parsed.error);
return this.executeValidated({ name, input: parsed.data }, signal);
}
const parsed = enhancedInputSchemas.find_code_for_string.safeParse(input);
const parsed = enhancedInputSchemas.trace_feature.safeParse(input);
if (!parsed.success) return invalidEnhancedInput(name, parsed.error);
return this.executeValidated({ name, input: parsed.data }, signal);
}
src/application/InstrumentedJavaScriptReplayHost.ts
-63
View File
@@ -1,63 +0,0 @@
import {
instrumentJavaScriptExport,
type JavaScriptExportInstrumentation,
} from "../domain/javascriptExportInstrumentation.js";
import type { NodeCharacterizationPreparationInput } from "../domain/nodeRuntimeCharacterization.js";
import type {
JavaScriptReplayHost,
ReplayExecutableIdentity,
ReplayRuntimeFileIdentity,
ReplaySourceBytes,
} from "./JavaScriptReplayPlanning.js";
/** Decorate one exact replay source with deterministic in-memory instrumentation. */
export class InstrumentedJavaScriptReplayHost implements JavaScriptReplayHost {
instrumentation: JavaScriptExportInstrumentation | null = null;
constructor(
private readonly delegate: JavaScriptReplayHost,
private readonly input: NodeCharacterizationPreparationInput["instrumentation"],
) {}
async readSource(
path: string,
maximumBytes: number,
): Promise<ReplaySourceBytes> {
const source = await this.delegate.readSource(path, maximumBytes);
if (path !== this.input.artifact_path) return source;
const instrumented = instrumentJavaScriptExport(source.bytes, this.input);
if (instrumented.bytes.byteLength > maximumBytes)
throw new RangeError(
"Instrumented JavaScript exceeds the replay module byte limit",
);
this.instrumentation = instrumented;
return { canonicalPath: source.canonicalPath, bytes: instrumented.bytes };
}
identifyExecutable(
path: string,
versionArguments: readonly string[],
): Promise<ReplayExecutableIdentity> {
return this.delegate.identifyExecutable(path, versionArguments);
}
identifyWorker(): Promise<ReplayExecutableIdentity> {
return this.delegate.identifyWorker();
}
identifyRuntimeClosure(
nodePath: string,
): Promise<readonly ReplayRuntimeFileIdentity[]> {
return this.delegate.identifyRuntimeClosure(nodePath);
}
seccompDigest(): string {
return this.delegate.seccompDigest();
}
probe(
configuration: Parameters<JavaScriptReplayHost["probe"]>[0],
): Promise<void> {
return this.delegate.probe(configuration);
}
}
src/application/JavaScriptReplayPlanning.test.ts
-107
View File
@@ -1,107 +0,0 @@
import { describe, expect, it, vi } from "vitest";
import {
prepareReplayPlan,
type JavaScriptReplayConfiguration,
type JavaScriptReplayHost,
} from "./JavaScriptReplayPlanning.js";
import { controlledReplayInputSchema } from "../domain/javascriptReplay.js";
const root = "/replay-fixture";
const configuration: JavaScriptReplayConfiguration = {
nodePath: "/runtime/node",
bubblewrapPath: "/runtime/bwrap",
systemdRunPath: "/runtime/systemd-run",
systemctlPath: "/runtime/systemctl",
shellPath: "/runtime/sh",
};
const inputWithExpandedManifest = () => {
const modules = Array.from({ length: 130 }, (_, index) => ({
alias: `module-${index}`,
path: `${root}/module.mjs`,
format: "esm" as const,
role: "module" as const,
dependencies: {},
}));
const first = modules[0];
if (first === undefined) throw new TypeError("Missing fixture module");
modules[0] = {
...first,
dependencies: Object.fromEntries(
Array.from({ length: 129 }, (_, index) => [
`dependency-${index}`,
`module-${index + 1}`,
]),
),
};
return controlledReplayInputSchema.parse({
mode: "plan",
left: {
modules,
entry_alias: "module-0",
entry_export: "default",
},
cases: Array.from({ length: 129 }, (_, index) => ({
case_id: `case-${index}`,
arguments: Array.from({ length: 17 }, () => "value"),
})),
});
};
describe("controlled replay planning input bounds", () => {
it("accepts module, dependency, case, and argument counts beyond old quotas", () => {
const input = inputWithExpandedManifest();
expect(input.left.modules).toHaveLength(130);
expect(Object.keys(input.left.modules[0]?.dependencies ?? {})).toHaveLength(
129,
);
expect(input.cases).toHaveLength(129);
expect(input.cases[0]?.arguments).toHaveLength(17);
});
it("rejects an oversized protocol manifest before probing or reading", async () => {
const probe = vi.fn(async () => undefined);
const readSource = vi.fn(async () => ({
canonicalPath: `${root}/module.mjs`,
bytes: new Uint8Array(),
}));
const host: JavaScriptReplayHost = {
readSource,
identifyExecutable: vi.fn(async (path) => ({
path,
version: "1",
sha256: "0".repeat(64),
})),
identifyWorker: vi.fn(async () => ({
path: "/worker.js",
version: "1",
sha256: "0".repeat(64),
})),
identifyRuntimeClosure: vi.fn(async () => []),
seccompDigest: () => "0".repeat(64),
probe,
};
const input = controlledReplayInputSchema.parse({
mode: "plan",
left: {
modules: Array.from({ length: 300 }, (_, index) => ({
alias: `module-${index}`,
path: `/${"p".repeat(4_095)}`,
format: "esm",
dependencies: {},
})),
entry_alias: "module-0",
},
cases: [{ case_id: "empty", arguments: [] }],
limits: { protocol_bytes: 1024 * 1024 },
});
await expect(prepareReplayPlan(input, configuration, host)).rejects.toThrow(
"Replay plan input exceeds its protocol byte limit",
);
expect(probe).not.toHaveBeenCalled();
expect(readSource).not.toHaveBeenCalled();
});
});
src/application/JavaScriptReplayPlanning.ts
-325
View File
@@ -1,325 +0,0 @@
import canonicalize from "canonicalize";
import { createHash } from "node:crypto";
import type { JsonValue } from "../domain/jsonValue.js";
import {
createSeededRandomUint,
NON_DEGENERATE_XORSHIFT32_SEED,
} from "../replay/seededRandom.js";
import type {
ControlledReplayInput,
ReplayExecutionResult,
ReplayPlan,
} from "../domain/javascriptReplay.js";
const REPLAY_POLICY_VERSION = "linux-bwrap-systemd-v1" as const;
const REPLAY_POLICY_DOCUMENT = Object.freeze({
version: REPLAY_POLICY_VERSION,
namespaces: ["user", "pid", "network", "ipc", "uts", "cgroup"],
capabilities: "drop-all",
network: "none",
filesystem: "minimal-read-only-runtime-plus-private-tmpfs",
seccomp: "rea-linux-x86_64-v1",
resource_manager: "systemd-user-cgroup-v2",
nested_user_namespaces: false,
});
export interface ReplaySourceBytes {
readonly canonicalPath: string;
readonly bytes: Uint8Array;
}
export interface ReplayExecutableIdentity {
readonly path: string;
readonly version: string;
readonly sha256: string;
}
export interface ReplayRuntimeFileIdentity {
readonly sourcePath: string;
readonly destinationPath: string;
readonly sha256: string;
}
/** Executable paths required to establish isolated Linux JavaScript replay. */
export interface JavaScriptReplayConfiguration {
readonly nodePath: string;
readonly bubblewrapPath: string;
readonly systemdRunPath: string;
readonly systemctlPath: string;
readonly shellPath: string;
}
export interface JavaScriptReplayHost {
readonly readSource: (
path: string,
maximumBytes: number,
) => Promise<ReplaySourceBytes>;
readonly identifyExecutable: (
path: string,
versionArguments: readonly string[],
) => Promise<ReplayExecutableIdentity>;
readonly identifyWorker: () => Promise<ReplayExecutableIdentity>;
readonly identifyRuntimeClosure: (
nodePath: string,
) => Promise<readonly ReplayRuntimeFileIdentity[]>;
readonly seccompDigest: () => string;
readonly probe: (
configuration: JavaScriptReplayConfiguration,
) => Promise<void>;
}
export interface PreparedReplayPlan {
readonly publicPlan: ReplayPlan;
readonly leftSources: Readonly<Record<string, string>>;
readonly rightSources?: Readonly<Record<string, string>>;
}
export interface JavaScriptReplayRunner {
readonly execute: (
prepared: PreparedReplayPlan,
configuration: JavaScriptReplayConfiguration,
signal?: AbortSignal,
) => Promise<ReplayExecutionResult>;
}
const assertReplayManifestWithinProtocolLimit = (
input: ControlledReplayInput,
): void => {
if (
new TextEncoder().encode(JSON.stringify(input)).byteLength >
input.limits.protocol_bytes
)
throw new RangeError("Replay plan input exceeds its protocol byte limit");
};
/** Read, normalize, and content-address every input to one replay experiment. */
export const prepareReplayPlan = async (
input: ControlledReplayInput,
configuration: JavaScriptReplayConfiguration,
host: JavaScriptReplayHost,
): Promise<PreparedReplayPlan> => {
assertReplayManifestWithinProtocolLimit(input);
const cases = replayCases(input).map((item) => ({
...item,
sha256: digestJson(item.arguments),
}));
if (
new TextEncoder().encode(
JSON.stringify(cases.map(({ arguments: args }) => args)),
).byteLength > input.limits.input_bytes
)
throw new RangeError("Replay case inputs exceed the aggregate byte limit");
await host.probe(configuration);
const [
runtime,
bubblewrap,
systemdRun,
systemctl,
shell,
worker,
runtimeFiles,
left,
right,
] = await Promise.all([
host.identifyExecutable(configuration.nodePath, ["--version"]),
host.identifyExecutable(configuration.bubblewrapPath, ["--version"]),
host.identifyExecutable(configuration.systemdRunPath, ["--version"]),
host.identifyExecutable(configuration.systemctlPath, ["--version"]),
host.identifyExecutable(configuration.shellPath, ["--version"]),
host.identifyWorker(),
host.identifyRuntimeClosure(configuration.nodePath),
prepareSide(input.left, input.limits.module_bytes, host),
input.right === undefined
? Promise.resolve(undefined)
: prepareSide(input.right, input.limits.module_bytes, host),
]);
const withoutDigest = {
policy_version: REPLAY_POLICY_VERSION,
policy_sha256: digestJson(REPLAY_POLICY_DOCUMENT),
network: "none" as const,
filesystem: {
host_writes: false as const,
private_tmpfs_bytes: input.limits.tmpfs_bytes,
},
runtime: {
executable: runtime,
worker,
read_only_files: runtimeFiles.map((item) => ({
source_path: item.sourcePath,
destination_path: item.destinationPath,
sha256: item.sha256,
})),
},
sandbox: {
bubblewrap,
systemd_run: systemdRun,
systemctl,
shell,
seccomp_sha256: host.seccompDigest(),
},
left: left.commitment,
...(right === undefined ? {} : { right: right.commitment }),
cases,
determinism: input.determinism,
limits: input.limits,
effects: [
"Execute selected extracted modules in a disposable Linux OS sandbox",
"Create and remove one private cgroup and mount namespace",
"Retain bounded return, exception, termination, and diagnostic evidence",
"Perform no host filesystem writes and expose no host or external network",
...(input.reproducer_export === undefined
? []
: ["Write one reproducer after sandbox cleanup"]),
],
reproducer_export:
input.reproducer_export === undefined
? null
: {
path: input.reproducer_export.path,
include_sources: input.reproducer_export.include_sources,
},
};
const publicPlan: ReplayPlan = {
...withoutDigest,
plan_digest: digestJson(withoutDigest),
};
return {
publicPlan,
leftSources: left.sources,
...(right === undefined ? {} : { rightSources: right.sources }),
};
};
const prepareSide = async (
side: ControlledReplayInput["left"],
maximumBytes: number,
host: JavaScriptReplayHost,
) => {
const aliases = new Set(side.modules.map(({ alias }) => alias));
if (aliases.size !== side.modules.length || !aliases.has(side.entry_alias))
throw new TypeError("Replay manifest aliases or entry alias are invalid");
for (const module of side.modules)
for (const dependency of Object.values(module.dependencies))
if (!aliases.has(dependency))
throw new TypeError(
`Undeclared replay dependency alias: ${dependency}`,
);
const loaded: {
readonly module: (typeof side.modules)[number];
readonly source: ReplaySourceBytes;
}[] = [];
let totalBytes = 0;
for (const module of side.modules) {
const remaining = maximumBytes - totalBytes;
if (remaining <= 0)
throw new RangeError("Replay module bytes exceed the aggregate limit");
const source = await host.readSource(module.path, remaining);
if (source.bytes.byteLength > remaining)
throw new RangeError("Replay module bytes exceed the aggregate limit");
totalBytes += source.bytes.byteLength;
loaded.push({ module, source });
}
return {
commitment: {
modules: loaded.map(({ module, source }) => ({
alias: module.alias,
canonical_path: source.canonicalPath,
format: module.format,
role: module.role,
byte_count: source.bytes.byteLength,
sha256: digestBytes(source.bytes),
dependencies: module.dependencies,
})),
entry_alias: side.entry_alias,
entry_export: side.entry_export,
},
sources: Object.fromEntries(
loaded.map(({ module, source }) => [
module.alias,
new TextDecoder("utf-8", { fatal: true }).decode(source.bytes),
]),
),
};
};
const replayCases = (
input: ControlledReplayInput,
): readonly { readonly case_id: string; readonly arguments: JsonValue[] }[] => {
const explicit = input.cases.map((item) => ({
case_id: item.case_id,
arguments: item.arguments,
}));
if (input.generator === undefined) return uniqueCases(explicit);
const next = createSeededRandomUint(
input.generator.seed || NON_DEGENERATE_XORSHIFT32_SEED,
);
const corpus = boundaryCorpus(input.generator.preset);
const generated: {
readonly case_id: string;
readonly arguments: JsonValue[];
}[] = Array.from({ length: input.generator.count }, (_, index) => ({
case_id: `generated-${input.generator?.preset ?? "boundary"}-${String(index)}`,
arguments: [corpus[next() % corpus.length] ?? ""],
}));
return uniqueCases([...explicit, ...generated]);
};
const uniqueCases = <Case extends { readonly case_id: string }>(
cases: readonly Case[],
): readonly Case[] => {
const ids = new Set<string>();
for (const item of cases) {
if (ids.has(item.case_id))
throw new TypeError(`Duplicate replay case ID: ${item.case_id}`);
ids.add(item.case_id);
}
return cases;
};
const boundaryCorpus = (
preset: NonNullable<ControlledReplayInput["generator"]>["preset"],
): readonly JsonValue[] => {
if (preset === "sanitizer-boundaries")
return [
"",
"<script>alert(1)</script>",
"<img src=x onerror=alert(1)>",
"javascript:alert(1)",
"<b>ok</b>",
"\u0000",
];
if (preset === "clipboard-boundaries")
return [
"",
"plain",
"a\r\nb",
"a\nb",
"\t",
"😀",
"text/html",
"text/plain",
];
return [
"",
" ",
"\n",
"# heading",
"**bold**",
"[x](javascript:y)",
"`code`",
"😀",
"\u0000",
"a".repeat(1024),
];
};
export const digestBytes = (value: Uint8Array): string =>
createHash("sha256").update(value).digest("hex");
export const digestJson = (value: unknown): string => {
const encoded = canonicalize(value);
if (encoded === undefined)
throw new TypeError("Replay commitment is not canonical JSON");
return createHash("sha256").update(encoded).digest("hex");
};
src/application/JavaScriptReplayService.ts
-430
View File
@@ -1,430 +0,0 @@
import { z } from "zod";
import writeFileAtomic from "write-file-atomic";
import {
controlledReplayInputSchema,
replayExecutionResultSchema,
controlledReplayOutputSchema,
replayEvidenceSchema,
type ControlledReplayExecutionInput,
type ControlledReplayInput,
} from "../domain/javascriptReplay.js";
import {
AnalysisCapabilityUnavailableError,
AnalysisCancelledError,
AnalysisInputError,
AnalysisProtocolError,
ReplayPlanStaleError,
type AnalysisError,
} from "../domain/errors.js";
import { createEvidence } from "../domain/evidence.js";
import { projectInputIssues } from "../domain/inputIssueProjection.js";
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import type { ExecutionOptions } from "./AnalysisProvider.js";
import {
digestBytes,
prepareReplayPlan,
type JavaScriptReplayConfiguration,
type JavaScriptReplayHost,
type JavaScriptReplayRunner,
} from "./JavaScriptReplayPlanning.js";
const OPERATION = "run_controlled_replay" as const;
const isAborted = (signal: AbortSignal | undefined): boolean =>
signal?.aborted === true;
export interface JavaScriptReplayDependencies {
readonly configuration: () => JavaScriptReplayConfiguration;
readonly host: JavaScriptReplayHost;
readonly runner: JavaScriptReplayRunner;
}
/** Plan or execute one content-bound extracted-module replay experiment. */
export const runControlledReplay = async (
dependencies: JavaScriptReplayDependencies,
rawInput: unknown,
options: ExecutionOptions = {},
): Promise<Result<JsonValue, AnalysisError>> => {
const parsed = controlledReplayInputSchema.safeParse(rawInput);
if (!parsed.success)
return err(
new AnalysisInputError(
OPERATION,
{ cause: parsed.error },
projectInputIssues(parsed.error.issues, rawInput),
),
);
return runControlledReplayValidated(dependencies, parsed.data, options);
};
/** Plan or execute replay input already parsed by a trusted adapter. */
export const runControlledReplayValidated = async (
dependencies: JavaScriptReplayDependencies,
input: ControlledReplayInput,
options: ExecutionOptions = {},
): Promise<Result<JsonValue, AnalysisError>> => {
if (isAborted(options.signal))
return err(new AnalysisCancelledError(OPERATION));
const configuration = dependencies.configuration();
const prepared = await prepareValidatedReplay(
dependencies,
configuration,
input,
options.signal,
);
if (!prepared.ok) return prepared;
if (isAborted(options.signal))
return err(new AnalysisCancelledError(OPERATION));
if (input.mode === "plan")
return ok(
asJson({
phase: "plan",
plan: prepared.value.publicPlan,
source_evidence: [],
evidence: null,
}),
);
if (input.plan_digest !== prepared.value.publicPlan.plan_digest)
return err(
new ReplayPlanStaleError(
input.plan_digest,
prepared.value.publicPlan.plan_digest,
),
);
const exportContext = prepareReproducerExport(input);
if (!exportContext.ok) return exportContext;
const executed = await executeValidatedReplay({
runner: dependencies.runner,
configuration,
prepared: prepared.value,
exportContext: exportContext.value,
options,
});
if (!executed.ok) return executed;
try {
return ok(
buildReplayOutput(
prepared.value,
executed.value.executed,
executed.value.sourceEvidence,
),
);
} catch (cause: unknown) {
return err(
new AnalysisProtocolError(
cause instanceof z.ZodError
? "Controlled replay produced an invalid bounded result"
: "Controlled replay could not establish or clean up its sandbox",
{ cause },
),
);
}
};
const prepareValidatedReplay = async (
dependencies: JavaScriptReplayDependencies,
configuration: JavaScriptReplayConfiguration,
input: ControlledReplayInput,
signal: AbortSignal | undefined,
): Promise<
Result<Awaited<ReturnType<typeof prepareReplayPlan>>, AnalysisError>
> => {
if (isAborted(signal)) return err(new AnalysisCancelledError(OPERATION));
try {
return ok(await prepareReplayPlan(input, configuration, dependencies.host));
} catch (cause: unknown) {
return err(
new AnalysisCapabilityUnavailableError(
"rea-javascript-replay",
OPERATION,
cause instanceof Error ? cause.message : "replay planning failed",
),
);
}
};
interface ReproducerExportContext {
readonly path: string;
readonly includeSources: boolean;
}
const prepareReproducerExport = (
input: ControlledReplayExecutionInput,
): Result<ReproducerExportContext | undefined, AnalysisError> => {
if (input.reproducer_export === undefined) return ok(undefined);
return ok({
path: input.reproducer_export.path,
includeSources: input.reproducer_export.include_sources === true,
});
};
interface ReplayExecutionContext {
readonly runner: JavaScriptReplayRunner;
readonly configuration: JavaScriptReplayConfiguration;
readonly prepared: Awaited<ReturnType<typeof prepareReplayPlan>>;
readonly exportContext: ReproducerExportContext | undefined;
readonly options: ExecutionOptions;
}
const executeValidatedReplay = async (
context: ReplayExecutionContext,
): Promise<
Result<
{
readonly executed: z.infer<typeof replayExecutionResultSchema>;
readonly sourceEvidence: ReturnType<typeof createReplayEvidence>[];
},
AnalysisError
>
> => {
const { runner, configuration, prepared, exportContext, options } = context;
await options.progress?.report({
phase: OPERATION,
completed: 0,
total: 1,
message: "Starting the isolated replay worker",
});
if (isAborted(options.signal))
return err(new AnalysisCancelledError(OPERATION));
try {
let executed = replayExecutionResultSchema.parse(
await runner.execute(prepared, configuration, options.signal),
);
await options.progress?.report({
phase: OPERATION,
completed: 1,
total: 1,
message: "Controlled replay stopped and sandbox cleanup was observed",
terminal: true,
});
if (executed.termination === "cancelled")
return err(new AnalysisCancelledError(OPERATION));
executed = await applyReproducerExport(executed, exportContext, prepared);
const sourceEvidence = buildSourceEvidence(prepared, executed);
return ok({ executed, sourceEvidence });
} catch (cause: unknown) {
return err(
new AnalysisProtocolError(
cause instanceof z.ZodError
? "Controlled replay produced an invalid bounded result"
: "Controlled replay could not establish or clean up its sandbox",
{ cause },
),
);
}
};
const applyReproducerExport = async (
executed: z.infer<typeof replayExecutionResultSchema>,
exportContext: ReproducerExportContext | undefined,
prepared: Awaited<ReturnType<typeof prepareReplayPlan>>,
): Promise<z.infer<typeof replayExecutionResultSchema>> => {
if (exportContext === undefined || executed.cleanup.state !== "complete")
return executed;
const manifest = {
plan: prepared.publicPlan,
result: executed,
sources: exportContext.includeSources
? {
left: prepared.leftSources,
...(prepared.rightSources === undefined
? {}
: { right: prepared.rightSources }),
}
: null,
};
const encoded = Buffer.from(`${JSON.stringify(manifest, null, 2)}\n`);
try {
await writeFileAtomic(exportContext.path, encoded, {
mode: 0o600,
});
return replayExecutionResultSchema.parse({
...executed,
reproducer: {
state: "written",
path: exportContext.path,
sha256: digestBytes(encoded),
},
});
} catch (cause: unknown) {
return replayExecutionResultSchema.parse({
...executed,
limitations: [
...executed.limitations,
"Replay completed, but the requested reproducer export failed.",
],
reproducer: {
state: "failed",
path: exportContext.path,
error:
cause instanceof Error
? cause.message
: "unknown reproducer export failure",
},
});
}
};
const buildSourceEvidence = (
prepared: Awaited<ReturnType<typeof prepareReplayPlan>>,
executed: z.infer<typeof replayExecutionResultSchema>,
): ReturnType<typeof createReplayEvidence>[] => {
const leftCount = prepared.publicPlan.cases.length;
return [
createReplayEvidence(
prepared,
executed,
"left",
executed.outcomes.slice(0, leftCount),
),
...(prepared.publicPlan.right === undefined
? []
: [
createReplayEvidence(
prepared,
executed,
"right",
executed.outcomes.slice(leftCount),
),
]),
];
};
const buildReplayOutput = (
prepared: Awaited<ReturnType<typeof prepareReplayPlan>>,
executed: z.infer<typeof replayExecutionResultSchema>,
sourceEvidence: ReturnType<typeof createReplayEvidence>[],
): JsonValue => {
const subject = prepared.publicPlan.left.modules[0];
const evidence = createEvidence(
subject === undefined
? undefined
: {
path: subject.canonical_path,
sha256: subject.sha256,
format: "javascript",
},
{
id: "rea-javascript-replay",
name: "REA isolated JavaScript replay",
version: "1",
},
{
predicateType: "javascript-controlled-replay",
operation: OPERATION,
parameters: {
replay_plan: jsonValueSchema.parse(
JSON.parse(JSON.stringify(prepared.publicPlan)),
),
},
result: jsonValueSchema.parse(JSON.parse(JSON.stringify(executed))),
rawResult: jsonValueSchema.parse(
JSON.parse(
JSON.stringify({
cases: prepared.publicPlan.cases,
outcomes: executed.outcomes,
stderr: executed.stderr,
}),
),
),
confidence: executed.comparison === undefined ? "observed" : "derived",
authority: "controlled-replay",
environment: {
id: prepared.publicPlan.plan_digest,
platform: process.platform,
architecture: process.arch,
isolation: "container",
},
limitations: executed.limitations,
locations: prepared.publicPlan.left.modules.map((module) => ({
kind: "artifact-path" as const,
path: module.canonical_path,
})),
evidenceLinks: sourceEvidence.map(({ evidence_id: id }) => id),
},
);
return asJson({
phase: "execute",
plan: null,
source_evidence: sourceEvidence.map((item) =>
replayEvidenceSchema.parse(item),
),
evidence: replayEvidenceSchema.parse(evidence),
});
};
const createReplayEvidence = (
prepared: Awaited<ReturnType<typeof prepareReplayPlan>>,
executed: z.infer<typeof replayExecutionResultSchema>,
side: "left" | "right",
outcomes: z.infer<typeof replayExecutionResultSchema>["outcomes"],
) => {
const manifest =
side === "left" ? prepared.publicPlan.left : prepared.publicPlan.right;
const subject = manifest?.modules[0];
const result = replayExecutionResultSchema.parse({
...executed,
outcomes,
comparison: undefined,
reproducer: null,
});
return createEvidence(
subject === undefined
? undefined
: {
path: subject.canonical_path,
sha256: subject.sha256,
format: "javascript",
},
{
id: "rea-javascript-replay",
name: "REA isolated JavaScript replay",
version: "1",
},
{
predicateType: "javascript-controlled-replay-observation",
operation: OPERATION,
parameters: {
replay_plan: jsonValueSchema.parse(
JSON.parse(JSON.stringify(prepared.publicPlan)),
),
side,
},
result: jsonValueSchema.parse(JSON.parse(JSON.stringify(result))),
rawResult: jsonValueSchema.parse(
JSON.parse(
JSON.stringify({ cases: prepared.publicPlan.cases, outcomes }),
),
),
confidence: "observed",
authority: "controlled-replay",
environment: {
id: prepared.publicPlan.plan_digest,
platform: process.platform,
architecture: process.arch,
isolation: "container",
},
limitations: executed.limitations,
locations: (manifest?.modules ?? []).map((module) => ({
kind: "artifact-path" as const,
path: module.canonical_path,
})),
},
);
};
const asJson = (value: unknown): JsonValue => {
const serialized: unknown = JSON.parse(
JSON.stringify(controlledReplayOutputSchema.parse(value)),
);
return jsonValueSchema.parse(serialized);
};
src/application/LoopbackReplay.ts
-531
View File
@@ -1,531 +0,0 @@
import {
createServer,
type IncomingMessage,
type Server,
type ServerResponse,
} from "node:http";
import { WebSocketServer, type WebSocket } from "ws";
import type {
ProcessScenario,
ProtocolEvent,
RecordProcessCaptureEvent,
} from "../domain/processCapture.js";
import {
type ReplayMachineDecision,
type ReplayTransitionRecord,
} from "../domain/replayMachineRuntime.js";
import {
closeReplayServer,
listenOnLoopback,
recordMachineEvent,
readReplayRequestBody,
ReplayRecorder,
ReplayDelayScheduler,
requestHeaders,
} from "./LoopbackReplayRecorder.js";
/** Owned loopback replay endpoints and their bounded protocol observations. */
export interface LoopbackReplay {
readonly httpUrl: string;
readonly websocketUrl: string;
readonly events: ProtocolEvent[];
readonly transitions: readonly ReplayTransitionRecord[];
readonly truncated: boolean;
close(): Promise<void>;
}
interface HttpReplayRequest {
readonly request: IncomingMessage;
readonly response: ServerResponse;
readonly method: string;
readonly path: string;
readonly body: string;
readonly rawAtMs: number;
readonly recordedAtMs: number;
}
const recordHttpResponse = (
recorder: ReplayRecorder,
request: Pick<HttpReplayRequest, "method" | "path">,
data: string,
outcome: ProtocolEvent["outcome"],
): void =>
recorder.record({
at_ms: recorder.atMs(),
protocol: "http",
direction: "response",
method: request.method,
path: request.path,
data,
outcome,
});
const handleMachineHttp = async (
recorder: ReplayRecorder,
request: HttpReplayRequest,
delays: ReplayDelayScheduler,
): Promise<void> => {
const machine = recorder.machine;
if (machine === undefined) return;
const decision = recorder.dispatchMachine({
protocol: "http",
connection: "not_applicable",
at_ms: request.rawAtMs,
recorded_at_ms: request.recordedAtMs,
method: request.method,
path: request.path,
headers: requestHeaders(request.request),
body: request.body,
});
recordMachineEvent(
recorder,
{
at_ms: request.recordedAtMs,
protocol: "http",
direction: "request",
method: request.method,
path: request.path,
data: request.body,
},
decision,
);
if (decision.outcome !== "matched") {
request.response.statusCode = decision.outcome === "unmatched" ? 404 : 409;
request.response.end();
return;
}
for (const action of decision.actions) {
if (action.type === "delay") await delays.wait(action.duration_ms);
else if (action.type === "disconnect") {
request.response.destroy();
recordHttpResponse(recorder, request, "", "disconnected");
return;
} else if (action.type === "http_response") {
request.response.statusCode = action.status;
for (const [name, value] of Object.entries(action.headers))
request.response.setHeader(name, value);
request.response.end(action.body);
recordHttpResponse(
recorder,
request,
machine.redact(action.body),
"matched",
);
}
}
};
const handleStaticHttp = async (
recorder: ReplayRecorder,
request: HttpReplayRequest,
calls: Map<number, number>,
delays: ReplayDelayScheduler,
): Promise<void> => {
const index = recorder.scenario.replay.http.findIndex((route) =>
route.method !== request.method || route.path !== request.path
? false
: (route.request_body === undefined ||
route.request_body === request.body) &&
Object.entries(route.request_headers).every(
([name, value]) =>
request.request.headers[name.toLowerCase()] === value,
),
);
const route = recorder.scenario.replay.http[index];
const used = index < 0 ? 0 : (calls.get(index) ?? 0);
const outcome =
route === undefined
? "unmatched"
: used >= route.max_calls
? "script_exhausted"
: "matched";
recorder.record({
at_ms: recorder.atMs(),
protocol: "http",
direction: "request",
method: request.method,
path: request.path,
data: request.body.length === 0 ? "" : "<redacted-request-body>",
outcome,
});
if (route === undefined || outcome === "script_exhausted") {
request.response.statusCode = route === undefined ? 404 : 409;
request.response.end();
return;
}
calls.set(index, used + 1);
if (route.delay_ms > 0) await delays.wait(route.delay_ms);
if (route.disconnect) {
request.response.destroy();
recordHttpResponse(recorder, request, "", "disconnected");
return;
}
request.response.statusCode = route.status;
for (const [name, value] of Object.entries(route.response_headers))
request.response.setHeader(name, value);
request.response.end(route.body);
recordHttpResponse(recorder, request, route.body, "matched");
};
const handleHttpRequest = async (
recorder: ReplayRecorder,
request: IncomingMessage,
response: ServerResponse,
calls: Map<number, number>,
delays: ReplayDelayScheduler,
): Promise<void> => {
const body = await readReplayRequestBody(
request,
recorder.scenario.limits.protocol_body_bytes,
);
const method = request.method ?? "GET";
const requestUrl = new URL(request.url ?? "/", "http://127.0.0.1");
const path = `${requestUrl.pathname}${requestUrl.search}`;
if (body === undefined) {
recorder.truncated = true;
response.statusCode = 413;
response.end();
recorder.record({
at_ms: recorder.atMs(),
protocol: "http",
direction: "request",
method,
path,
data: "<body-over-limit>",
outcome: "unmatched",
});
return;
}
const replayRequest = {
request,
response,
method,
path,
body,
rawAtMs: recorder.rawAtMs(),
recordedAtMs: recorder.atMs(),
};
await (recorder.machine === undefined
? handleStaticHttp(recorder, replayRequest, calls, delays)
: recorder.enqueueMachine(() =>
handleMachineHttp(recorder, replayRequest, delays),
));
};
const createHttpHandler = (
recorder: ReplayRecorder,
delays: ReplayDelayScheduler,
): ((request: IncomingMessage, response: ServerResponse) => void) => {
const calls = new Map<number, number>();
return (request, response) => {
void handleHttpRequest(recorder, request, response, calls, delays).catch(
() => {
recorder.truncated = true;
if (response.headersSent) response.destroy();
else {
response.statusCode = 500;
response.end();
}
},
);
};
};
const sendWebSocketScript = async (
client: WebSocket,
connection: number,
recorder: ReplayRecorder,
delays: ReplayDelayScheduler,
): Promise<void> => {
const scripts = recorder.scenario.replay.websocket_connections;
const script = scripts[connection - 1];
if (scripts.length > 0 && script === undefined) {
recorder.record({
at_ms: recorder.atMs(),
protocol: "websocket",
direction: "sent",
method: null,
path: "/ws",
data: "",
outcome: "script_exhausted",
});
client.terminate();
return;
}
const messages =
script?.messages ??
recorder.scenario.replay.websocket_messages.map((data) => ({
data,
delay_ms: 0,
}));
for (const message of messages) {
if (message.delay_ms > 0) await delays.wait(message.delay_ms);
if (client.readyState !== 1) return;
client.send(message.data);
recorder.record({
at_ms: recorder.atMs(),
protocol: "websocket",
direction: "sent",
method: null,
path: "/ws",
data: message.data,
outcome: "matched",
});
}
if (script?.disconnect_after === true) {
client.close();
recorder.record({
at_ms: recorder.atMs(),
protocol: "websocket",
direction: "sent",
method: null,
path: "/ws",
data: "",
outcome: "disconnected",
});
}
};
const runWebSocketActions = async (
client: WebSocket,
recorder: ReplayRecorder,
decision: Extract<ReplayMachineDecision, { readonly outcome: "matched" }>,
path: string,
delays: ReplayDelayScheduler,
): Promise<void> => {
for (const action of decision.actions) {
if (action.type === "delay") {
await delays.wait(action.duration_ms);
continue;
}
if (action.type === "disconnect") {
client.close();
recorder.record({
at_ms: recorder.atMs(),
protocol: "websocket",
direction: "sent",
method: null,
path,
data: "",
outcome: "disconnected",
});
return;
}
if (action.type === "websocket_send" && client.readyState === 1) {
client.send(action.data);
recorder.record({
at_ms: recorder.atMs(),
protocol: "websocket",
direction: "sent",
method: null,
path,
data: recorder.machine?.redact(action.data) ?? action.data,
outcome: "matched",
});
}
}
};
const handleStaticWebSocket = (
client: WebSocket,
recorder: ReplayRecorder,
connection: number,
path: string,
delays: ReplayDelayScheduler,
): void => {
client.on("message", (value) =>
recorder.record({
at_ms: recorder.atMs(),
protocol: "websocket",
direction: "received",
method: null,
path,
data: value.toString(),
outcome: "matched",
}),
);
void sendWebSocketScript(client, connection, recorder, delays);
};
const handleMachineWebSocket = (options: {
readonly client: WebSocket;
readonly request: IncomingMessage;
readonly recorder: ReplayRecorder;
readonly connection: number;
readonly path: string;
readonly delays: ReplayDelayScheduler;
}): void => {
const { client, request, recorder, connection, path, delays } = options;
const machine = recorder.machine;
if (machine === undefined) return;
void recorder
.enqueueMachine(async () => {
const connectedAt = recorder.atMs();
const connectionDecision = recorder.dispatchMachine({
protocol: "websocket_connect",
connection: connection === 1 ? "initial" : "reconnect",
at_ms: recorder.rawAtMs(),
recorded_at_ms: connectedAt,
method: null,
path,
headers: requestHeaders(request),
body: "",
});
recordMachineEvent(
recorder,
{
at_ms: connectedAt,
protocol: "websocket",
direction: "received",
method: null,
path,
data: "",
},
connectionDecision,
);
if (connectionDecision.outcome === "matched")
await runWebSocketActions(
client,
recorder,
connectionDecision,
path,
delays,
);
else client.close();
})
.catch(() => {
recorder.truncated = true;
client.terminate();
});
client.on("message", (value) => {
const body = value.toString();
const rawAtMs = recorder.rawAtMs();
const recordedAtMs = recorder.atMs();
void recorder
.enqueueMachine(async () => {
const decision = recorder.dispatchMachine({
protocol: "websocket_message",
connection: connection === 1 ? "initial" : "reconnect",
at_ms: rawAtMs,
recorded_at_ms: recordedAtMs,
method: null,
path,
headers: {},
body,
});
recordMachineEvent(
recorder,
{
at_ms: recordedAtMs,
protocol: "websocket",
direction: "received",
method: null,
path,
data: body,
},
decision,
);
if (decision.outcome === "matched")
await runWebSocketActions(client, recorder, decision, path, delays);
})
.catch(() => {
recorder.truncated = true;
client.terminate();
});
});
};
const createWebSocketReplay = (
server: Server,
recorder: ReplayRecorder,
delays: ReplayDelayScheduler,
): WebSocketServer => {
let connections = 0;
const websocket = new WebSocketServer({
noServer: true,
maxPayload: recorder.scenario.limits.protocol_body_bytes,
clientTracking: true,
});
server.on("upgrade", (request, socket, head) => {
if (
recorder.machine === undefined &&
new URL(request.url ?? "/", "http://127.0.0.1").pathname !== "/ws"
) {
socket.destroy();
return;
}
websocket.handleUpgrade(request, socket, head, (client) =>
websocket.emit("connection", client, request),
);
});
websocket.on("connection", (client: WebSocket, request: IncomingMessage) => {
connections += 1;
if (connections > recorder.scenario.limits.connections) {
recorder.truncated = true;
client.terminate();
return;
}
client.on("error", () => undefined);
const path = new URL(request.url ?? "/", "http://127.0.0.1").pathname;
if (recorder.machine === undefined)
handleStaticWebSocket(client, recorder, connections, path, delays);
else
handleMachineWebSocket({
client,
request,
recorder,
connection: connections,
path,
delays,
});
});
return websocket;
};
/** Start bounded HTTP and WebSocket replay endpoints on IPv4 loopback only. */
export const startLoopbackReplay = async (
scenario: ProcessScenario,
recordEvent: RecordProcessCaptureEvent = () => undefined,
): Promise<LoopbackReplay> => {
const recorder = new ReplayRecorder(scenario, recordEvent);
const delays = new ReplayDelayScheduler();
const server = createServer(createHttpHandler(recorder, delays));
const websocket = createWebSocketReplay(server, recorder, delays);
let closePromise: Promise<void> | undefined;
let port: number;
try {
port = await listenOnLoopback(server);
} catch (cause: unknown) {
await closeReplayServer(server).catch(() => undefined);
throw cause;
}
return {
httpUrl: `http://127.0.0.1:${String(port)}`,
websocketUrl: `ws://127.0.0.1:${String(port)}/ws`,
get events() {
return recorder.events.map((event) => ({ ...event }));
},
get transitions() {
return recorder.transitions.map((transition) => ({
...transition,
sensitive_aliases: [...transition.sensitive_aliases],
}));
},
get truncated() {
return recorder.truncated;
},
async close() {
closePromise ??= (async () => {
delays.close();
recorder.stopMachineAdmission();
await recorder.drainMachine();
for (const client of websocket.clients) client.terminate();
await new Promise<void>((resolveClose) =>
websocket.close(() => resolveClose()),
);
await closeReplayServer(server);
})();
await closePromise;
},
};
};
src/application/LoopbackReplayRecorder.ts
-180
View File
@@ -1,180 +0,0 @@
import type { IncomingMessage, Server } from "node:http";
import type {
ProcessScenario,
ProtocolEvent,
RecordProcessCaptureEvent,
} from "../domain/processCapture.js";
import {
ReplayMachineRuntime,
type ReplayMachineDecision,
type ReplayTransitionRecord,
} from "../domain/replayMachineRuntime.js";
import { scheduleProcessDelay, type ProcessTimer } from "./ProcessTimer.js";
/** Bounded protocol and replay-machine observations for one process run. */
export class ReplayRecorder {
readonly events: ProtocolEvent[] = [];
readonly started = Date.now();
readonly machine: ReplayMachineRuntime | undefined;
#machineQueue: Promise<void> = Promise.resolve();
#machineAdmissionOpen = true;
truncated = false;
constructor(
readonly scenario: ProcessScenario,
private readonly recordEvent: RecordProcessCaptureEvent = () => undefined,
) {
this.machine =
scenario.replay.machine === null
? undefined
: new ReplayMachineRuntime(scenario.replay.machine);
}
get transitions(): readonly ReplayTransitionRecord[] {
return this.machine?.timeline ?? [];
}
atMs(): number {
return (
Math.floor(
(Date.now() - this.started) /
this.scenario.normalization.time_bucket_ms,
) * this.scenario.normalization.time_bucket_ms
);
}
rawAtMs(): number {
return Date.now() - this.started;
}
enqueueMachine<T>(operation: () => Promise<T>): Promise<T> {
if (!this.#machineAdmissionOpen)
return Promise.reject(new Error("replay machine is closing"));
const result = this.#machineQueue.then(operation);
this.#machineQueue = result.then(
() => undefined,
() => undefined,
);
return result;
}
stopMachineAdmission(): void {
this.#machineAdmissionOpen = false;
}
async drainMachine(): Promise<void> {
await this.#machineQueue;
}
dispatchMachine(
event: Parameters<ReplayMachineRuntime["dispatch"]>[0],
): ReplayMachineDecision {
if (this.machine === undefined)
throw new Error("replay machine is not configured");
return this.machine.dispatch(event);
}
recordTransition(decision: ReplayMachineDecision): void {
if (decision.transition !== null)
this.recordEvent("replay_transitions", decision.transition.sequence);
}
record(event: Omit<ProtocolEvent, "sequence">): void {
if (this.events.length >= this.scenario.limits.protocol_events) {
this.truncated = true;
return;
}
const sequence = this.events.length;
this.events.push({ sequence, ...event });
this.recordEvent("protocol_events", sequence);
}
}
export const requestHeaders = (
request: IncomingMessage,
): Readonly<Record<string, string>> =>
Object.fromEntries(
Object.entries(request.headers).flatMap(([name, value]) =>
value === undefined
? []
: [[name, Array.isArray(value) ? value.join(", ") : value]],
),
);
export const readReplayRequestBody = async (
request: IncomingMessage,
limit: number,
): Promise<string | undefined> => {
const chunks: Buffer[] = [];
let bytes = 0;
for await (const chunk of request) {
const value = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
bytes += value.byteLength;
if (bytes > limit) return undefined;
chunks.push(value);
}
return Buffer.concat(chunks).toString("utf8");
};
export const recordMachineEvent = (
recorder: ReplayRecorder,
event: Omit<ProtocolEvent, "sequence" | "outcome">,
decision: ReplayMachineDecision,
): void => {
recorder.record({
...event,
data: recorder.machine?.redact(event.data) ?? event.data,
outcome: decision.outcome,
});
recorder.recordTransition(decision);
};
/** Owns replay delays so endpoint shutdown can release every waiting handler. */
export class ReplayDelayScheduler {
readonly #pending = new Map<ProcessTimer, () => void>();
#closed = false;
wait(durationMs: number): Promise<void> {
if (this.#closed) return Promise.resolve();
return new Promise((resolveDelay) => {
let timer: ProcessTimer;
const finish = (): void => {
this.#pending.delete(timer);
resolveDelay();
};
timer = scheduleProcessDelay(durationMs, finish);
this.#pending.set(timer, finish);
});
}
close(): void {
if (this.#closed) return;
this.#closed = true;
for (const [timer, finish] of this.#pending) {
timer.cancel();
finish();
}
this.#pending.clear();
}
}
export const listenOnLoopback = async (server: Server): Promise<number> => {
await new Promise<void>((resolveListen, rejectListen) => {
server.once("error", rejectListen);
server.listen(0, "127.0.0.1", () => {
server.off("error", rejectListen);
resolveListen();
});
});
const address = server.address();
if (address === null || typeof address === "string")
throw new Error("loopback replay did not acquire a TCP port");
return address.port;
};
export const closeReplayServer = (server: Server): Promise<void> =>
new Promise((resolveClose, rejectClose) => {
server.close((error) =>
error === undefined ? resolveClose() : rejectClose(error),
);
});
src/application/ManagedRuntimeCorrelationService.ts
-95
View File
@@ -1,95 +0,0 @@
import {
AnalysisInputError,
AnalysisProtocolError,
type AnalysisError,
} from "../domain/errors.js";
import { createEvidence, type Evidence } from "../domain/evidence.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import {
managedRuntimeCorrelationInputSchema,
planManagedRuntimeCorrelation,
type ManagedRuntimeCorrelationInput,
} from "../domain/managedRuntimeCorrelation.js";
import { err, ok, type Result } from "../domain/result.js";
import { MANAGED_WORKFLOW_PROVIDER } from "./InvestigationProviders.js";
const OPERATION = "plan_managed_runtime_correlation" as const;
/** Configured local runtime executable used when preparing a correlation plan. */
export interface ManagedRuntimeConfiguration {
readonly executablePath: string | undefined;
}
/** Live managed runtime configuration source, evaluated for each request. */
export interface ManagedRuntimeCorrelationDependencies {
readonly configuration: () => ManagedRuntimeConfiguration;
}
/** Create a non-executing managed runtime correlation plan Evidence record. */
export const planManagedRuntimeCorrelationEvidence = async (
dependencies: ManagedRuntimeCorrelationDependencies,
rawInput: unknown,
): Promise<Result<Evidence, AnalysisError>> => {
const parsed = managedRuntimeCorrelationInputSchema.safeParse(rawInput);
if (!parsed.success)
return err(new AnalysisInputError(OPERATION, { cause: parsed.error }));
return planManagedRuntimeCorrelationEvidenceValidated(
dependencies,
parsed.data,
);
};
/** Plan runtime correlation from input parsed by a trusted adapter. */
export const planManagedRuntimeCorrelationEvidenceValidated = async (
dependencies: ManagedRuntimeCorrelationDependencies,
input: ManagedRuntimeCorrelationInput,
): Promise<Result<Evidence, AnalysisError>> => {
const executablePath =
dependencies.configuration().executablePath ?? "dotnet";
try {
const result = planManagedRuntimeCorrelation(input, executablePath);
return ok(
createEvidence(
{
path: result.static_observation.artifact_path,
sha256: result.static_observation.artifact_sha256,
format: "pe",
},
MANAGED_WORKFLOW_PROVIDER,
{
predicateType: "rea.managed-runtime-correlation-plan",
operation: OPERATION,
parameters: {
static_members_evidence_id: result.static_observation.evidence_id,
method: jsonValueSchema.parse(input.method),
requested_effect: input.requested_effect,
host: jsonValueSchema.parse(input.host),
bounds: jsonValueSchema.parse(input.bounds),
},
result: jsonValueSchema.parse(result),
rawResult: null,
confidence: "derived",
authority: "analyst-inference",
environment: null,
limitations: result.limitations,
locations: [
{
kind: "artifact-path",
path: result.static_observation.artifact_path,
},
],
evidenceLinks: result.evidence_links,
},
),
);
} catch (cause: unknown) {
return err(
cause instanceof TypeError
? new AnalysisInputError(OPERATION, { cause })
: new AnalysisProtocolError(
"Managed runtime correlation planning produced an invalid result",
{ cause },
),
);
}
};
src/application/NodeRuntimeCharacterizationService.ts
-342
View File
@@ -1,342 +0,0 @@
import { createHash } from "node:crypto";
import { z } from "zod";
import {
nodeCharacterizationExecutionInputSchema,
nodeCharacterizationExecutionOutputSchema,
nodeCharacterizationPreparationInputSchema,
nodeCharacterizationPreparationOutputSchema,
type NodeCharacterizationPreparationInput,
} from "../domain/nodeRuntimeCharacterization.js";
import {
AnalysisInputError,
AnalysisProtocolError,
ReplayPlanStaleError,
type AnalysisError,
} from "../domain/errors.js";
import { createEvidence, type Evidence } from "../domain/evidence.js";
import {
controlledReplayOutputSchema,
type ControlledReplayExecutionOutput,
} from "../domain/javascriptReplay.js";
import type { ReplayPlan } from "../domain/javascriptReplay.js";
import type { JavaScriptExportInstrumentation } from "../domain/javascriptExportInstrumentation.js";
import {
createRuntimeCharacterizationPlan,
type RuntimeCharacterizationPlan,
} from "../domain/runtimeCharacterization.js";
import { err, ok, type Result } from "../domain/result.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import type { ExecutionOptions } from "./AnalysisProvider.js";
import { InstrumentedJavaScriptReplayHost } from "./InstrumentedJavaScriptReplayHost.js";
import {
runControlledReplayValidated,
type JavaScriptReplayDependencies,
} from "./JavaScriptReplayService.js";
const PREPARE_OPERATION = "prepare_node_characterization" as const;
const EXECUTE_OPERATION = "execute_node_characterization" as const;
const PROVIDER = {
id: "rea-node-characterization",
name: "REA Node runtime characterization",
version: "1",
} as const;
/** Prepare exact Node/JavaScript characterization without executing target code. */
export const prepareNodeCharacterization = async (
dependencies: JavaScriptReplayDependencies,
rawInput: unknown,
options: ExecutionOptions = {},
): Promise<Result<JsonValue, AnalysisError>> => {
const parsed = nodeCharacterizationPreparationInputSchema.safeParse(rawInput);
if (!parsed.success)
return err(
new AnalysisInputError(PREPARE_OPERATION, { cause: parsed.error }),
);
const prepared = await prepareValidated(dependencies, parsed.data, options);
return prepared.ok
? ok(
jsonValueSchema.parse(
nodeCharacterizationPreparationOutputSchema.parse(prepared.value),
),
)
: prepared;
};
/** Execute only when the supplied plan hash matches freshly prepared content. */
export const executeNodeCharacterization = async (
dependencies: JavaScriptReplayDependencies,
rawInput: unknown,
options: ExecutionOptions = {},
): Promise<Result<JsonValue, AnalysisError>> => {
const parsed = nodeCharacterizationExecutionInputSchema.safeParse(rawInput);
if (!parsed.success)
return err(
new AnalysisInputError(EXECUTE_OPERATION, { cause: parsed.error }),
);
const prepared = await prepareValidated(
dependencies,
parsed.data.preparation,
options,
);
if (!prepared.ok) return prepared;
if (prepared.value.plan.plan_sha256 !== parsed.data.plan_sha256)
return err(
new ReplayPlanStaleError(
parsed.data.plan_sha256,
prepared.value.plan.plan_sha256,
),
);
const plannedReplay = controlledReplayOutputSchema.parse(
prepared.value.replay,
);
if (plannedReplay.phase !== "plan")
return err(
new AnalysisProtocolError(
"Node characterization preparation did not produce a replay plan",
),
);
const host = new InstrumentedJavaScriptReplayHost(
dependencies.host,
parsed.data.preparation.instrumentation,
);
const executed = await runControlledReplayValidated(
{ ...dependencies, host },
{
...parsed.data.preparation.replay,
mode: "execute",
plan_digest: plannedReplay.plan.plan_digest,
},
options,
);
if (!executed.ok) return executed;
const replayOutput = controlledReplayOutputSchema.parse(executed.value);
if (replayOutput.phase !== "execute")
return err(
new AnalysisProtocolError(
"Node characterization execution omitted replay Evidence",
),
);
if (host.instrumentation === null)
return err(
new AnalysisProtocolError(
"Node characterization execution omitted instrumentation evidence",
),
);
return ok(
jsonValueSchema.parse(
nodeCharacterizationExecutionOutputSchema.parse({
phase: "execution",
plan: prepared.value.plan,
transformation: host.instrumentation.manifest,
transformation_evidence: prepared.value.transformation_evidence,
evidence: createCharacterizationEvidence(prepared.value, replayOutput),
replay: replayOutput,
}),
),
);
};
const prepareValidated = async (
dependencies: JavaScriptReplayDependencies,
input: NodeCharacterizationPreparationInput,
options: ExecutionOptions,
): Promise<
Result<
{
readonly phase: "preparation";
readonly plan: RuntimeCharacterizationPlan;
readonly transformation: NonNullable<
InstrumentedJavaScriptReplayHost["instrumentation"]
>["manifest"];
readonly transformation_evidence: Evidence;
readonly replay: z.output<typeof controlledReplayOutputSchema>;
},
AnalysisError
>
> => {
const host = new InstrumentedJavaScriptReplayHost(
dependencies.host,
input.instrumentation,
);
const replay = await runControlledReplayValidated(
{ ...dependencies, host },
input.replay,
options,
);
if (!replay.ok) return replay;
const output = controlledReplayOutputSchema.parse(replay.value);
if (output.phase !== "plan" || host.instrumentation === null)
return err(
new AnalysisProtocolError(
"Node characterization preparation omitted committed plan evidence",
),
);
const module = output.plan.left.modules.find(
({ alias }) => alias === input.selected_alias,
);
if (module === undefined)
return err(
new AnalysisProtocolError(
"Node characterization replay plan omitted selected module",
),
);
const plan = createCharacterizationPlan(
input,
host.instrumentation,
output.plan,
);
return ok({
phase: "preparation",
plan,
transformation: host.instrumentation.manifest,
transformation_evidence: createTransformationEvidence(
input,
host.instrumentation,
),
replay: output,
});
};
const createTransformationEvidence = (
input: NodeCharacterizationPreparationInput,
instrumentation: JavaScriptExportInstrumentation,
): Evidence =>
createEvidence(
{
path: input.instrumentation.artifact_path,
sha256: input.instrumentation.artifact_sha256,
format: "javascript-bundle",
},
PROVIDER,
{
predicateType: "rea.javascript-export-transformation",
operation: PREPARE_OPERATION,
parameters: {
selected_alias: input.selected_alias,
selected_sha256: input.instrumentation.selection.selected_sha256,
},
result: jsonValueSchema.parse(instrumentation.manifest),
confidence: "derived",
authority: "shipped-artifact",
locations: [
{
kind: "file-offset-range",
start: input.instrumentation.selection.byte_start,
end: input.instrumentation.selection.byte_end,
},
],
limitations: [
"The transformation is deterministic derived evidence; execution results are limited to the characterization plan whose content hash matched.",
],
},
);
const createCharacterizationEvidence = (
prepared: {
readonly plan: RuntimeCharacterizationPlan;
readonly transformation_evidence: Evidence;
},
replay: ControlledReplayExecutionOutput,
): Evidence => {
return createEvidence(
{
path: prepared.plan.artifact.path,
sha256: prepared.plan.artifact.sha256,
format: "javascript-bundle",
},
PROVIDER,
{
predicateType: "rea.runtime-characterization",
operation: EXECUTE_OPERATION,
parameters: {
plan_sha256: prepared.plan.plan_sha256,
replay_evidence_id: replay.evidence.evidence_id,
},
result: jsonValueSchema.parse({
plan: prepared.plan,
transformation_evidence_id:
prepared.transformation_evidence.evidence_id,
replay_evidence_id: replay.evidence.evidence_id,
execution: replay.evidence.normalized_result,
}),
confidence: "observed",
authority: "controlled-replay",
environment: replay.evidence.environment,
limitations: [
"This Evidence proves only the finite hash-matched characterization plan and transformed controlled-replay result, not unmodified shipped-runtime equivalence.",
...replay.evidence.limitations,
],
evidenceLinks: [
prepared.transformation_evidence.evidence_id,
replay.evidence.evidence_id,
],
},
);
};
const createCharacterizationPlan = (
input: NodeCharacterizationPreparationInput,
instrumentation: JavaScriptExportInstrumentation,
replayPlan: ReplayPlan,
): RuntimeCharacterizationPlan =>
createRuntimeCharacterizationPlan({
preparation_sha256: digestPreparation(
instrumentation.manifest.instrumented_sha256,
replayPlan.plan_digest,
),
artifact: {
path: input.instrumentation.artifact_path,
sha256: input.instrumentation.artifact_sha256,
byte_length: instrumentation.manifest.original_byte_length,
},
runtime: {
family: "ecmascript",
provider_id: "node-javascript",
executable_path: replayPlan.runtime.executable.path,
executable_sha256: replayPlan.runtime.executable.sha256,
version: replayPlan.runtime.executable.version,
profile_sha256: replayPlan.policy_sha256,
},
callable: {
callable_id: `callable/${input.instrumentation.selection.selected_sha256}`,
module_id: stableModuleId(input.selected_alias),
export_name: input.instrumentation.selection.export_name,
semantic_evidence_id: null,
selector_sha256: input.instrumentation.selection.selected_sha256,
},
working_directory: "/work",
isolated_home: "/work",
expected_effect: input.expected_effect,
allowed_boundaries: [],
limits: {
max_calls: replayPlan.cases.length,
max_processes: 0,
max_files: 0,
max_bytes: replayPlan.limits.output_bytes,
max_handles: replayPlan.limits.tasks,
timeout_ms: replayPlan.limits.wall_time_ms,
idle_timeout_ms: replayPlan.limits.wall_time_ms,
},
determinism: {
clock: "fixed",
randomness: "seeded",
identifiers: "deterministic",
seed: replayPlan.determinism.random_seed,
},
});
const digestPreparation = (
instrumentedSha256: string,
replayPlanDigest: string,
): string =>
createHash("sha256")
.update(
`rea.node-characterization-preparation\0${instrumentedSha256}\0${replayPlanDigest}`,
)
.digest("hex");
const stableModuleId = (alias: string): string =>
`module/${createHash("sha256").update(alias).digest("hex")}`;
src/application/ProcessCaptureEnvironment.ts
+13 -27
View File
@@ -1,38 +1,24 @@
import { delimiter } from "node:path";
import type { ProcessScenario } from "../domain/processCapture.js";
import type { LoopbackReplay } from "./LoopbackReplay.js";
import type { CommandShimReplay } from "./CommandShimReplay.js";
interface ProcessCaptureEnvironmentOptions {
readonly scenario: ProcessScenario;
readonly replay: LoopbackReplay;
readonly shimReplay: CommandShimReplay;
readonly runId: string;
readonly hostEnvironment: Readonly<Record<string, string | undefined>>;
}
/** Build the inherited environment with scenario overrides and replay instrumentation. */
/** Build the child environment from the host plus caller-selected overrides. */
export const makeProcessCaptureEnvironment = (
options: ProcessCaptureEnvironmentOptions,
): Record<string, string> => {
const { scenario, replay, shimReplay, runId, hostEnvironment } = options;
const environment: Record<string, string> = {
...Object.fromEntries(
Object.entries(hostEnvironment).filter(
(entry): entry is [string, string] => entry[1] !== undefined,
),
): Record<string, string> => ({
...Object.fromEntries(
Object.entries(options.hostEnvironment).filter(
(entry): entry is [string, string] => entry[1] !== undefined,
),
...scenario.environment,
TERM: scenario.environment.TERM ?? hostEnvironment.TERM ?? "xterm-256color",
REA_PROCESS_RUN_ID: runId,
};
environment.REA_REPLAY_HTTP_URL = replay.httpUrl;
environment.REA_REPLAY_WEBSOCKET_URL = replay.websocketUrl;
environment.REA_SHIM_LEDGER_URL = shimReplay.url;
// Shims are instrumentation; all remaining executable lookup uses host PATH.
environment.PATH = [shimReplay.binPath, environment.PATH ?? ""]
.filter((part) => part.length > 0)
.join(delimiter);
return environment;
};
),
...options.scenario.environment,
TERM:
options.scenario.environment.TERM ??
options.hostEnvironment.TERM ??
"xterm-256color",
REA_PROCESS_RUN_ID: options.runId,
});
src/application/ProcessCaptureJournal.test.ts
+6 -23
View File
@@ -1,35 +1,18 @@
import { expect, it } from "vitest";
import type { ProcessCaptureEventJournalEntry } from "../domain/processCapture.js";
import { createProcessCaptureJournal } from "./ProcessCaptureJournal.js";
it("publishes journal entries in capture order and supports nested recording", () => {
it("records capture events in monotonic observation order", () => {
const journal = createProcessCaptureJournal();
const observed: ProcessCaptureEventJournalEntry[] = [];
const unsubscribe = journal.subscribe((entry) => observed.push(entry));
journal.recordEvent("filesystem_checkpoints", 0);
journal.recordEvent("lifecycle", 0);
const settlementEntry = journal.record("lifecycle", 1);
unsubscribe();
journal.recordEvent("frames", 0);
journal.recordEvent("filesystem_checkpoints", 1);
expect(journal.entries).toEqual([
{ capture_order: 0, collection: "lifecycle", index: 0 },
{ capture_order: 1, collection: "lifecycle", index: 1 },
{ capture_order: 0, collection: "filesystem_checkpoints", index: 0 },
{ capture_order: 1, collection: "lifecycle", index: 0 },
{ capture_order: 2, collection: "frames", index: 0 },
{ capture_order: 3, collection: "filesystem_checkpoints", index: 1 },
]);
expect(observed).toEqual(journal.entries.slice(0, 2));
expect(settlementEntry).toEqual(journal.entries[1]);
const nestedJournal = createProcessCaptureJournal();
const notifications: string[] = [];
nestedJournal.subscribe((entry) => {
notifications.push(`a:${String(entry.capture_order)}`);
if (entry.capture_order === 0) nestedJournal.record("frames", 1);
});
nestedJournal.subscribe((entry) =>
notifications.push(`b:${String(entry.capture_order)}`),
);
nestedJournal.record("frames", 0);
expect(notifications).toEqual(["a:0", "b:0", "a:1", "b:1"]);
});
src/application/ProcessCaptureJournal.ts
+4 -47
View File
@@ -17,63 +17,20 @@ import { scheduleProcessDelay, type ProcessTimer } from "./ProcessTimer.js";
export interface ProcessCaptureJournal {
readonly entries: readonly ProcessCaptureEventJournalEntry[];
readonly recordEvent: RecordProcessCaptureEvent;
readonly record: (
collection: ProcessCaptureEventJournalEntry["collection"],
index: number,
) => ProcessCaptureEventJournalEntry;
readonly subscribe: (
listener: (entry: ProcessCaptureEventJournalEntry) => void,
) => () => void;
}
/** Create one monotonic event journal for a process capture. */
export const createProcessCaptureJournal = (): ProcessCaptureJournal => {
const entries: ProcessCaptureEventJournalEntry[] = [];
const listeners = new Set<(entry: ProcessCaptureEventJournalEntry) => void>();
const notifications: ProcessCaptureEventJournalEntry[] = [];
let delivering = false;
const deliverNotifications = (): void => {
if (delivering) return;
delivering = true;
const failures: unknown[] = [];
try {
for (;;) {
const notification = notifications.shift();
if (notification === undefined) break;
for (const listener of Array.from(listeners))
try {
listener(notification);
} catch (cause: unknown) {
failures.push(cause);
}
}
} finally {
delivering = false;
}
if (failures.length > 0)
throw new AggregateError(
failures,
"process capture journal subscriber failed",
);
};
const record = (
const recordEvent: RecordProcessCaptureEvent = (
collection: ProcessCaptureEventJournalEntry["collection"],
index: number,
): ProcessCaptureEventJournalEntry => {
const entry = { capture_order: entries.length, collection, index };
entries.push(entry);
notifications.push(entry);
deliverNotifications();
return entry;
): void => {
entries.push({ capture_order: entries.length, collection, index });
};
return {
entries,
record,
recordEvent: (collection, index) => void record(collection, index),
subscribe: (listener) => {
listeners.add(listener);
return () => listeners.delete(listener);
},
recordEvent,
};
};
src/application/ProcessCaptureLifecycle.ts
+25 -49
View File
@@ -25,24 +25,18 @@ import {
import { PRODUCT_IDENTITY } from "../identity.js";
import type { SnapshotResult } from "./FilesystemSnapshot.js";
import { snapshotRoots } from "./FilesystemSnapshot.js";
import type { LoopbackReplay } from "./LoopbackReplay.js";
import { classifyFilesystemEffects } from "./ProcessFilesystemEffects.js";
import {
cleanupOwnedProcessGroup,
observeOwnedProcessGroup,
} from "../process/ProcessOwnership.js";
import {
classifyFilesystemEffects,
ProcessCheckpoints,
} from "./ProcessCheckpoints.js";
import { ProcessCaptureError } from "./ProcessCaptureError.js";
import { assertNotCancelled } from "./ProcessScenarioRuntimeValidation.js";
import {
normalizeProcessElapsedTime,
normalizeProcessSamples,
normalizeProcessText,
normalizeProtocolEvents,
} from "./ProcessNormalization.js";
import type { CommandShimReplay } from "./CommandShimReplay.js";
import { PROCESS_PROVIDER } from "./ProcessEvidence.js";
import { TerminalRenderer } from "./TerminalRenderer.js";
import { scheduleProcessInterval, type ProcessTimer } from "./ProcessTimer.js";
@@ -67,7 +61,6 @@ interface CaptureResultOptions {
readonly reason: "exited" | "timeout" | "idle_timeout";
};
readonly samples: readonly ProcessSample[];
readonly replay: LoopbackReplay;
readonly before: SnapshotResult;
readonly after: SnapshotResult;
readonly truncated: boolean;
@@ -77,7 +70,6 @@ interface CaptureResultOptions {
readonly renderedFrames: UnverifiedProcessCapture["rendered_frames"];
readonly interactions: readonly InteractionEvent[];
readonly checkpoints: readonly FilesystemCheckpoint[];
readonly shimEvents: UnverifiedProcessCapture["shim_events"];
readonly settlement: ObservedProcessSettlement;
readonly manifest: UnverifiedProcessCapture["manifest"];
readonly eventJournal: readonly ProcessCaptureEventJournalEntry[];
@@ -94,14 +86,32 @@ export type ObservedProcessSettlement =
"state" | "elapsed_ms"
>;
/** Capture observations that cannot yet claim final cleanup or reactive state. */
/** Capture observations before owned-resource cleanup has been verified. */
export type PendingProcessCapture = Omit<
UnverifiedProcessCapture,
"cleanup" | "reactive_run" | "settlement"
"cleanup" | "settlement"
> & {
readonly settlement: ObservedProcessSettlement;
};
/** Wait for trailing PTY callbacks before the capture becomes immutable. */
export const settleProcessCaptureJournal = async (
journal: readonly ProcessCaptureEventJournalEntry[],
quietMs = 25,
maxWaitMs = 500,
): Promise<void> => {
const deadline = Date.now() + maxWaitMs;
let observed = journal.length;
let lastChangeAt = Date.now();
while (Date.now() < deadline) {
await new Promise<void>((resolve) => setTimeout(resolve, 4));
if (journal.length !== observed) {
observed = journal.length;
lastChangeAt = Date.now();
} else if (Date.now() - lastChangeAt >= quietMs) return;
}
};
export const buildCaptureResult = (
options: CaptureResultOptions,
): PendingProcessCapture => {
@@ -112,10 +122,6 @@ export const buildCaptureResult = (
"Sensitive scripted input values are redacted from process capture Evidence.";
const filesystemObservationUnknown =
"No filesystem observation paths were selected; filesystem effects remain unknown.";
const protocolEvents = normalizeProtocolEvents(
options.replay.events,
options.scenario,
);
const hasFilesystemObservations =
options.scenario.filesystem_observation_paths.length > 0;
return {
@@ -139,9 +145,6 @@ export const buildCaptureResult = (
options.rootPid,
),
filesystem_checkpoints: options.checkpoints,
shim_events: options.shimEvents,
protocol_events: protocolEvents,
replay_transitions: options.replay.transitions,
event_journal: options.eventJournal,
files_before: options.before.files,
files_after: options.after.files,
@@ -156,7 +159,6 @@ export const buildCaptureResult = (
? ["Process-tree sampling ended with an incomplete observation."]
: []),
"Filesystem observations are before/after snapshots, not syscall traces.",
"The harness does not enforce external network isolation.",
"Inherited host environment variables are not recorded and may affect results.",
...(!hasFilesystemObservations ? [filesystemObservationUnknown] : []),
...(hasSensitiveScriptedInput ? [sensitiveInputUnknown] : []),
@@ -167,14 +169,14 @@ export const buildCaptureResult = (
reason:
"Process trees are sampled and may omit short-lived descendants.",
},
{
scope: "network",
reason: "External network isolation is not enforced by this adapter.",
},
{
scope: "environment",
reason: "Inherited host environment variables are not recorded.",
},
{
scope: "network",
reason: "Network activity is not observed by this capture.",
},
...(!hasFilesystemObservations
? [
{
@@ -217,14 +219,10 @@ export const createRunManifest = async (
completed_at: completedAt.toISOString(),
scenario: scenarioCommitment,
comparison_contract: comparisonContract,
shim_plan: scenario.command_shims,
replay_plan: scenario.replay,
full_scenario_sha256: digestProcessCommitment(scenarioCommitment),
comparison_contract_sha256: digestProcessCommitment(comparisonContract),
executable_sha256: executableSha256,
normalization_sha256: digestProcessCommitment(scenario.normalization),
shim_plan_sha256: digestProcessCommitment(scenario.command_shims),
replay_plan_sha256: digestProcessCommitment(scenario.replay),
};
};
@@ -346,11 +344,8 @@ const processCaptureCleanupHost: ProcessCaptureCleanupHost = {
export const releaseProcessResources = async (options: {
readonly timers: ReadonlySet<ProcessTimer>;
readonly replay: LoopbackReplay | undefined;
readonly terminal: IPty | undefined;
readonly renderer: TerminalRenderer | undefined;
readonly shimReplay: CommandShimReplay | undefined;
readonly checkpoints: ProcessCheckpoints | undefined;
readonly runId: string;
readonly temporaryRoot: string;
readonly capturedProcessGroupIds: readonly number[];
@@ -359,26 +354,11 @@ export const releaseProcessResources = async (options: {
const host = options.host ?? processCaptureCleanupHost;
for (const timer of options.timers) timer.cancel();
let failure: string | undefined;
try {
await options.replay?.close();
} catch {
failure = "loopback replay cleanup failed";
}
try {
await options.shimReplay?.close();
} catch {
failure ??= "command shim replay cleanup failed";
}
try {
await options.renderer?.dispose();
} catch {
failure ??= "terminal renderer cleanup failed";
}
try {
await options.checkpoints?.dispose();
} catch {
failure ??= "filesystem checkpoint cleanup failed";
}
if (options.terminal !== undefined && host.platform !== "win32") {
for (const processGroupId of new Set(options.capturedProcessGroupIds)) {
const cleaned = await host.cleanupProcessGroup({
@@ -405,7 +385,6 @@ export const captureTerminalFrames = (options: {
readonly temporaryRoot: string;
readonly onOutput: () => void;
readonly renderer: TerminalRenderer;
readonly checkpoints: ProcessCheckpoints;
readonly recordEvent: RecordProcessCaptureEvent;
}): (() => boolean) => {
let outputBytes = 0;
@@ -434,11 +413,8 @@ export const captureTerminalFrames = (options: {
at_ms: atMs,
data: normalized,
});
// Queue the observed output before publishing it: reactive subscribers may
// synchronously resize the terminal in response to this journal entry.
options.renderer.write(data, atMs);
options.recordEvent("frames", sequence);
options.checkpoints.observeTerminal(normalized);
});
return () => truncated;
};
src/application/ProcessCaptureObservation.test.ts
+9 -41
View File
@@ -2,7 +2,6 @@ import { expect, it } from "vitest";
import { snapshotRoots } from "./FilesystemSnapshot.js";
import { buildCaptureResult } from "./ProcessCaptureLifecycle.js";
import { ProcessCheckpoints } from "./ProcessCheckpoints.js";
import { normalizeProcessSamples } from "./ProcessNormalization.js";
import { isInitializedPtyRoot, readLinuxChildren } from "./ProcessSampling.js";
import { TerminalRenderer } from "./TerminalRenderer.js";
@@ -19,7 +18,7 @@ const base = {
working_directory: "/tmp",
};
it("returns detached terminal and filesystem checkpoint observations", async () => {
it("returns detached terminal observations", async () => {
const observed: string[] = [];
const renderer = new TerminalRenderer({
columns: 40,
@@ -46,21 +45,6 @@ it("returns detached terminal and filesystem checkpoint observations", async ()
expect(reread).toHaveLength(frames.length);
expect(reread[0]?.cursor_x).toBe(originalCursor);
await renderer.dispose();
const scenario = parseProcessScenario(base);
const checkpoints = new ProcessCheckpoints(
scenario,
Date.now(),
{ files: [], truncated: false },
{ signal: undefined },
);
const first = await checkpoints.finish({ files: [], truncated: false });
const checkpoint = first[0];
if (checkpoint === undefined) throw new Error("expected checkpoint");
Reflect.set(checkpoint, "name", "tampered");
const second = await checkpoints.finish({ files: [], truncated: false });
expect(second[0]?.name).toBe("before");
await checkpoints.dispose();
});
it("preserves rendered observation order instead of timestamp sorting", () => {
@@ -93,14 +77,6 @@ it("preserves rendered observation order instead of timestamp sorting", () => {
frames: [],
exit: { exitCode: 0, reason: "exited" },
samples: [],
replay: {
httpUrl: "http://127.0.0.1",
websocketUrl: "ws://127.0.0.1/ws",
events: [],
transitions: [],
truncated: false,
close: () => Promise.resolve(),
},
before: { files: [], truncated: false },
after: { files: [], truncated: false },
truncated: false,
@@ -110,7 +86,6 @@ it("preserves rendered observation order instead of timestamp sorting", () => {
renderedFrames,
interactions: [],
checkpoints: capture.filesystem_checkpoints,
shimEvents: [],
settlement: {
state: capture.settlement.state,
elapsed_ms: capture.settlement.elapsed_ms,
@@ -128,14 +103,6 @@ it("marks redacted scripted input as an interaction unknown", () => {
frames: [],
exit: { exitCode: 0, reason: "exited" },
samples: [],
replay: {
httpUrl: "http://127.0.0.1",
websocketUrl: "ws://127.0.0.1/ws",
events: [],
transitions: [],
truncated: false,
close: () => Promise.resolve(),
},
before: { files: [], truncated: false },
after: { files: [], truncated: false },
truncated: false,
@@ -157,7 +124,6 @@ it("marks redacted scripted input as an interaction unknown", () => {
},
],
checkpoints: capture.filesystem_checkpoints,
shimEvents: [],
settlement: {
state: capture.settlement.state,
elapsed_ms: capture.settlement.elapsed_ms,
@@ -259,7 +225,7 @@ it("admits PTY samples only after stable session and token setup", () => {
).toBe(true);
});
it("keeps interaction and shim residual uncertainty in separate scopes", () => {
it("keeps interaction and process residual uncertainty in separate scopes", () => {
const baseCapture = emptyCapture();
const interaction = parseProcessCapture({
...baseCapture,
@@ -267,22 +233,24 @@ it("keeps interaction and shim residual uncertainty in separate scopes", () => {
{ scope: "interaction", reason: "Interaction capture was partial." },
],
});
const shim = parseProcessCapture({
const process = parseProcessCapture({
...baseCapture,
residual_unknowns: [{ scope: "shim", reason: "Shim capture was partial." }],
residual_unknowns: [
{ scope: "process", reason: "Process sampling was partial." },
],
});
expect(compareProcessCaptures(interaction, baseCapture)).toMatchObject({
status: "unknown",
terminal: "unchanged",
interaction: "unknown",
shim: "unchanged",
process: "unchanged",
});
expect(compareProcessCaptures(shim, baseCapture)).toMatchObject({
expect(compareProcessCaptures(process, baseCapture)).toMatchObject({
status: "unknown",
terminal: "unchanged",
interaction: "unchanged",
shim: "unknown",
process: "unknown",
});
});
src/application/ProcessCheckpoints.ts
-216
View File
@@ -1,216 +0,0 @@
import type {
FileState,
FilesystemCheckpoint,
ProcessScenario,
RecordProcessCaptureEvent,
} from "../domain/processCapture.js";
import { snapshotRoots, type SnapshotResult } from "./FilesystemSnapshot.js";
import { scheduleProcessDelay, type ProcessTimer } from "./ProcessTimer.js";
/** Classify path-stable filesystem effects between two bounded states. */
export const classifyFilesystemEffects = (
before: readonly FileState[],
after: readonly FileState[],
): FilesystemCheckpoint["effects"] => {
const beforeByPath = new Map(before.map((file) => [file.path, file]));
const afterByPath = new Map(after.map((file) => [file.path, file]));
return [...new Set([...beforeByPath.keys(), ...afterByPath.keys()])]
.sort()
.map((path) => {
const beforeFile = beforeByPath.get(path) ?? null;
const afterFile = afterByPath.get(path) ?? null;
if (beforeFile === null) {
if (afterFile === null)
throw new TypeError(`Filesystem effect ${path} has no file state`);
return {
path,
status: "created",
before: null,
after: afterFile,
} as const;
}
if (afterFile === null)
return {
path,
status: "deleted",
before: beforeFile,
after: null,
} as const;
return {
path,
status:
JSON.stringify(beforeFile) === JSON.stringify(afterFile)
? "unchanged"
: "modified",
before: beforeFile,
after: afterFile,
} as const;
});
};
/** Coordinates named lifecycle filesystem observations without overlapping scans. */
export class ProcessCheckpoints {
readonly #captures: FilesystemCheckpoint[] = [];
readonly #captured = new Set<string>();
readonly #terminalCounts = new Map<string, number>();
readonly #terminalTails = new Map<string, string>();
readonly #timers = new Set<ProcessTimer>();
readonly #signal: AbortSignal | undefined;
readonly #recordEvent: RecordProcessCaptureEvent;
#pending: Promise<void> = Promise.resolve();
constructor(
private readonly scenario: ProcessScenario,
private readonly started: number,
before: SnapshotResult,
options: {
readonly signal: AbortSignal | undefined;
readonly recordEvent?: RecordProcessCaptureEvent;
} = { signal: undefined },
) {
this.#signal = options.signal;
this.#recordEvent = options.recordEvent ?? (() => undefined);
this.#captures.push({
name: "before",
at_ms: 0,
files: before.files,
effects: [],
truncated: before.truncated,
});
this.#recordEvent("filesystem_checkpoints", 0);
this.#captured.add("before");
for (const checkpoint of scenario.checkpoints) {
if (checkpoint.trigger.type !== "time") continue;
const timer = scheduleProcessDelay(checkpoint.trigger.at_ms, () =>
this.capture(checkpoint.name),
);
this.#timers.add(timer);
}
}
/** Observe normalized terminal text and fire matching literal checkpoints once. */
observeTerminal(data: string): void {
for (const checkpoint of this.scenario.checkpoints) {
const trigger = checkpoint.trigger;
if (
trigger.type !== "terminal_literal" ||
this.#captured.has(checkpoint.name)
)
continue;
const combined = `${this.#terminalTails.get(checkpoint.name) ?? ""}${data}`;
const count =
(this.#terminalCounts.get(checkpoint.name) ?? 0) +
countOccurrences(combined, trigger.value);
this.#terminalCounts.set(checkpoint.name, count);
this.#terminalTails.set(
checkpoint.name,
trigger.value.length === 1
? ""
: combined.slice(-(trigger.value.length - 1)),
);
if (count >= trigger.occurrence) this.capture(checkpoint.name);
}
}
/** Capture every checkpoint assigned to the specified lifecycle trigger. */
trigger(type: "root_exit" | "settled"): void {
for (const checkpoint of this.scenario.checkpoints)
if (checkpoint.trigger.type === type) this.capture(checkpoint.name);
}
/** Queue one idempotent named snapshot. */
capture(name: string): void {
this.#scheduleCapture(name, this.#signal);
}
/** Read one committed checkpoint for synchronous journal projection. */
at(index: number): FilesystemCheckpoint | undefined {
const checkpoint = this.#captures[index];
return checkpoint === undefined ? undefined : structuredClone(checkpoint);
}
#scheduleCapture(name: string, signal: AbortSignal | undefined): void {
if (this.#captured.has(name)) return;
this.#captured.add(name);
// Serialize scans even when time, terminal, and lifecycle triggers fire
// together. Each effect set is intentionally relative to the immediately
// preceding checkpoint, which preserves transient create/delete behavior.
this.#pending = this.#pending.then(async () => {
const snapshot = await snapshotRoots(this.scenario, signal);
const previous = this.#captures.at(-1)?.files ?? [];
const index = this.#captures.length;
this.#captures.push({
name,
at_ms: Math.max(0, Date.now() - this.started),
files: snapshot.files,
effects: classifyFilesystemEffects(previous, snapshot.files),
truncated: snapshot.truncated,
});
this.#recordEvent("filesystem_checkpoints", index);
});
}
/** Queue one named snapshot and return its exact captured record. */
async captureAndRead(
name: string,
signal?: AbortSignal,
): Promise<{
readonly index: number;
readonly checkpoint: FilesystemCheckpoint;
}> {
const captureSignal =
signal === undefined
? this.#signal
: this.#signal === undefined
? signal
: AbortSignal.any([this.#signal, signal]);
this.#scheduleCapture(name, captureSignal);
const scheduled = this.#pending;
await scheduled;
const index = this.#captures.findIndex(
({ name: capturedName }) => capturedName === name,
);
const checkpoint = this.#captures[index];
if (index < 0 || checkpoint === undefined)
throw new Error(`filesystem checkpoint was not captured: ${name}`);
return { index, checkpoint: structuredClone(checkpoint) };
}
/** Finish pending snapshots and return them in observation order. */
async finish(
after: SnapshotResult,
): Promise<readonly FilesystemCheckpoint[]> {
for (const timer of this.#timers) timer.cancel();
await this.#pending;
const previous = this.#captures.at(-1)?.files ?? [];
const index = this.#captures.length;
this.#captures.push({
name: "after_settlement",
at_ms: Math.max(0, Date.now() - this.started),
files: after.files,
effects: classifyFilesystemEffects(previous, after.files),
truncated: after.truncated,
});
this.#recordEvent("filesystem_checkpoints", index);
return structuredClone(this.#captures);
}
/** Cancel scheduled captures and await any snapshot already in progress. */
async dispose(): Promise<void> {
for (const timer of this.#timers) timer.cancel();
this.#timers.clear();
await this.#pending;
}
}
const countOccurrences = (text: string, value: string): number => {
let count = 0;
let offset = 0;
while (offset <= text.length - value.length) {
const found = text.indexOf(value, offset);
if (found < 0) break;
count += 1;
offset = found + value.length;
}
return count;
};
src/application/ProcessEvidence.test.ts
+1 -40
View File
@@ -2,7 +2,6 @@ import { expect, it } from "vitest";
import { createProcessCaptureEvidence } from "./ProcessEvidence.js";
import { createRunManifest } from "./ProcessCaptureLifecycle.js";
import { normalizeProtocolEvents } from "./ProcessNormalization.js";
import { emptyUnverifiedProcessCapture } from "../domain/processCapture.fixture.js";
import {
compareProcessCaptures,
@@ -61,28 +60,6 @@ const captureEvidenceForEnvironmentSecret = async (value: string) => {
working_directory: "/tmp",
environment: { API_TOKEN: value },
events: [{ type: "input", at_ms: 0, data: value }],
command_shims: [
{
name: "echo",
routes: [
{
arguments: [],
outputs: [{ at_ms: 0, stream: "stdout", data: value }],
termination: { type: "exit", code: 0 },
},
],
},
],
replay: {
http: [
{
method: "GET",
path: `/token?token=${value}`,
status: 200,
body: value,
},
],
},
});
const base = emptyUnverifiedProcessCapture();
const manifest = await createRunManifest(scenario, new Date(0), new Date(1));
@@ -94,21 +71,6 @@ const captureEvidenceForEnvironmentSecret = async (value: string) => {
...base,
normalization: scenario.normalization,
frames: [{ sequence: 0, at_ms: 0, data: "observed-output" }],
protocol_events: normalizeProtocolEvents(
[
{
sequence: 0,
at_ms: 0,
protocol: "http",
direction: "request",
method: "GET",
path: `/token?token=${value}`,
data: "",
outcome: "matched",
},
],
scenario,
),
residual_unknowns: [
{
scope: "environment",
@@ -143,7 +105,7 @@ it("does not treat captures with different redacted inputs as equivalent", () =>
).toMatchObject({ status: "unknown", interaction: "unknown" });
});
it("preserves selected local environment, shim, replay, and protocol values", async () => {
it("preserves selected local environment and input values", async () => {
const left = await captureEvidenceForEnvironmentSecret("secret-left");
const right = await captureEvidenceForEnvironmentSecret("secret-right");
const serialized = JSON.stringify(left);
@@ -151,7 +113,6 @@ it("preserves selected local environment, shim, replay, and protocol values", as
const rightCapture = parseProcessCapture(right.normalized_result);
expect(serialized).toContain("secret-left");
expect(serialized).toContain("/token?token=secret-left");
expect(leftCapture.manifest.scenario).toMatchObject({
environment: { API_TOKEN: "secret-left" },
});
src/application/ProcessEvidence.ts
+1 -3
View File
@@ -9,7 +9,7 @@ import type {
/** Provider identity for controlled process capture evidence. */
export const PROCESS_PROVIDER = {
id: "rea-process",
name: "REA deterministic process harness",
name: "REA process capture",
version: "3",
} as const;
@@ -22,8 +22,6 @@ const processEvidenceParameters = (
event_count: scenario.events.length,
filesystem_observation_path_count:
scenario.filesystem_observation_paths.length,
checkpoint_count: scenario.checkpoints.length,
command_shim_count: scenario.command_shims.length,
normalization: scenario.normalization,
});
src/application/ProcessFilesystemEffects.ts
+45
View File
@@ -0,0 +1,45 @@
import type {
FileState,
FilesystemCheckpoint,
} from "../domain/processCapture.js";
/** Classify path-stable filesystem effects between two bounded states. */
export const classifyFilesystemEffects = (
before: readonly FileState[],
after: readonly FileState[],
): FilesystemCheckpoint["effects"] => {
const beforeByPath = new Map(before.map((file) => [file.path, file]));
const afterByPath = new Map(after.map((file) => [file.path, file]));
return [...new Set([...beforeByPath.keys(), ...afterByPath.keys()])]
.sort()
.map((path) => {
const beforeFile = beforeByPath.get(path) ?? null;
const afterFile = afterByPath.get(path) ?? null;
if (beforeFile === null) {
if (afterFile === null)
throw new TypeError(`Filesystem effect ${path} has no file state`);
return {
path,
status: "created",
before: null,
after: afterFile,
} as const;
}
if (afterFile === null)
return {
path,
status: "deleted",
before: beforeFile,
after: null,
} as const;
return {
path,
status:
JSON.stringify(beforeFile) === JSON.stringify(afterFile)
? "unchanged"
: "modified",
before: beforeFile,
after: afterFile,
} as const;
});
};
src/application/ProcessHarness.test.ts
+37
View File
@@ -2,6 +2,43 @@ import { expect, it } from "vitest";
import { processScenarioSchema } from "../domain/processScenario.js";
import { AnalysisCapabilityUnavailableError } from "../domain/errors.js";
import { captureProcessScenario } from "./ProcessHarness.js";
import { settleProcessCaptureJournal } from "./ProcessCaptureLifecycle.js";
import { processCaptureSchema } from "../domain/processCapture.js";
import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from "../domain/processCapture.fixture.js";
it("rejects legacy replay output instead of silently discarding it", () => {
expect(
processCaptureSchema.safeParse({
...EMPTY_PROCESS_CAPTURE_EXAMPLE,
protocol_events: [],
}).success,
).toBe(false);
});
it("waits for terminal observations delivered after the exit callback", async () => {
const journal: Array<{
capture_order: number;
collection: "frames";
index: number;
}> = [{ capture_order: 0, collection: "frames", index: 0 }];
const lateFrame = new Promise<void>((resolve) => {
setTimeout(() => {
journal.push({ capture_order: 1, collection: "frames", index: 1 });
resolve();
}, 12);
});
let settled = false;
const wait = settleProcessCaptureJournal(journal, 20, 200).then(() => {
settled = true;
});
await lateFrame;
expect(settled).toBe(false);
await wait;
expect(journal).toHaveLength(2);
expect(journal[1]).toMatchObject({ collection: "frames", index: 1 });
});
it("fails closed on Windows before resolving or launching scenario paths", async () => {
const scenario = processScenarioSchema.parse({
src/application/ProcessHarness.ts
+24 -157
View File
@@ -20,20 +20,11 @@ import {
processCaptureCancelled,
} from "./ProcessCaptureError.js";
export { ProcessCaptureError } from "./ProcessCaptureError.js";
import { startLoopbackReplay, type LoopbackReplay } from "./LoopbackReplay.js";
import { startProcessSampler } from "./ProcessSampling.js";
import { snapshotRoots } from "./FilesystemSnapshot.js";
import { ProcessCheckpoints } from "./ProcessCheckpoints.js";
import { TerminalRenderer } from "./TerminalRenderer.js";
import type { ProcessTimer } from "./ProcessTimer.js";
import {
startCommandShimReplay,
type CommandShimReplay,
} from "./CommandShimReplay.js";
import {
normalizeProcessShimEvent,
normalizeProcessText,
} from "./ProcessNormalization.js";
import { normalizeProcessText } from "./ProcessNormalization.js";
import { selectCapturedProcessGroupIds } from "../process/ProcessOwnership.js";
import {
awaitTerminalExit,
@@ -44,6 +35,7 @@ import {
prepareProcessCapture,
releaseProcessResources,
resolveProcessResult,
settleProcessCaptureJournal,
type PendingProcessCapture,
} from "./ProcessCaptureLifecycle.js";
import {
@@ -53,40 +45,27 @@ import {
import {
createProcessCaptureJournal,
scheduleScenarioInteractions,
type ProcessCaptureJournal,
} from "./ProcessCaptureJournal.js";
import {
assertSupportedReactiveScenario,
projectProcessReactiveRun,
startProcessReactiveHarness,
type ProcessReactiveHarness,
} from "./ProcessReactiveHarness.js";
import { makeProcessCaptureEnvironment } from "./ProcessCaptureEnvironment.js";
import { classifyFilesystemEffects } from "./ProcessFilesystemEffects.js";
import { processCaptureOwnershipUnavailableReason } from "./ProcessCaptureCapability.js";
export { probeProcessCaptureCapability } from "./ProcessCaptureCapability.js";
interface StartedCaptureRuntime {
readonly replay: LoopbackReplay;
readonly renderer: TerminalRenderer;
readonly checkpoints: ProcessCheckpoints;
readonly shimReplay: CommandShimReplay;
readonly terminal: IPty;
readonly started: number;
readonly startedAt: Date;
readonly lastOutput: () => number;
readonly framesTruncated: () => boolean;
readonly stopSampler: () => Promise<{ readonly partial: boolean }>;
readonly reactive: ProcessReactiveHarness | undefined;
}
const cleanupFailedStartup = async (options: {
readonly cause: unknown;
readonly timers: Set<ProcessTimer>;
readonly replay: LoopbackReplay | undefined;
readonly terminal: IPty | undefined;
readonly renderer: TerminalRenderer | undefined;
readonly shimReplay: CommandShimReplay | undefined;
readonly checkpoints: ProcessCheckpoints | undefined;
readonly runId: string;
readonly temporaryRoot: string;
}): Promise<never> => {
@@ -125,67 +104,23 @@ interface StartCaptureRuntimeOptions {
readonly hostEnvironment: Readonly<Record<string, string | undefined>>;
readonly temporaryRoot: string;
readonly runId: string;
readonly before: Awaited<ReturnType<typeof snapshotRoots>>;
readonly frames: TerminalFrame[];
readonly samples: ProcessSample[];
readonly interactions: InteractionEvent[];
readonly timers: Set<ProcessTimer>;
readonly dispatchedEventIndexes: Set<number>;
readonly recordEvent: RecordProcessCaptureEvent;
readonly journal: ProcessCaptureJournal;
readonly signal?: AbortSignal;
}
/**
* Wait until the capture journal stops growing.
*
* node-pty reports buffered output through `onData` callbacks rather than an
* awaitable read, so chunks can still be in flight after the child exits. The
* quiet interval is measured in elapsed time rather than scheduler turns:
* consecutive `setImmediate` turns elapse in microseconds, so any fixed number
* of them finishes long before a delayed pty callback under host contention,
* which is exactly when the trailing chunk matters. `maxWaitMs` bounds the
* total wait so a pathological stream cannot stall exit.
*/
const settleTrailingJournal = async (
journal: readonly ProcessCaptureEventJournalEntry[],
quietMs = 25,
maxWaitMs = 500,
): Promise<void> => {
const deadline = Date.now() + maxWaitMs;
let observed = journal.length;
let lastChangeAt = Date.now();
while (Date.now() < deadline) {
await new Promise<void>((resolve) => setTimeout(resolve, 4));
if (journal.length !== observed) {
observed = journal.length;
lastChangeAt = Date.now();
} else if (Date.now() - lastChangeAt >= quietMs) return;
}
};
const reactiveCapture = (
options: StartCaptureRuntimeOptions,
protocolEvents: () => LoopbackReplay["events"],
): Parameters<typeof startProcessReactiveHarness>[0]["capture"] => ({
...options,
processSamples: options.samples,
protocolEvents,
});
const startCaptureRuntime = async (
options: StartCaptureRuntimeOptions,
): Promise<StartedCaptureRuntime> => {
const { scenario } = options;
let replay: LoopbackReplay | undefined;
let renderer: TerminalRenderer | undefined;
let checkpoints: ProcessCheckpoints | undefined;
let shimReplay: CommandShimReplay | undefined;
let terminal: IPty | undefined;
let reactive: ProcessReactiveHarness | undefined;
try {
const { spawn } = await import("@lydell/node-pty");
replay = await startLoopbackReplay(scenario, options.recordEvent);
const startedAt = new Date();
const started = Date.now();
let lastOutput = started;
@@ -195,19 +130,9 @@ const startCaptureRuntime = async (
() => terminal?.pid ?? -1,
options.recordEvent,
);
checkpoints = new ProcessCheckpoints(scenario, started, options.before, {
signal: options.signal,
recordEvent: options.recordEvent,
});
shimReplay = await startCommandShimReplay(
scenario,
options.temporaryRoot,
started,
options.recordEvent,
);
terminal = spawn(scenario.executable, [...scenario.arguments], {
cwd: scenario.working_directory,
env: makeProcessCaptureEnvironment({ ...options, replay, shimReplay }),
env: makeProcessCaptureEnvironment(options),
cols: scenario.terminal.columns,
rows: scenario.terminal.rows,
name: "xterm-256color",
@@ -218,16 +143,6 @@ const startCaptureRuntime = async (
started,
onOutput: () => (lastOutput = Date.now()),
renderer,
checkpoints,
});
reactive = startProcessReactiveHarness({
capture: reactiveCapture(options, () => replay?.events ?? []),
scenario,
terminal: () => terminal,
renderer,
checkpoints,
shimReplay,
started,
});
scheduleScenarioInteractions({
...options,
@@ -244,49 +159,26 @@ const startCaptureRuntime = async (
recordEvent: options.recordEvent,
});
return {
replay,
renderer,
checkpoints,
shimReplay,
terminal,
started,
startedAt,
lastOutput: () => lastOutput,
framesTruncated,
stopSampler,
reactive,
};
} catch (cause: unknown) {
reactive?.unsubscribe();
await reactive?.coordinator.close();
return cleanupFailedStartup({
cause,
timers: options.timers,
replay,
terminal,
renderer,
shimReplay,
checkpoints,
runId: options.runId,
temporaryRoot: options.temporaryRoot,
});
}
};
const normalizedShimEvents = (
runtime: StartedCaptureRuntime,
scenario: ProcessScenario,
temporaryRoot: string,
): UnverifiedProcessCapture["shim_events"] =>
runtime.shimReplay.events.map((event) =>
normalizeProcessShimEvent(
event,
scenario,
temporaryRoot,
runtime.terminal.pid,
),
);
const finishProcessRun = async (options: {
readonly runtime: StartedCaptureRuntime | undefined;
readonly timers: Set<ProcessTimer>;
@@ -297,15 +189,11 @@ const finishProcessRun = async (options: {
readonly capture: PendingProcessCapture | undefined;
readonly executionFailure: unknown;
}): Promise<ProcessCapture> => {
options.runtime?.reactive?.unsubscribe();
await options.stopSampler();
const cleanupFailure = await releaseProcessResources({
timers: options.timers,
replay: options.runtime?.replay,
terminal: options.runtime?.terminal,
renderer: options.runtime?.renderer,
shimReplay: options.runtime?.shimReplay,
checkpoints: options.runtime?.checkpoints,
runId: options.runId,
temporaryRoot: options.temporaryRoot,
capturedProcessGroupIds:
@@ -316,11 +204,6 @@ const finishProcessRun = async (options: {
options.samples,
),
});
if (cleanupFailure !== undefined)
await options.runtime?.reactive?.coordinator.submit({
kind: "cleanup_failed",
});
await options.runtime?.reactive?.coordinator.close();
let { capture, executionFailure } = options;
let verifiedCapture: ProcessCapture | undefined;
if (capture !== undefined && cleanupFailure === undefined) {
@@ -330,7 +213,6 @@ const finishProcessRun = async (options: {
: { ...capture.settlement, cleanup_outcome: "cleaned" };
const completedCapture: UnverifiedProcessCapture = {
...capture,
reactive_run: projectProcessReactiveRun(options.runtime?.reactive),
settlement,
cleanup: {
owned_process_group: "verified",
@@ -369,45 +251,36 @@ const completeCapture = async (options: {
readonly recordEvent: RecordProcessCaptureEvent;
}): Promise<PendingProcessCapture> => {
const { runtime, scenario } = options;
runtime.checkpoints.trigger("root_exit");
const { reason } = options.exit;
if (reason === "cancelled") {
if (runtime.reactive !== undefined) {
await runtime.reactive.coordinator.submit({ kind: "cancelled" });
runtime.reactive.unsubscribe();
}
throw processCaptureCancelled();
}
if (reason === "cancelled") throw processCaptureCancelled();
const settlement = await observeSettlement(
options.runId,
selectCapturedProcessGroupIds(runtime.terminal.pid, options.samples),
scenario.settle_ms,
options.recordEvent,
);
runtime.checkpoints.trigger("settled");
const samplingPartial = (await runtime.stopSampler()).partial;
if (runtime.reactive !== undefined) {
// Process, filesystem, shim, and protocol observations can be recorded
// during settlement after the PTY exits. Make target loss terminal only
// after that journal has drained, otherwise the coordinator's control
// priority discards a valid multi-source completion predicate.
//
// Draining the coordinator is not sufficient on its own: node-pty delivers
// buffered output through onData callbacks, which run outside the
// coordinator's queue. A trailing chunk that has not been delivered yet has
// produced no journal entry, so the drain has nothing to wait for and the
// target is declared lost while a terminal trigger is still satisfiable.
// Let any trailing chunks land first.
await settleTrailingJournal(options.eventJournal);
await runtime.reactive.coordinator.drain();
await runtime.reactive.coordinator.submit({ kind: "target_lost" });
runtime.reactive.unsubscribe();
}
await settleProcessCaptureJournal(options.eventJournal);
assertNotCancelled(options.signal);
const after = await snapshotRoots(scenario, options.signal);
options.recordEvent("filesystem_checkpoints", 1);
const renderedFrames = await runtime.renderer.frames();
const checkpoints = await runtime.checkpoints.finish(after);
await runtime.replay.close();
const checkpoints: UnverifiedProcessCapture["filesystem_checkpoints"] = [
{
name: "before",
at_ms: 0,
files: options.before.files,
effects: [],
truncated: options.before.truncated,
},
{
name: "after_settlement",
at_ms: Math.max(0, Date.now() - runtime.started),
files: after.files,
effects: classifyFilesystemEffects(options.before.files, after.files),
truncated: after.truncated,
},
];
const manifest = await createRunManifest(
scenario,
runtime.startedAt,
@@ -416,8 +289,6 @@ const completeCapture = async (options: {
const truncated =
options.initiallyTruncated ||
after.truncated ||
runtime.replay.truncated ||
runtime.shimReplay.truncated ||
runtime.framesTruncated() ||
checkpoints.some(({ truncated: partial }) => partial) ||
samplingPartial ||
@@ -426,7 +297,6 @@ const completeCapture = async (options: {
frames: options.frames,
exit: { ...options.exit, reason },
samples: options.samples,
replay: runtime.replay,
before: options.before,
after,
truncated,
@@ -436,7 +306,6 @@ const completeCapture = async (options: {
renderedFrames,
interactions: options.interactions,
checkpoints,
shimEvents: normalizedShimEvents(runtime, scenario, options.temporaryRoot),
settlement,
manifest,
eventJournal: options.eventJournal,
@@ -449,7 +318,6 @@ const runProcessScenario = async (
signal?: AbortSignal,
hostEnvironment: Readonly<Record<string, string | undefined>> = process.env,
): Promise<ProcessCapture> => {
assertSupportedReactiveScenario(scenario);
const { temporaryRoot, runId, before } = await prepareProcessCapture(
scenario,
signal,
@@ -458,6 +326,7 @@ const runProcessScenario = async (
const samples: ProcessSample[] = [];
const journal = createProcessCaptureJournal();
const { entries: eventJournal, recordEvent } = journal;
recordEvent("filesystem_checkpoints", 0);
let runtime: StartedCaptureRuntime | undefined;
const timers = new Set<ProcessTimer>();
let capture: PendingProcessCapture | undefined;
@@ -472,14 +341,12 @@ const runProcessScenario = async (
hostEnvironment,
temporaryRoot,
runId,
before,
frames,
samples,
interactions,
timers,
dispatchedEventIndexes,
recordEvent,
journal,
...(signal === undefined ? {} : { signal }),
});
stopSampler = runtime.stopSampler;
src/application/ProcessNormalization.ts
-30
View File
@@ -1,8 +1,6 @@
import type {
ProcessCapture,
ProcessSample,
ProcessScenario,
ShimEvent,
} from "../domain/processCapture.js";
/** Bucket one elapsed process-capture timestamp under scenario normalization. */
@@ -88,25 +86,6 @@ export const normalizeProcessSamples = (
}));
};
/** Normalize one shim observation identically for live matching and capture. */
export const normalizeProcessShimEvent = (
event: ShimEvent,
scenario: ProcessScenario,
temporaryRoot: string,
rootPid: number,
): ShimEvent => ({
...event,
arguments: event.arguments.map((argument) =>
normalizeProcessText(argument, scenario, temporaryRoot, rootPid),
),
working_directory: normalizeProcessText(
event.working_directory,
scenario,
temporaryRoot,
rootPid,
),
});
const normalizePidTokens = (
value: string,
identifiers: readonly number[],
@@ -121,12 +100,3 @@ const normalizePidTokens = (
"<pid>",
);
};
export const normalizeProtocolEvents = (
events: readonly ProcessCapture["protocol_events"][number][],
scenario: ProcessScenario,
): readonly ProcessCapture["protocol_events"][number][] =>
events.map((event) => ({
...event,
data: normalizeProcessText(event.data, scenario, "<no-temporary-root>", -1),
}));
src/application/ProcessReactiveCoordinator.test.ts
-301
View File
@@ -1,301 +0,0 @@
import { describe, expect, it } from "vitest";
import {
ProcessReactiveCoordinator,
type ProcessReactiveTimerHost,
} from "./ProcessReactiveCoordinator.js";
import { executeProcessReactiveEffects } from "./ProcessReactiveEffects.js";
import { createProcessCaptureJournal } from "./ProcessCaptureJournal.js";
import { subscribeProcessReactiveObservations } from "./ProcessReactiveObservations.js";
import type {
FilesystemCheckpoint,
InteractionEvent,
TerminalFrame,
} from "../domain/processCapture.js";
import { parseProcessScenario } from "../domain/processScenario.js";
import {
processReactiveScenarioSchema,
type ProcessReactiveScenario,
} from "../domain/processReactiveScenario.js";
const terminalTrigger = () => ({
kind: "terminal_text" as const,
literal: "Ready",
occurrence: 1,
since: { kind: "scenario_start" as const },
consume: true,
});
const timerHost = () => {
const scheduled: Array<{
readonly callback: () => void;
readonly delayMs: number;
cancelled: boolean;
}> = [];
const host: ProcessReactiveTimerHost = {
schedule: (callback, delayMs) => {
const timer = { callback, delayMs, cancelled: false };
scheduled.push(timer);
return { cancel: () => (timer.cancelled = true) };
},
};
return { host, scheduled };
};
const twoStateScenario = (): ProcessReactiveScenario =>
processReactiveScenarioSchema.parse({
initial_state: "starting",
deadline_ms: 30000,
states: [
{
id: "starting",
max_visits: 1,
deadline_ms: 5000,
on: [
{
id: "send",
priority: 1,
max_uses: 1,
when: terminalTrigger(),
actions: [{ type: "send_input", data: "go" }],
target: { kind: "goto", state: "sent" },
},
],
},
{
id: "sent",
max_visits: 1,
deadline_ms: 5000,
on: [
{
id: "confirm",
priority: 1,
max_uses: 1,
when: {
kind: "event",
source: "interaction",
exact: { type: "input", data: "go", outcome: "dispatched" },
ignore_fields: [
"sequence",
"scheduled_at_ms",
"dispatched_at_ms",
],
since: { kind: "scenario_start" },
consume: true,
cardinality: { min: 1, max: 1 },
},
actions: [],
target: { kind: "finish" },
},
],
},
],
});
const createHarness = (scenario: ProcessReactiveScenario) => {
const journal = createProcessCaptureJournal();
const frames: TerminalFrame[] = [];
const interactions: InteractionEvent[] = [];
const checkpoints: FilesystemCheckpoint[] = [];
const terminalCalls: string[] = [];
const timers = timerHost();
let now = 100;
const effectHost = {
scenario: parseProcessScenario({ executable: "node" }),
terminal: () => ({
write: (data: string) => terminalCalls.push(`write:${data}`),
resize: (columns: number, rows: number) =>
terminalCalls.push(`resize:${String(columns)}x${String(rows)}`),
closeInput: () => terminalCalls.push("close-input"),
kill: (signal: "SIGINT" | "SIGTERM" | "SIGKILL") =>
terminalCalls.push(`kill:${signal}`),
}),
renderer: {
resize: (columns: number, rows: number) =>
terminalCalls.push(`render:${String(columns)}x${String(rows)}`),
},
checkpoints: {
captureAndRead: async (name: string) => {
const index = checkpoints.length;
const checkpoint: FilesystemCheckpoint = {
name,
at_ms: now - 100,
files: [],
effects: [],
truncated: false,
};
checkpoints.push(checkpoint);
journal.record("filesystem_checkpoints", index);
return { index, checkpoint };
},
},
interactions,
journal,
startedAtMs: 100,
now: () => now,
// Matches the processScenario default; rendered frames are normalized with
// this so they share a clock with captured frames.
timeBucketMs: 10,
};
const coordinator = new ProcessReactiveCoordinator({
scenario,
executor: {
execute: (actions, signal) =>
executeProcessReactiveEffects(effectHost, actions, signal),
},
timerHost: timers.host,
});
const unsubscribe = subscribeProcessReactiveObservations({
journal,
frames,
interactions,
checkpointAt: (index) => checkpoints[index],
processSampleAt: () => undefined,
protocolEventAt: () => undefined,
shimEventAt: () => undefined,
sink: coordinator,
});
return {
coordinator,
frames,
interactions,
checkpoints,
journal,
terminalCalls,
timers: timers.scheduled,
setNow: (value: number) => (now = value),
unsubscribe,
};
};
describe("process reactive coordinator ordering", () => {
it("commits terminal effects before evaluating their journal observations", async () => {
const harness = createHarness(twoStateScenario());
harness.frames.push({ sequence: 0, at_ms: 0, data: "Ready" });
harness.journal.record("frames", 0);
await harness.coordinator.drain();
expect(harness.coordinator.snapshot).toMatchObject({
status: "finished",
outcome: "passed",
active_state: "sent",
});
expect(
harness.coordinator.snapshot.transitions.map(
({ transition_id }) => transition_id,
),
).toEqual(["send", "confirm"]);
expect(harness.terminalCalls).toEqual(["write:go"]);
expect(harness.journal.entries).toEqual([
{ collection: "frames", index: 0, capture_order: 0 },
{ collection: "interaction_events", index: 0, capture_order: 1 },
]);
expect(harness.interactions[0]).toMatchObject({
type: "input",
data: "go",
outcome: "dispatched",
});
harness.unsubscribe();
await harness.coordinator.close();
});
it("executes input, resize, root signal, and checkpoint in journal order", async () => {
const scenario = processReactiveScenarioSchema.parse({
initial_state: "ready",
deadline_ms: 30000,
states: [
{
id: "ready",
max_visits: 1,
deadline_ms: 5000,
on: [
{
id: "act",
priority: 1,
max_uses: 1,
when: terminalTrigger(),
actions: [
{ type: "send_input", data: "secret", sensitive: true },
{ type: "resize", columns: 100, rows: 40 },
{ type: "close_stdin" },
{
type: "send_signal",
target: { kind: "root" },
signal: "SIGINT",
},
{ type: "checkpoint", name: "stopped" },
],
target: { kind: "finish" },
},
],
},
],
});
const harness = createHarness(scenario);
harness.setNow(125);
harness.frames.push({ sequence: 0, at_ms: 20, data: "Ready" });
harness.journal.record("frames", 0);
await harness.coordinator.drain();
expect(harness.terminalCalls).toEqual([
"write:secret",
"resize:100x40",
"render:100x40",
"close-input",
"kill:SIGINT",
]);
expect(harness.interactions.map(({ data }) => data)).toEqual([
"<redacted-input:6-bytes>",
"100x40",
"",
"SIGINT",
]);
expect(harness.checkpoints).toHaveLength(1);
expect(
harness.coordinator.snapshot.transitions[0]?.action_event_ids,
).toEqual([
"obs.interaction_events.0",
"obs.interaction_events.1",
"obs.interaction_events.2",
"obs.interaction_events.3",
"obs.filesystem_checkpoints.0",
]);
await harness.coordinator.close();
});
it("ignores a stale state timer after a transition", async () => {
const base = twoStateScenario();
const first = base.states[0];
const second = base.states[1];
if (first === undefined || second === undefined)
throw new Error("two-state fixture is incomplete");
const scenario = processReactiveScenarioSchema.parse({
...base,
states: [
{
...first,
on: first.on.map((transition) => ({
...transition,
actions: [],
})),
},
{
...second,
on: second.on.map((transition) => ({
...transition,
when: { ...terminalTrigger(), literal: "Later" },
})),
},
],
});
const harness = createHarness(scenario);
const oldStateTimer = harness.timers[1];
expect(oldStateTimer?.delayMs).toBe(5000);
harness.frames.push({ sequence: 0, at_ms: 0, data: "Ready" });
harness.journal.record("frames", 0);
await harness.coordinator.drain();
expect(oldStateTimer?.cancelled).toBe(true);
oldStateTimer?.callback();
await harness.coordinator.drain();
expect(harness.coordinator.snapshot).toMatchObject({
status: "running",
active_state: "sent",
outcome: null,
});
const currentStateTimer = harness.timers[2];
currentStateTimer?.callback();
await harness.coordinator.drain();
expect(harness.coordinator.snapshot.outcome).toBe("predicate_timeout");
await harness.coordinator.close();
});
});
src/application/ProcessReactiveCoordinator.ts
-300
View File
@@ -1,300 +0,0 @@
import {
commitProcessReactiveProposal,
createProcessReactiveSnapshot,
reduceProcessReactiveScenario,
type ProcessReactiveDecision,
type ProcessReactiveEffectResult,
type ProcessReactiveInput,
type ProcessReactiveSnapshot,
} from "../domain/processReactiveRuntime.js";
import type {
ProcessReactiveAction,
ProcessReactiveScenario,
} from "../domain/processReactiveScenario.js";
import { scheduleProcessDelay } from "./ProcessTimer.js";
/** Effect boundary owned by the serialized reactive coordinator. */
export interface ProcessReactiveExecutor {
execute(
actions: readonly ProcessReactiveAction[],
signal: AbortSignal,
): Promise<readonly ProcessReactiveEffectResult[]>;
}
/** Cancel handle for one coordinator-owned deadline. */
export type ProcessReactiveTimer = ReturnType<typeof scheduleProcessDelay>;
/** Injectable deadline scheduler used by deterministic coordinator tests. */
export interface ProcessReactiveTimerHost {
readonly schedule: (
callback: () => void,
delayMs: number,
) => ProcessReactiveTimer;
}
const systemTimerHost: ProcessReactiveTimerHost = {
schedule: (callback, delayMs) => scheduleProcessDelay(delayMs, callback),
};
/**
* Serializes live observations, deadlines, and effect commits for one run.
* Producer callbacks enqueue work without awaiting it, preventing reentrancy.
*/
export class ProcessReactiveCoordinator {
readonly #scenario: ProcessReactiveScenario;
readonly #executor: ProcessReactiveExecutor;
readonly #timerHost: ProcessReactiveTimerHost;
readonly #now: () => number;
readonly #scenarioDeadlineAt: number;
readonly #onDecision:
| ((decision: ProcessReactiveDecision, input: ProcessReactiveInput) => void)
| undefined;
#snapshot: ProcessReactiveSnapshot;
readonly #pendingInputs: ProcessReactiveInput[] = [];
#tail: Promise<void> = Promise.resolve();
#running = false;
#failure: unknown;
#stateTimer: ProcessReactiveTimer | undefined;
#scenarioTimer: ProcessReactiveTimer | undefined;
#stateDeadlineAt = 0;
#effectAbort: AbortController | undefined;
#accepting = true;
#closed = false;
constructor(options: {
readonly scenario: ProcessReactiveScenario;
readonly executor: ProcessReactiveExecutor;
readonly timerHost?: ProcessReactiveTimerHost;
readonly now?: () => number;
readonly onDecision?: (
decision: ProcessReactiveDecision,
input: ProcessReactiveInput,
) => void;
}) {
this.#scenario = options.scenario;
this.#executor = options.executor;
this.#timerHost = options.timerHost ?? systemTimerHost;
this.#now = options.now ?? Date.now;
this.#scenarioDeadlineAt = this.#now() + options.scenario.deadline_ms;
this.#onDecision = options.onDecision;
this.#snapshot = createProcessReactiveSnapshot(options.scenario);
this.#scheduleScenarioDeadline();
this.#scheduleStateDeadline();
}
/** Current immutable reducer snapshot after all completed turns. */
get snapshot(): ProcessReactiveSnapshot {
return this.#snapshot;
}
/** Enqueue producer work without creating a floating promise. */
enqueue(input: ProcessReactiveInput): void {
if (!this.#accepting) return;
if (this.#interruptsEffects(input)) this.#effectAbort?.abort();
this.#enqueueOwned(input);
}
/** Submit one input and wait until every turn queued before it completes. */
async submit(input: ProcessReactiveInput): Promise<ProcessReactiveSnapshot> {
this.enqueue(input);
await this.drain();
return this.#snapshot;
}
/** Wait for currently queued reducer/effect work and surface host failures. */
async drain(): Promise<void> {
for (;;) {
const observedTail = this.#tail;
await observedTail;
if (observedTail === this.#tail) break;
}
if (this.#failure !== undefined) throw this.#failure;
}
/** Stop owned deadlines after queued work settles. */
async close(): Promise<void> {
this.#accepting = false;
this.#clearTimers();
await this.drain();
this.#closed = true;
}
#enqueueOwned(input: ProcessReactiveInput): void {
if (this.#closed || this.#failure !== undefined) return;
this.#pendingInputs.push(input);
this.#startWorker();
}
#startWorker(): void {
if (this.#running || this.#pendingInputs.length === 0) return;
this.#running = true;
this.#tail = this.#runQueue()
.catch((cause: unknown) => this.#fail(cause))
.finally(() => {
this.#running = false;
if (this.#pendingInputs.length > 0) this.#startWorker();
});
}
async #runQueue(): Promise<void> {
for (;;) {
const input = this.#nextInput();
if (input === undefined) return;
await this.#advance(input);
}
}
#nextInput(): ProcessReactiveInput | undefined {
const terminalIndex = this.#pendingInputs.findIndex(
({ kind }) => kind === "cancelled" || kind === "cleanup_failed",
);
if (terminalIndex >= 0)
return this.#pendingInputs.splice(terminalIndex, 1)[0];
let observationIndex = -1;
let captureOrder = Number.POSITIVE_INFINITY;
for (const [index, input] of this.#pendingInputs.entries())
if (
input.kind === "observation" &&
input.observation.capture_order < captureOrder
) {
observationIndex = index;
captureOrder = input.observation.capture_order;
}
return observationIndex >= 0
? this.#pendingInputs.splice(observationIndex, 1)[0]
: this.#pendingInputs.shift();
}
#fail(cause: unknown): void {
this.#failure ??= cause;
this.#accepting = false;
this.#closed = true;
this.#effectAbort?.abort();
this.#pendingInputs.length = 0;
this.#clearTimers();
}
async #advance(input: ProcessReactiveInput): Promise<void> {
const before = this.#snapshot;
let decision = reduceProcessReactiveScenario(this.#scenario, before, input);
let effectResults: readonly ProcessReactiveEffectResult[] = [];
if (decision.kind === "proposal") {
const effectAbort = new AbortController();
this.#effectAbort = effectAbort;
try {
effectResults = await this.#executor.execute(
decision.effects,
effectAbort.signal,
);
} catch (cause: unknown) {
if (effectAbort.signal.aborted) return;
throw cause;
} finally {
if (this.#effectAbort === effectAbort) this.#effectAbort = undefined;
}
if (effectAbort.signal.aborted) return;
const expired = this.#expiredDeadlineInput();
if (expired !== undefined) {
this.#enqueueOwned(expired);
return;
}
decision = commitProcessReactiveProposal(
this.#scenario,
decision,
effectResults,
);
}
this.#snapshot = decision.snapshot;
this.#onDecision?.(decision, input);
if (this.#snapshot.status === "finished") this.#clearTimers();
else if (
this.#accepting &&
(before.active_state !== this.#snapshot.active_state ||
before.state_entry_capture_order !==
this.#snapshot.state_entry_capture_order)
)
this.#scheduleStateDeadline();
for (const result of effectResults)
if (
result.status === "succeeded" &&
!this.#hasObservation(result.observation.event_id)
)
this.#enqueueOwned({
kind: "observation",
observation: result.observation,
});
}
#hasObservation(eventId: string): boolean {
return (
this.#snapshot.observations.some(
(observation) => observation.event_id === eventId,
) ||
this.#pendingInputs.some(
(input) =>
input.kind === "observation" &&
input.observation.event_id === eventId,
)
);
}
#interruptsEffects(input: ProcessReactiveInput): boolean {
if (
input.kind === "cancelled" ||
input.kind === "cleanup_failed" ||
input.kind === "scenario_deadline" ||
input.kind === "target_lost"
)
return true;
return (
input.kind === "state_deadline" &&
input.state_id === this.#snapshot.active_state &&
input.state_entry_capture_order ===
this.#snapshot.state_entry_capture_order
);
}
#expiredDeadlineInput(): ProcessReactiveInput | undefined {
const now = this.#now();
if (now >= this.#scenarioDeadlineAt) return { kind: "scenario_deadline" };
if (now < this.#stateDeadlineAt) return undefined;
return {
kind: "state_deadline",
state_id: this.#snapshot.active_state,
state_entry_capture_order: this.#snapshot.state_entry_capture_order,
};
}
#scheduleScenarioDeadline(): void {
if (!this.#accepting) return;
this.#scenarioTimer = this.#timerHost.schedule(
() => this.enqueue({ kind: "scenario_deadline" }),
this.#scenario.deadline_ms,
);
}
#scheduleStateDeadline(): void {
if (!this.#accepting) return;
this.#stateTimer?.cancel();
const state = this.#scenario.states.find(
({ id }) => id === this.#snapshot.active_state,
);
if (state === undefined) return;
this.#stateDeadlineAt = this.#now() + state.deadline_ms;
const identity = {
state_id: state.id,
state_entry_capture_order: this.#snapshot.state_entry_capture_order,
};
this.#stateTimer = this.#timerHost.schedule(
() => this.enqueue({ kind: "state_deadline", ...identity }),
state.deadline_ms,
);
}
#clearTimers(): void {
this.#stateTimer?.cancel();
this.#scenarioTimer?.cancel();
this.#stateTimer = undefined;
this.#scenarioTimer = undefined;
}
}
src/application/ProcessReactiveCoordinatorFailures.fixture.ts
-77
View File
@@ -1,77 +0,0 @@
import type { ProcessReactiveTimerHost } from "./ProcessReactiveCoordinator.js";
import {
processReactiveScenarioSchema,
type ProcessReactiveScenario,
} from "../domain/processReactiveScenario.js";
export const terminalTrigger = () => ({
kind: "terminal_text" as const,
literal: "Ready",
occurrence: 1,
since: { kind: "scenario_start" as const },
consume: true,
});
export const timerHost = () => {
const scheduled: Array<{
readonly callback: () => void;
readonly delayMs: number;
cancelled: boolean;
}> = [];
const host: ProcessReactiveTimerHost = {
schedule: (callback, delayMs) => {
const timer = { callback, delayMs, cancelled: false };
scheduled.push(timer);
return { cancel: () => (timer.cancelled = true) };
},
};
return { host, scheduled };
};
export const twoStateScenario = (): ProcessReactiveScenario =>
processReactiveScenarioSchema.parse({
initial_state: "starting",
deadline_ms: 30000,
states: [
{
id: "starting",
max_visits: 1,
deadline_ms: 5000,
on: [
{
id: "send",
priority: 1,
max_uses: 1,
when: terminalTrigger(),
actions: [{ type: "send_input", data: "go" }],
target: { kind: "goto", state: "sent" },
},
],
},
{
id: "sent",
max_visits: 1,
deadline_ms: 5000,
on: [
{
id: "confirm",
priority: 1,
max_uses: 1,
when: {
kind: "event",
source: "interaction",
exact: { type: "input", data: "go", outcome: "dispatched" },
ignore_fields: [
"sequence",
"scheduled_at_ms",
"dispatched_at_ms",
],
since: { kind: "scenario_start" },
consume: true,
cardinality: { min: 1, max: 1 },
},
actions: [],
target: { kind: "finish" },
},
],
},
],
});
src/application/ProcessReactiveCoordinatorFailures.test.ts
-198
View File
@@ -1,198 +0,0 @@
import { describe, expect, it } from "vitest";
import { ProcessReactiveCoordinator } from "./ProcessReactiveCoordinator.js";
import { unsupportedProcessReactiveFeatures } from "./ProcessReactiveEffects.js";
import { createProcessObservation } from "../domain/processObservation.js";
import type { ProcessReactiveEffectResult } from "../domain/processReactiveRuntime.js";
import { processReactiveScenarioSchema } from "../domain/processReactiveScenario.js";
import {
timerHost,
twoStateScenario,
} from "./ProcessReactiveCoordinatorFailures.fixture.js";
describe("process reactive coordinator failures: host preflight and ordering", () => {
it("preflights signal selectors the host cannot execute", () => {
const input = twoStateScenario();
const first = input.states[0];
const second = input.states[1];
const transition = first?.on[0];
if (first === undefined || second === undefined || transition === undefined)
throw new Error("two-state fixture is incomplete");
const scenario = processReactiveScenarioSchema.parse({
...input,
states: [
{
...first,
on: [
{
...transition,
actions: [
{
type: "send_signal",
target: { kind: "process_group" },
signal: "SIGTERM",
},
],
},
],
},
second,
],
});
expect(unsupportedProcessReactiveFeatures(scenario)).toEqual([
"send:target:process_group",
]);
});
it("orders delayed effect observations before later producer records", async () => {
const timers = timerHost();
let executions = 0;
let resolveEffects:
| ((results: readonly ProcessReactiveEffectResult[]) => void)
| undefined;
const effects = new Promise<readonly ProcessReactiveEffectResult[]>(
(resolve) => (resolveEffects = resolve),
);
const coordinator = new ProcessReactiveCoordinator({
scenario: twoStateScenario(),
executor: {
execute: () => {
executions += 1;
return executions === 1 ? effects : Promise.resolve([]);
},
},
timerHost: timers.host,
});
coordinator.enqueue({
kind: "observation",
observation: createProcessObservation({
source: "terminal_raw",
source_sequence: 0,
captured_at_ms: 0,
subject_id: null,
location: { collection: "frames", index: 0, capture_order: 0 },
payload: { sequence: 0, at_ms: 0, data: "Ready" },
}),
});
await Promise.resolve();
coordinator.enqueue({
kind: "observation",
observation: createProcessObservation({
source: "terminal_raw",
source_sequence: 1,
captured_at_ms: 2,
subject_id: null,
location: { collection: "frames", index: 1, capture_order: 2 },
payload: { sequence: 1, at_ms: 2, data: "later" },
}),
});
if (resolveEffects === undefined)
throw new Error("effect executor did not start");
resolveEffects([
{
status: "succeeded",
observation: createProcessObservation({
source: "interaction",
source_sequence: 0,
captured_at_ms: 1,
subject_id: null,
location: {
collection: "interaction_events",
index: 0,
capture_order: 1,
},
payload: {
sequence: 0,
scheduled_at_ms: 1,
dispatched_at_ms: 1,
type: "input",
data: "go",
outcome: "dispatched",
},
}),
},
]);
await coordinator.drain();
expect(coordinator.snapshot).toMatchObject({
status: "finished",
outcome: "passed",
});
expect(
coordinator.snapshot.transitions.map(
({ transition_id }) => transition_id,
),
).toEqual(["send", "confirm"]);
await coordinator.close();
});
it("cancels timers and rejects later work after an executor failure", async () => {
const timers = timerHost();
let calls = 0;
const coordinator = new ProcessReactiveCoordinator({
scenario: twoStateScenario(),
executor: {
execute: () => {
calls += 1;
throw new Error("executor failed");
},
},
timerHost: timers.host,
});
const ready = createProcessObservation({
source: "terminal_raw",
source_sequence: 0,
captured_at_ms: 0,
subject_id: null,
location: { collection: "frames", index: 0, capture_order: 0 },
payload: { sequence: 0, at_ms: 0, data: "Ready" },
});
coordinator.enqueue({ kind: "observation", observation: ready });
await expect(coordinator.drain()).rejects.toThrow("executor failed");
coordinator.enqueue({ kind: "observation", observation: ready });
expect(calls).toBe(1);
expect(timers.scheduled.every(({ cancelled }) => cancelled)).toBe(true);
});
});
describe("process reactive coordinator failures: effect deadlines", () => {
it("aborts an in-flight effect before committing a state deadline", async () => {
const timers = timerHost();
let effectAborted = false;
const coordinator = new ProcessReactiveCoordinator({
scenario: twoStateScenario(),
executor: {
execute: (_actions, signal) =>
new Promise((resolve) => {
signal.addEventListener(
"abort",
() => {
effectAborted = true;
resolve([]);
},
{ once: true },
);
}),
},
timerHost: timers.host,
});
coordinator.enqueue({
kind: "observation",
observation: createProcessObservation({
source: "terminal_raw",
source_sequence: 0,
captured_at_ms: 0,
subject_id: null,
location: { collection: "frames", index: 0, capture_order: 0 },
payload: { sequence: 0, at_ms: 0, data: "Ready" },
}),
});
await Promise.resolve();
timers.scheduled[1]?.callback();
await coordinator.drain();
expect(effectAborted).toBe(true);
expect(coordinator.snapshot).toMatchObject({
status: "finished",
outcome: "predicate_timeout",
active_state: "starting",
transitions: [],
});
await coordinator.close();
});
});
src/application/ProcessReactiveEffects.ts
-254
View File
@@ -1,254 +0,0 @@
import type {
FilesystemCheckpoint,
InteractionEvent,
} from "../domain/processCapture.js";
import { createProcessObservation } from "../domain/processObservation.js";
import type { ProcessReactiveEffectResult } from "../domain/processReactiveRuntime.js";
import type {
ProcessReactiveAction,
ProcessReactiveScenario,
} from "../domain/processReactiveScenario.js";
import type { ProcessScenario } from "../domain/processScenario.js";
import type { ProcessCaptureJournal } from "./ProcessCaptureJournal.js";
import {
normalizeProcessElapsedTime,
normalizeProcessText,
} from "./ProcessNormalization.js";
/** Minimal PTY authority used by the admitted reactive action slice. */
export interface ProcessReactiveTerminal {
write(data: string): void;
resize(columns: number, rows: number): void;
closeInput(): void;
kill(signal: "SIGINT" | "SIGTERM" | "SIGKILL"): void;
}
/** Rendered-terminal resize seam kept in lockstep with the PTY. */
export interface ProcessReactiveRenderer {
resize(columns: number, rows: number, atMs: number): void;
}
/** Awaited filesystem checkpoint seam used by reactive effects. */
export interface ProcessReactiveCheckpoints {
captureAndRead(
name: string,
signal?: AbortSignal,
): Promise<{
readonly index: number;
readonly checkpoint: FilesystemCheckpoint;
}>;
}
/** Host dependencies for journal-backed reactive process actions. */
export interface ProcessReactiveEffectHost {
readonly scenario: ProcessScenario;
readonly terminal: () => ProcessReactiveTerminal | undefined;
readonly renderer: ProcessReactiveRenderer;
readonly checkpoints: ProcessReactiveCheckpoints;
readonly interactions: InteractionEvent[];
readonly journal: ProcessCaptureJournal;
readonly startedAtMs: number;
readonly now: () => number;
/** Time bucket the scenario normalizes rendered-frame timestamps with. */
readonly timeBucketMs: number;
}
const elapsed = (host: ProcessReactiveEffectHost): number =>
Math.max(0, host.now() - host.startedAtMs);
const interactionData = (
action: Exclude<ProcessReactiveAction, { readonly type: "checkpoint" }>,
scenario: ProcessScenario,
): string => {
if (action.type === "send_input")
return action.sensitive
? `<redacted-input:${String(Buffer.byteLength(action.data))}-bytes>`
: normalizeProcessText(action.data, scenario, "<no-temporary-root>", -1);
if (action.type === "resize")
return `${String(action.columns)}x${String(action.rows)}`;
if (action.type === "close_stdin") return "";
return action.signal;
};
const interactionType = (
action: Exclude<ProcessReactiveAction, { readonly type: "checkpoint" }>,
): InteractionEvent["type"] => {
if (action.type === "send_input") return "input";
if (action.type === "resize") return "resize";
if (action.type === "close_stdin") return "stdin_close";
return "signal";
};
const performTerminalAction = (
host: Pick<ProcessReactiveEffectHost, "renderer" | "timeBucketMs">,
terminal: ProcessReactiveTerminal,
action: Exclude<ProcessReactiveAction, { readonly type: "checkpoint" }>,
atMs: number,
): void => {
if (action.type === "send_input") terminal.write(action.data);
else if (action.type === "resize") {
terminal.resize(action.columns, action.rows);
// Rendered frames share the normalized clock used by captured terminal
// frames and by the scheduled-event resize path. Passing the raw elapsed
// time here stamped this frame on a different clock, so the next bucketed
// frame could appear to move backwards and fail capture validation with
// "rendered_frames: timestamps must be ordered".
host.renderer.resize(
action.columns,
action.rows,
normalizeProcessElapsedTime(atMs, host.timeBucketMs),
);
} else if (action.type === "close_stdin") terminal.closeInput();
else terminal.kill(action.signal);
};
const recordInteraction = (
host: ProcessReactiveEffectHost,
action: Exclude<ProcessReactiveAction, { readonly type: "checkpoint" }>,
atMs: number,
outcome: InteractionEvent["outcome"],
): ProcessReactiveEffectResult => {
const index = host.interactions.length;
const event: InteractionEvent = {
sequence: index,
scheduled_at_ms: atMs,
dispatched_at_ms: atMs,
type: interactionType(action),
data: interactionData(action, host.scenario),
outcome,
};
host.interactions.push(event);
const location = host.journal.record("interaction_events", index);
if (outcome !== "dispatched") return { status: "rejected" };
return {
status: "succeeded",
observation: createProcessObservation({
source: "interaction",
source_sequence: index,
captured_at_ms: atMs,
subject_id: null,
location,
payload: event,
}),
};
};
const executeTerminalAction = (
host: ProcessReactiveEffectHost,
action: Exclude<ProcessReactiveAction, { readonly type: "checkpoint" }>,
): ProcessReactiveEffectResult => {
if (action.type === "send_signal" && action.target.kind !== "root")
return { status: "rejected" };
const terminal = host.terminal();
if (terminal === undefined) return { status: "target_lost" };
const atMs = elapsed(host);
let outcome: InteractionEvent["outcome"] = "dispatched";
try {
performTerminalAction(host, terminal, action, atMs);
} catch {
outcome = "failed";
}
return recordInteraction(host, action, atMs, outcome);
};
const executeCheckpoint = async (
host: ProcessReactiveEffectHost,
action: Extract<ProcessReactiveAction, { readonly type: "checkpoint" }>,
signal: AbortSignal,
): Promise<ProcessReactiveEffectResult> => {
try {
const { index, checkpoint } = await host.checkpoints.captureAndRead(
action.name,
signal,
);
const location = host.journal.entries.findLast(
(entry) =>
entry.collection === "filesystem_checkpoints" && entry.index === index,
);
if (location === undefined) return { status: "rejected" };
return {
status: "succeeded",
observation: createProcessObservation({
source: "filesystem",
source_sequence: index,
captured_at_ms: checkpoint.at_ms,
subject_id: `checkpoint:${checkpoint.name}`,
location,
payload: checkpoint,
}),
};
} catch {
return { status: "rejected" };
}
};
/** Execute proposed effects in order and stop after the first failure. */
export const executeProcessReactiveEffects = async (
host: ProcessReactiveEffectHost,
actions: readonly ProcessReactiveAction[],
signal: AbortSignal,
): Promise<readonly ProcessReactiveEffectResult[]> => {
const results: ProcessReactiveEffectResult[] = [];
for (const action of actions) {
if (signal.aborted) return results;
const result =
action.type === "checkpoint"
? await executeCheckpoint(host, action, signal)
: executeTerminalAction(host, action);
results.push(result);
if (result.status !== "succeeded") break;
}
return results;
};
const triggerSources = (
trigger: ProcessReactiveScenario["states"][number]["on"][number]["when"],
): readonly string[] => {
if (trigger.kind === "terminal_text") return [];
if (trigger.kind === "event")
return [
"terminal_raw",
"interaction",
"process",
"filesystem",
"http",
"websocket",
"shim",
].includes(trigger.source)
? []
: [trigger.source];
return trigger.kind === "repeat"
? triggerSources(trigger.trigger)
: trigger.triggers.flatMap(triggerSources);
};
/** Return unsupported sources, selectors, and checkpoint collisions pre-launch. */
export const unsupportedProcessReactiveFeatures = (
scenario: ProcessReactiveScenario,
reservedCheckpointNames: ReadonlySet<string> = new Set([
"before",
"after_settlement",
]),
): readonly string[] => [
...scenario.states.flatMap((state) =>
state.on.flatMap((transition) =>
triggerSources(transition.when).map(
(source) => `${transition.id}:source:${source}`,
),
),
),
...scenario.states.flatMap((state) =>
state.on.flatMap((transition) =>
transition.actions.flatMap((action) => {
if (action.type === "send_signal" && action.target.kind !== "root")
return [`${transition.id}:target:${action.target.kind}`];
if (
action.type === "checkpoint" &&
reservedCheckpointNames.has(action.name)
)
return [`${transition.id}:checkpoint:${action.name}`];
return [];
}),
),
),
];
src/application/ProcessReactiveHarness.ts
-185
View File
@@ -1,185 +0,0 @@
import type { IPty } from "@lydell/node-pty";
import type {
InteractionEvent,
ProcessCapture,
ProcessReactiveControlRecord,
ProcessSample,
ProcessScenario,
ProtocolEvent,
TerminalFrame,
} from "../domain/processCapture.js";
import type { ProcessReactiveInput } from "../domain/processReactiveRuntime.js";
import { ProcessCaptureError } from "./ProcessCaptureError.js";
import type { ProcessCaptureJournal } from "./ProcessCaptureJournal.js";
import type { ProcessCheckpoints } from "./ProcessCheckpoints.js";
import type { CommandShimReplay } from "./CommandShimReplay.js";
import { ProcessReactiveCoordinator } from "./ProcessReactiveCoordinator.js";
import {
executeProcessReactiveEffects,
type ProcessReactiveTerminal,
unsupportedProcessReactiveFeatures,
} from "./ProcessReactiveEffects.js";
import { subscribeProcessReactiveObservations } from "./ProcessReactiveObservations.js";
import {
normalizeProcessSamples,
normalizeProcessShimEvent,
normalizeProtocolEvents,
} from "./ProcessNormalization.js";
import type { TerminalRenderer } from "./TerminalRenderer.js";
/** Coordinator and subscription owned by one live process capture. */
export interface ProcessReactiveHarness {
readonly coordinator: ProcessReactiveCoordinator;
readonly controls: readonly ProcessReactiveControlRecord[];
readonly unsubscribe: () => void;
}
const reactiveTerminalFor = (
terminal: IPty | undefined,
): ProcessReactiveTerminal | undefined => {
if (terminal === undefined) return undefined;
return {
write: (data) => terminal.write(data),
resize: (columns, rows) => terminal.resize(columns, rows),
closeInput: () => {
const end: unknown = Reflect.get(terminal, "end");
if (typeof end !== "function")
throw new ProcessCaptureError(
"PTY backend does not expose an stdin-close operation",
);
Reflect.apply(end, terminal, []);
},
kill: (signal) => terminal.kill(signal),
};
};
const controlKind = (
input: ProcessReactiveInput,
): ProcessReactiveControlRecord["kind"] | undefined =>
input.kind === "observation" ? undefined : input.kind;
/** Reject reactive declarations the current process adapter cannot execute. */
export const assertSupportedReactiveScenario = (
scenario: ProcessScenario,
): void => {
if (scenario.reactive === null) return;
const unsupported = unsupportedProcessReactiveFeatures(scenario.reactive);
if (unsupported.length === 0) return;
throw new ProcessCaptureError(
`reactive scenario requests unsupported features: ${unsupported.join(", ")}`,
);
};
/**
* Attach the serialized reactive runtime once the target PID is available.
*
* Subscription replays journal entries admitted during launcher startup before
* observing new records, so PID-dependent normalization cannot drop them.
*/
export const startProcessReactiveHarness = (options: {
readonly scenario: ProcessScenario;
readonly terminal: () => IPty | undefined;
readonly renderer: TerminalRenderer;
readonly checkpoints: ProcessCheckpoints;
readonly shimReplay: CommandShimReplay;
readonly started: number;
readonly capture: {
readonly interactions: InteractionEvent[];
readonly journal: ProcessCaptureJournal;
readonly frames: readonly TerminalFrame[];
readonly processSamples: readonly ProcessSample[];
readonly protocolEvents: () => readonly ProtocolEvent[];
readonly temporaryRoot: string;
};
}): ProcessReactiveHarness | undefined => {
if (options.scenario.reactive === null) return undefined;
const controls: ProcessReactiveControlRecord[] = [];
const coordinator = new ProcessReactiveCoordinator({
scenario: options.scenario.reactive,
executor: {
execute: (actions, signal) =>
executeProcessReactiveEffects(
{
scenario: options.scenario,
terminal: () => reactiveTerminalFor(options.terminal()),
renderer: options.renderer,
checkpoints: options.checkpoints,
interactions: options.capture.interactions,
journal: options.capture.journal,
startedAtMs: options.started,
now: Date.now,
timeBucketMs: options.scenario.normalization.time_bucket_ms,
},
actions,
signal,
),
},
onDecision: (decision, input) => {
const kind = controlKind(input);
if (kind === undefined || decision.kind !== "finished") return;
controls.push({
sequence: controls.length,
kind,
after_capture_order:
decision.snapshot.observations.at(-1)?.capture_order ?? -1,
});
},
});
return {
coordinator,
controls,
unsubscribe: subscribeProcessReactiveObservations({
journal: options.capture.journal,
frames: options.capture.frames,
interactions: options.capture.interactions,
checkpointAt: (index) => options.checkpoints.at(index),
processSampleAt: (index) => {
const rootPid = options.terminal()?.pid;
return rootPid === undefined
? undefined
: normalizeProcessSamples(
options.capture.processSamples,
options.scenario,
rootPid,
)[index];
},
protocolEventAt: (index) =>
normalizeProtocolEvents(
options.capture.protocolEvents().slice(index, index + 1),
options.scenario,
)[0],
shimEventAt: (index) => {
const event = options.shimReplay.events[index];
const rootPid = options.terminal()?.pid;
return event === undefined || rootPid === undefined
? undefined
: normalizeProcessShimEvent(
event,
options.scenario,
options.capture.temporaryRoot,
rootPid,
);
},
sink: coordinator,
}),
};
};
/** Project internal reducer state into the stable Process Capture shape. */
export const projectProcessReactiveRun = (
harness: ProcessReactiveHarness | undefined,
): ProcessCapture["reactive_run"] => {
if (harness === undefined) return null;
const snapshot = harness.coordinator.snapshot;
const completion =
snapshot.status === "running"
? { status: snapshot.status, outcome: null }
: { status: snapshot.status, outcome: snapshot.outcome };
return {
...completion,
active_state: snapshot.active_state,
transitions: snapshot.transitions,
controls: harness.controls,
};
};
src/application/ProcessReactiveObservations.ts
-164
View File
@@ -1,164 +0,0 @@
import type {
InteractionEvent,
FilesystemCheckpoint,
ProcessCaptureEventJournalEntry,
ProcessSample,
ProtocolEvent,
ShimEvent,
TerminalFrame,
} from "../domain/processCapture.js";
import {
createProcessObservation,
processObservationSubject,
type ProcessObservation,
} from "../domain/processObservation.js";
import type { ProcessReactiveInput } from "../domain/processReactiveRuntime.js";
import type { ProcessCaptureJournal } from "./ProcessCaptureJournal.js";
/** Non-blocking input seam implemented by the reactive coordinator. */
export interface ProcessReactiveObservationSink {
enqueue(input: ProcessReactiveInput): void;
}
interface ProcessReactiveObservationOptions {
readonly journal: ProcessCaptureJournal;
readonly frames: readonly TerminalFrame[];
readonly interactions: readonly InteractionEvent[];
readonly checkpointAt: (index: number) => FilesystemCheckpoint | undefined;
readonly processSampleAt: (index: number) => ProcessSample | undefined;
readonly protocolEventAt: (index: number) => ProtocolEvent | undefined;
readonly shimEventAt: (index: number) => ShimEvent | undefined;
readonly sink: ProcessReactiveObservationSink;
}
const terminalObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const value = options.frames[entry.index];
return value === undefined
? undefined
: createProcessObservation({
source: "terminal_raw",
source_sequence: value.sequence,
captured_at_ms: value.at_ms,
subject_id: null,
location: entry,
payload: value,
});
};
const interactionObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const value = options.interactions[entry.index];
return value === undefined
? undefined
: createProcessObservation({
source: "interaction",
source_sequence: value.sequence,
captured_at_ms: value.dispatched_at_ms,
subject_id: null,
location: entry,
payload: value,
});
};
const filesystemObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const value = options.checkpointAt(entry.index);
return value === undefined
? undefined
: createProcessObservation({
source: "filesystem",
source_sequence: entry.index,
captured_at_ms: value.at_ms,
subject_id: processObservationSubject("filesystem", value),
location: entry,
payload: value,
});
};
const processObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const value = options.processSampleAt(entry.index);
return value === undefined
? undefined
: createProcessObservation({
source: "process",
source_sequence: entry.index,
captured_at_ms: value.at_ms,
subject_id: processObservationSubject("process", value),
location: entry,
payload: value,
});
};
const protocolObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const value = options.protocolEventAt(entry.index);
return value === undefined
? undefined
: createProcessObservation({
source: value.protocol,
source_sequence: value.sequence,
captured_at_ms: value.at_ms,
subject_id: processObservationSubject(value.protocol, value),
location: entry,
payload: value,
});
};
const shimObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const normalized = options.shimEventAt(entry.index);
if (normalized === undefined) return undefined;
return createProcessObservation({
source: "shim",
source_sequence: normalized.sequence,
captured_at_ms: normalized.at_ms,
subject_id: processObservationSubject("shim", normalized),
location: entry,
payload: normalized,
});
};
const observationFor = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
if (entry.collection === "frames") return terminalObservation(options, entry);
if (entry.collection === "interaction_events")
return interactionObservation(options, entry);
if (entry.collection === "filesystem_checkpoints")
return filesystemObservation(options, entry);
if (entry.collection === "process_samples")
return processObservation(options, entry);
if (entry.collection === "protocol_events")
return protocolObservation(options, entry);
if (entry.collection === "shim_events")
return shimObservation(options, entry);
return undefined;
};
/** Subscribe the reactive coordinator to admitted normalized journal records. */
export const subscribeProcessReactiveObservations = (
options: ProcessReactiveObservationOptions,
): (() => void) => {
const enqueue = (entry: ProcessCaptureEventJournalEntry): void => {
const observation = observationFor(options, entry);
if (observation !== undefined)
options.sink.enqueue({ kind: "observation", observation });
};
for (const entry of options.journal.entries) enqueue(entry);
return options.journal.subscribe(enqueue);
};
src/application/ProcessTimer.test.ts
-12
View File
@@ -1,6 +1,5 @@
import { describe, expect, it } from "vitest";
import { ReplayDelayScheduler } from "./LoopbackReplayRecorder.js";
import { scheduleProcessDelay } from "./ProcessTimer.js";
describe("process delay scheduling", () => {
@@ -36,15 +35,4 @@ describe("process delay scheduling", () => {
scheduled[1]?.callback();
expect(fired).toBe(false);
});
it("releases owned replay delays when the endpoint closes", async () => {
const scheduler = new ReplayDelayScheduler();
const waiting = scheduler.wait(Number.MAX_SAFE_INTEGER);
scheduler.close();
scheduler.close();
await waiting;
await scheduler.wait(Number.MAX_SAFE_INTEGER);
});
});
src/application/RuntimeExecutableDiagnostics.ts
-390
View File
@@ -1,390 +0,0 @@
import { spawn, type ChildProcess } from "node:child_process";
import { constants } from "node:fs";
import { access, realpath } from "node:fs/promises";
import { delimiter, join } from "node:path";
const TOOL_NAMES = ["node", "npm", "npx"] as const;
const PROBE_CONCURRENCY = 4;
type RuntimeToolName = (typeof TOOL_NAMES)[number];
type RuntimeProbeFailureCode =
| "runtime_dynamic_library_missing"
| "runtime_invalid_version_output"
| "runtime_timeout"
| "runtime_signal"
| "runtime_spawn_failed"
| "runtime_nonzero_exit";
/** Classification of one failed executable probe with its complete stderr. */
interface RuntimeProbeFailure {
readonly code: RuntimeProbeFailureCode;
readonly exit_code: number | null;
readonly signal: NodeJS.Signals | null;
readonly dependency: string | null;
readonly stderr: string;
}
interface RuntimeExecutableIdentity {
readonly tool: RuntimeToolName;
readonly lexical_path: string;
readonly canonical_path: string;
readonly path_index: number | null;
readonly selection: "rea-launcher" | "path-primary" | "path-shadowed";
}
/** One lexical PATH candidate and its canonical executable probe result. */
type RuntimeExecutableDiagnostic = RuntimeExecutableIdentity &
(
| {
readonly version: string;
readonly healthy: true;
readonly failure: null;
}
| {
readonly version: null;
readonly healthy: false;
readonly failure: RuntimeProbeFailure;
}
);
/** Runtime identities observed under one exact effective environment. */
export interface RuntimeExecutableInventory {
readonly launcher_node: string;
readonly candidates: readonly RuntimeExecutableDiagnostic[];
}
/** Host inputs that determine executable discovery and shebang resolution. */
export interface RuntimeExecutableInventoryOptions {
readonly platform: NodeJS.Platform;
readonly path: string;
readonly launcherNode: string;
readonly pathExtensions?: readonly string[];
readonly timeoutMs?: number;
}
/** Inventory and probe Node toolchain candidates under one exact PATH. */
export const inspectRuntimeExecutables = async (
options: RuntimeExecutableInventoryOptions,
): Promise<RuntimeExecutableInventory> => {
const pathEntries = unique(options.path.split(delimiter).filter(Boolean));
const candidates: Array<{
readonly tool: RuntimeToolName;
readonly path: string;
readonly pathIndex: number | null;
readonly selection: RuntimeExecutableDiagnostic["selection"];
}> = [
{
tool: "node",
path: options.launcherNode,
pathIndex: null,
selection: "rea-launcher",
},
];
for (const tool of TOOL_NAMES) {
const discovered = await discoverToolCandidates(
tool,
pathEntries,
options.platform,
options.pathExtensions,
);
for (const [index, candidate] of discovered.entries())
candidates.push({
tool,
path: candidate.path,
pathIndex: candidate.pathIndex,
selection: index === 0 ? "path-primary" : "path-shadowed",
});
}
const seen = new Set<string>();
const uniqueCandidates: typeof candidates = [];
for (const candidate of candidates) {
const key = `${candidate.tool}\0${candidate.path}`;
if (seen.has(key)) continue;
seen.add(key);
uniqueCandidates.push(candidate);
}
const diagnostics = await mapConcurrentBounded(
uniqueCandidates,
PROBE_CONCURRENCY,
(candidate) =>
probeCandidate(
candidate,
options.timeoutMs,
options.path,
options.platform,
),
);
return {
launcher_node: await canonicalPath(options.launcherNode),
candidates: diagnostics,
};
};
const discoverToolCandidates = async (
tool: RuntimeToolName,
pathEntries: readonly string[],
platform: NodeJS.Platform,
configuredExtensions: readonly string[] | undefined,
): Promise<
readonly { readonly path: string; readonly pathIndex: number }[]
> => {
const extensions =
platform === "win32"
? (configuredExtensions ?? [".COM", ".EXE", ".BAT", ".CMD"])
: [""];
const candidates: Array<{
readonly path: string;
readonly pathIndex: number;
}> = [];
for (const [pathIndex, directory] of pathEntries.entries()) {
for (const extension of extensions) {
const path = join(directory, `${tool}${extension}`);
if (await isExecutable(path, platform)) {
candidates.push({ path, pathIndex });
break;
}
}
}
return candidates;
};
const probeCandidate = async (
candidate: {
readonly tool: RuntimeToolName;
readonly path: string;
readonly pathIndex: number | null;
readonly selection: RuntimeExecutableDiagnostic["selection"];
},
timeoutMs: number | undefined,
effectivePath: string,
platform: NodeJS.Platform,
): Promise<RuntimeExecutableDiagnostic> => {
const canonical = await canonicalPath(candidate.path);
const result = await executeVersion(
candidate.path,
timeoutMs,
effectivePath,
platform,
);
const identity = {
tool: candidate.tool,
lexical_path: candidate.path,
canonical_path: canonical,
path_index: candidate.pathIndex,
selection: candidate.selection,
};
return result.ok
? { ...identity, version: result.version, healthy: true, failure: null }
: {
...identity,
version: null,
healthy: false,
failure: result.failure,
};
};
type VersionResult =
| { readonly ok: true; readonly version: string }
| { readonly ok: false; readonly failure: RuntimeProbeFailure };
const executeVersion = (
path: string,
timeoutMs: number | undefined,
effectivePath: string,
platform: NodeJS.Platform = process.platform,
): Promise<VersionResult> =>
new Promise((resolve) => {
// Node >= 18.20 refuses to spawn .cmd/.bat files without a shell
// (CVE-2024-27980 mitigation) and throws spawn EINVAL. On Windows, npm and
// npx PATH shims are .CMD files, so every probe of them failed. Route
// batch-file candidates through cmd.exe instead.
const command = windowsBatchCommand(path, platform);
const child = command
? spawn(command.executable, command.arguments, {
windowsHide: true,
windowsVerbatimArguments: true,
env: environmentWithPath(effectivePath),
stdio: ["ignore", "pipe", "pipe"],
})
: spawn(path, ["--version"], {
windowsHide: true,
env: environmentWithPath(effectivePath),
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
let spawnError: Error | undefined;
let timeout: NodeJS.Timeout | undefined;
let timedOut = false;
child.stdout?.setEncoding("utf8");
child.stderr?.setEncoding("utf8");
child.stdout?.on("data", (chunk: string) => {
stdout += chunk;
});
child.stderr?.on("data", (chunk: string) => {
stderr += chunk;
});
child.once("error", (cause: Error) => {
spawnError = cause;
});
if (timeoutMs !== undefined && timeoutMs > 0) {
timeout = setTimeout(() => {
timedOut = true;
child.kill();
}, timeoutMs);
timeout.unref();
}
child.once("close", () => {
if (timeout !== undefined) clearTimeout(timeout);
if (spawnError === undefined && child.exitCode === 0) {
const version = firstNonemptyLine(stdout) ?? firstNonemptyLine(stderr);
if (version !== undefined) {
resolve({ ok: true, version });
return;
}
resolve({
ok: false,
failure: failure("runtime_invalid_version_output", child, stderr),
});
return;
}
const cause =
spawnError ??
new Error("Runtime version probe did not exit successfully");
resolve({
ok: false,
failure: timedOut
? failure("runtime_timeout", child, stderr)
: classifyProbeFailure(cause, child, stderr),
});
});
});
/**
* Build the cmd.exe invocation for a Windows batch-file candidate
* (.cmd/.bat/.CMD/.BAT). Returns null for anything else, including on
* non-Windows platforms. The whole command line is passed as one
* /c argument so paths with spaces survive; `windowsVerbatimArguments`
* keeps Node from re-quoting it. Embedded double quotes are doubled,
* since a bare quote would terminate the /c string and allow argument
* injection via a crafted PATH entry.
*/
const windowsBatchCommand = (
path: string,
platform: NodeJS.Platform = process.platform,
systemRoot: string = process.env.SystemRoot ?? "C:\\Windows",
): { executable: string; arguments: string[] } | null => {
if (platform !== "win32") return null;
if (!/\.(cmd|bat)$/iu.test(path)) return null;
return {
executable: `${systemRoot}\\System32\\cmd.exe`,
arguments: ["/d", "/s", "/c", `"${path.replaceAll('"', '""')}" --version`],
};
};
/**
* Testing-only re-export of the batch-command builder. Pure and
* platform-injected, so it is fully exercisable on non-Windows hosts.
*/
export const windowsBatchCommandForTesting = windowsBatchCommand;
/** Testing-only re-export of the version probe. */
export const executeVersionForTesting = executeVersion;
const classifyProbeFailure = (
cause: Error,
child: ChildProcess,
stderr: string,
): RuntimeProbeFailure => {
const dependency = missingDynamicLibrary(stderr);
if (dependency !== null)
return failure(
"runtime_dynamic_library_missing",
child,
stderr,
dependency,
);
if ("killed" in cause && cause.killed === true)
return failure("runtime_timeout", child, stderr);
if (child.signalCode !== null)
return failure("runtime_signal", child, stderr);
if ("code" in cause && typeof cause.code === "string")
return failure("runtime_spawn_failed", child, stderr);
return failure("runtime_nonzero_exit", child, stderr);
};
const failure = (
code: RuntimeProbeFailureCode,
child: ChildProcess,
stderr: string,
dependency: string | null = null,
): RuntimeProbeFailure => ({
code,
exit_code: child.exitCode,
signal: child.signalCode,
dependency,
stderr: stderr.trim(),
});
const missingDynamicLibrary = (stderr: string): string | null =>
/(?:dyld(?:\[[^\]]+\])?: )?Library not loaded:\s*([^\r\n]+)/u
.exec(stderr)?.[1]
?.trim() ?? null;
const isExecutable = async (
path: string,
platform: NodeJS.Platform,
): Promise<boolean> => {
try {
await access(path, platform === "win32" ? constants.F_OK : constants.X_OK);
return true;
} catch {
return false;
}
};
const canonicalPath = async (path: string): Promise<string> => {
try {
return await realpath(path);
} catch {
return path;
}
};
const firstNonemptyLine = (value: string): string | undefined =>
value
.split(/\r?\n/u)
.find((line) => line.trim().length > 0)
?.trim();
const unique = (values: readonly string[]): string[] => [...new Set(values)];
const environmentWithPath = (path: string): NodeJS.ProcessEnv => {
const environment: NodeJS.ProcessEnv = {};
for (const [name, value] of Object.entries(process.env))
if (name.toUpperCase() !== "PATH") environment[name] = value;
environment.PATH = path;
return environment;
};
const mapConcurrentBounded = async <Input, Output>(
inputs: readonly Input[],
concurrency: number,
operation: (input: Input) => Promise<Output>,
): Promise<Output[]> => {
const outputs: Output[] = [];
let nextIndex = 0;
const workers = Array.from(
{ length: Math.min(concurrency, inputs.length) },
async () => {
while (nextIndex < inputs.length) {
const index = nextIndex;
nextIndex += 1;
const input = inputs[index];
if (input !== undefined) outputs[index] = await operation(input);
}
},
);
await Promise.all(workers);
return outputs;
};
src/application/RuntimeExecutableDiagnostics.win32.test.ts
-80
View File
@@ -1,80 +0,0 @@
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import {
executeVersionForTesting as executeVersion,
windowsBatchCommandForTesting as windowsBatchCommand,
} from "./RuntimeExecutableDiagnostics.js";
const roots: string[] = [];
afterEach(async () => {
await Promise.all(
roots.splice(0).map((root) => rm(root, { recursive: true, force: true })),
);
});
describe("windowsBatchCommand", () => {
it("returns null off Windows even for batch files", () => {
expect(windowsBatchCommand("C:\\tools\\npm.cmd", "linux")).toBeNull();
expect(windowsBatchCommand("C:\\tools\\npm.cmd", "darwin")).toBeNull();
});
it("returns null for non-batch candidates on win32", () => {
expect(windowsBatchCommand("C:\\tools\\node.exe", "win32")).toBeNull();
expect(windowsBatchCommand("/usr/local/bin/npm", "win32")).toBeNull();
});
it.each(["npm.cmd", "npm.CMD", "tool.bat", "tool.BAT"])(
"routes %s through cmd.exe on win32",
(file) => {
expect(windowsBatchCommand(`C:\\tools\\${file}`, "win32")).toEqual({
executable: "C:\\Windows\\System32\\cmd.exe",
arguments: ["/d", "/s", "/c", `"C:\\tools\\${file}" --version`],
});
},
);
it("preserves paths with spaces in a single /c argument", () => {
const command = windowsBatchCommand(
"C:\\Program Files\\nodejs\\npm.cmd",
"win32",
);
expect(command?.arguments[3]).toBe(
'"C:\\Program Files\\nodejs\\npm.cmd" --version',
);
});
it("doubles embedded quotes so they cannot break out of the /c string", () => {
const command = windowsBatchCommand('C:\\tools\\evil".cmd', "win32");
expect(command?.arguments[3]).toBe('"C:\\tools\\evil"".cmd" --version');
});
it("honours an explicit SystemRoot", () => {
const command = windowsBatchCommand(
"C:\\tools\\npm.cmd",
"win32",
"D:\\Win",
);
expect(command?.executable).toBe("D:\\Win\\System32\\cmd.exe");
});
});
describe.runIf(process.platform === "win32")("executeVersion on win32", () => {
it("probes a real .bat shim without spawn EINVAL", async () => {
const dir = await mkdtemp(join(tmpdir(), "rea-win32-"));
roots.push(dir);
const shim = join(dir, "probe.bat");
await writeFile(shim, "@echo 9.9.9\r\n");
const result = await executeVersion(
shim,
10_000,
process.env.PATH ?? "",
"win32",
);
expect(result).toMatchObject({ ok: true, version: "9.9.9" });
});
});
src/application/SetupSkill.ts
-1
View File
@@ -10,7 +10,6 @@ const SKILL_FILES = [
"references/javascript-applications.md",
"references/runtime-observation.md",
"references/evidence-workflows.md",
"references/controlled-replay.md",
] as const;
interface CanonicalSkillFile {
src/browser/BrowserScenario.test.ts
+20 -103
View File
@@ -18,7 +18,6 @@ const baseScenario = () => ({
url: "https://app.example.test/",
query: [{ name: "token", value: secret("session") }],
},
allowed_origins: ["https://api.example.test", "https://app.example.test"],
actions: [
{
step_id: "login",
@@ -60,28 +59,6 @@ const baseScenario = () => ({
},
],
},
request_replay: {
mode: "exact",
unmatched: "abort",
routes: [
{
route_id: "profile",
method: "GET",
request: { url: "https://api.example.test/profile" },
response: {
kind: "response",
status: 200,
headers: [
{
name: "content-type",
value: literal("application/json"),
},
],
body: literal('{"name":"Ada"}'),
},
},
],
},
secrets: [
{
secret_id: "session",
@@ -106,10 +83,6 @@ const baseScenario = () => ({
describe("browserScenarioSchema", () => {
it("accepts and normalizes a declared scenario", () => {
const parsed = browserScenarioSchema.parse(baseScenario());
expect(parsed.allowed_origins).toEqual([
"https://api.example.test",
"https://app.example.test",
]);
expect(parsed.browser).toMatchObject({ mode: "launch", headless: true });
expect(parsed.environment).toMatchObject({
viewport: { width: 1_280, height: 720, device_scale_factor: 1 },
@@ -119,11 +92,6 @@ describe("browserScenarioSchema", () => {
reduced_motion: "reduce",
service_workers: "block",
});
expect(parsed.request_replay).toEqual({
mode: "exact",
unmatched: "abort",
routes: expect.any(Array),
});
expect(parsed.capture).toEqual({
after_each_step: ["screenshot", "url", "accessibility"],
at_end: ["dom", "storage"],
@@ -197,44 +165,18 @@ describe("browserScenarioSchema", () => {
expect(browserScenarioSchema.safeParse(scenario).success).toBe(false);
});
it("rejects every undeclared navigation, storage, and replay origin", () => {
for (const mutate of [
(scenario: ReturnType<typeof baseScenario>) => {
scenario.start_url.url = "https://other.example.test/";
},
(scenario: ReturnType<typeof baseScenario>) => {
scenario.actions[2] = {
...scenario.actions[2],
destination: { url: "https://other.example.test/" },
} as never;
},
(scenario: ReturnType<typeof baseScenario>) => {
scenario.storage.local_storage[0]!.origin =
"https://other.example.test";
},
(scenario: ReturnType<typeof baseScenario>) => {
scenario.request_replay.routes[0]!.request.url =
"https://other.example.test/data";
},
]) {
const scenario = baseScenario();
mutate(scenario);
expect(browserScenarioSchema.safeParse(scenario).success).toBe(false);
}
it("accepts explicitly selected navigation and storage origins without an allowlist", () => {
const scenario = baseScenario();
scenario.actions[2] = {
...scenario.actions[2],
destination: { url: "https://other.example.test/" },
} as never;
scenario.storage.local_storage[0]!.origin = "https://storage.example.test";
expect(browserScenarioSchema.safeParse(scenario).success).toBe(true);
});
});
describe("browserScenarioSchema credential handling", () => {
it("rejects redirects outside declared origins", () => {
const scenario = baseScenario();
scenario.request_replay.routes[0]!.response = {
kind: "redirect",
status: 302,
destination: { url: "https://other.example.test/" },
} as never;
expect(browserScenarioSchema.safeParse(scenario).success).toBe(false);
});
describe("browserScenarioSchema secret declarations", () => {
it("rejects missing and duplicate secret declarations but accepts redaction-only declarations", () => {
const missing = baseScenario();
missing.secrets = missing.secrets.filter(
@@ -260,42 +202,17 @@ describe("browserScenarioSchema credential handling", () => {
expect(browserScenarioSchema.safeParse(unused).success).toBe(false);
});
it("accepts literal credential headers and rejects provider-owned headers", () => {
const credentialHeaders = baseScenario();
credentialHeaders.request_replay.routes[0]!.response = {
kind: "response",
status: 200,
headers: [
{ name: "authorization", value: literal("Bearer caller-value") },
{ name: "cookie", value: literal("session=caller-value") },
{ name: "set-cookie", value: literal("session=caller-value") },
],
} as never;
expect(browserScenarioSchema.safeParse(credentialHeaders).success).toBe(
true,
);
for (const name of ["content-length", "location", "transfer-encoding"]) {
const scenario = baseScenario();
scenario.request_replay.routes[0]!.response = {
kind: "response",
status: 200,
headers: [{ name, value: literal("raw") }],
} as never;
expect(browserScenarioSchema.safeParse(scenario).success).toBe(false);
}
});
it("rejects duplicate replay response headers case-insensitively", () => {
const scenario = baseScenario();
const response = scenario.request_replay.routes[0]!.response;
if (response.kind !== "response")
throw new Error("Expected replay response fixture");
response.headers.push({
name: "Content-Type",
value: literal("text/plain"),
});
expect(browserScenarioSchema.safeParse(scenario).success).toBe(false);
it("rejects removed replay and origin policy options", () => {
for (const legacyField of [
{ request_replay: { mode: "disabled" } },
{ allowed_origins: ["https://app.example.test"] },
])
expect(
browserScenarioSchema.safeParse({
...baseScenario(),
...legacyField,
}).success,
).toBe(false);
});
it("rejects raw secret-shaped action values", () => {
src/browser/PlaywrightBrowserScenarioProvider.test.ts
-2
View File
@@ -51,14 +51,12 @@ const scenario = (
executable_path: "/opt/chromium",
},
start_url: { url: "https://app.example.test/" },
allowed_origins: ["https://app.example.test"],
actions: Array.from({ length: options.actions ?? 1 }, (_, index) => ({
step_id: `wait_${index}`,
action: "wait_for_timeout",
duration_ms: 1,
})),
storage: {},
request_replay: { mode: "disabled" },
secrets: [],
capture: {
after_each_step: options.captures ?? ["url"],
src/browser/PlaywrightBrowserScenarioProvider.ts
-1
View File
@@ -288,7 +288,6 @@ const runScenario = async (
},
scenario: {
start_origin: new URL(scenario.start_url.url).origin,
allowed_origins: scenario.allowed_origins,
action_count: scenario.actions.length,
secret_references: scenario.secrets.map(({ secret_id: id }) => id).sort(),
},
src/browser/PlaywrightScenarioArtifacts.test.ts
-3
View File
@@ -11,7 +11,6 @@ const scenario = browserScenarioSchema.parse({
executable_path: "/opt/chromium",
},
start_url: { url: "https://app.example.test/" },
allowed_origins: ["https://app.example.test"],
actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }],
});
const secrets = BrowserScenarioSecrets.resolve(scenario, {});
@@ -27,7 +26,6 @@ it("returns a complete requested text artifact beyond the former inline ceiling"
const result = await capturePlaywrightStepArtifacts({
context: {} as BrowserContext,
page,
scenario,
secrets,
requested: new Set(["dom"]),
});
@@ -80,7 +78,6 @@ it("redacts declared secrets from current and historical URL artifacts", async (
const result = await capturePlaywrightStepArtifacts({
context: {} as BrowserContext,
page,
scenario: scenarioWithSecretUrl,
secrets: scenarioSecrets,
requested: new Set(["url", "history"]),
});
src/browser/PlaywrightScenarioArtifacts.ts
+9 -25
View File
@@ -134,12 +134,15 @@ const captureHistory = async (page: Page, secrets: BrowserScenarioSecrets) => {
const captureStorage = async (input: {
readonly context: BrowserContext;
readonly page: Page;
readonly scenario: BrowserScenario;
readonly secrets: BrowserScenarioSecrets;
}) => {
const { context, page, scenario, secrets } = input;
const { context, page, secrets } = input;
try {
const cookies = await context.cookies(scenario.allowed_origins);
const pageUrl = new URL(page.url());
const cookies =
pageUrl.protocol === "http:" || pageUrl.protocol === "https:"
? await context.cookies(pageUrl.href)
: [];
const pageStorage = storageValueSchema.parse(
await page.evaluate(`(() => ({
local_storage: Object.entries(window.localStorage),
@@ -171,39 +174,20 @@ const captureStorage = async (input: {
}
};
const isAllowedPage = (
page: Page,
allowedOrigins: ReadonlySet<string>,
): boolean => {
try {
return allowedOrigins.has(new URL(page.url()).origin);
} catch {
return false;
}
};
/** Capture selected step artifacts without retaining data after scope loss. */
export const capturePlaywrightStepArtifacts = async (input: {
readonly context: BrowserContext;
readonly page: Page;
readonly scenario: BrowserScenario;
readonly secrets: BrowserScenarioSecrets;
readonly requested: ReadonlySet<SnapshotKind>;
}): Promise<BrowserStepArtifacts> => {
const { context, page, scenario, secrets, requested } = input;
const allowed = isAllowedPage(page, new Set(scenario.allowed_origins));
const denied = () => missing("current page is outside approved origins");
const { context, page, secrets, requested } = input;
const state = <Value>(
kind: SnapshotKind,
capture: () => Promise<Value>,
): Promise<
Value | ReturnType<typeof notRequested> | ReturnType<typeof missing>
> =>
!requested.has(kind)
? Promise.resolve(notRequested())
: allowed || kind === "url"
? capture()
: Promise.resolve(denied());
> => (!requested.has(kind) ? Promise.resolve(notRequested()) : capture());
return browserStepArtifactsSchema.parse({
screenshot: await state("screenshot", () => captureScreenshot(page)),
dom: await state("dom", () => captureDom({ page, secrets })),
@@ -216,7 +200,7 @@ export const capturePlaywrightStepArtifacts = async (input: {
})),
history: await state("history", () => captureHistory(page, secrets)),
storage: await state("storage", () =>
captureStorage({ context, page, scenario, secrets }),
captureStorage({ context, page, secrets }),
),
});
};
src/browser/PlaywrightScenarioBrowser.ts
-18
View File
@@ -48,21 +48,9 @@ const allowedBrowserEnvironment = (
);
};
const originAllowed = (
value: string,
allowedOrigins: ReadonlySet<string>,
): boolean => {
try {
return allowedOrigins.has(new URL(value).origin);
} catch {
return false;
}
};
const findConnectedPage = async (
browser: Browser,
targetId: string,
allowedOrigins: ReadonlySet<string>,
): Promise<{ readonly context: BrowserContext; readonly page: Page }> => {
for (const context of browser.contexts())
for (const page of context.pages()) {
@@ -70,11 +58,6 @@ const findConnectedPage = async (
try {
const { targetInfo } = await session.send("Target.getTargetInfo");
if (targetInfo.targetId !== targetId) continue;
if (
page.url() !== "about:blank" &&
!originAllowed(page.url(), allowedOrigins)
)
throw new BrowserObservationError(OPERATION, "target_not_allowed");
return { context, page };
} finally {
await session.detach();
@@ -128,7 +111,6 @@ export const openPlaywrightScenarioBrowser = async (
const target = await findConnectedPage(
browser,
scenario.browser.target_id,
new Set(scenario.allowed_origins),
);
await configureAttachedEnvironment(target.context, target.page, scenario);
return { ...target, browser, profilePath: undefined };
src/browser/PlaywrightScenarioEvents.test.ts
-5
View File
@@ -20,7 +20,6 @@ describe("PlaywrightScenarioEvents", () => {
executable_path: "/opt/chromium",
},
start_url: { url: "https://app.example.test/" },
allowed_origins: ["https://app.example.test"],
environment: {
viewport: { width: 1_280, height: 720 },
locale: "en-US",
@@ -36,7 +35,6 @@ describe("PlaywrightScenarioEvents", () => {
},
],
storage: {},
request_replay: { mode: "disabled" },
secrets: [],
capture: {
after_each_step: [],
@@ -50,7 +48,6 @@ describe("PlaywrightScenarioEvents", () => {
page,
enabled: new Set(["page-errors"]),
secrets,
allowedOrigins: scenario.allowed_origins,
});
const error = new Error("m".repeat(70_000));
error.stack = "s".repeat(300_000);
@@ -79,7 +76,6 @@ describe("PlaywrightScenarioEvents", () => {
executable_path: "/opt/chromium",
},
start_url: { url: "https://app.example.test/" },
allowed_origins: ["https://app.example.test"],
actions: [
{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 },
],
@@ -91,7 +87,6 @@ describe("PlaywrightScenarioEvents", () => {
page,
enabled: new Set(["page-errors"]),
secrets,
allowedOrigins: scenario.allowed_origins,
});
for (let index = 0; index < 2_001; index += 1)
listeners
src/browser/PlaywrightScenarioEvents.ts
+36 -81
View File
@@ -36,7 +36,6 @@ interface EventCaptureOptions {
readonly page: Page;
readonly enabled: ReadonlySet<EventName>;
readonly secrets: BrowserScenarioSecrets;
readonly allowedOrigins: readonly string[];
}
/** Arrival-ordered Playwright event capture with current-step attribution. */
@@ -46,12 +45,10 @@ export class PlaywrightScenarioEvents {
private sequence = 0;
private readonly enabled: ReadonlySet<EventName>;
private readonly secrets: BrowserScenarioSecrets;
private readonly allowedOrigins: ReadonlySet<string>;
constructor(options: EventCaptureOptions) {
this.enabled = options.enabled;
this.secrets = options.secrets;
this.allowedOrigins = new Set(options.allowedOrigins);
this.observePage(options.page);
}
@@ -91,67 +88,38 @@ export class PlaywrightScenarioEvents {
private observePage(page: Page): void {
if (this.enabled.has("console"))
page.on("console", (message) => {
if (this.pageInScope(page)) this.console(message);
});
page.on("console", (message) => this.console(message));
if (this.enabled.has("page-errors"))
page.on("pageerror", (error) => {
if (this.pageInScope(page))
this.push({
kind: "page-error",
message: this.secrets.redact(error.message),
stack:
error.stack === undefined
? null
: this.secrets.redact(error.stack),
});
this.push({
kind: "page-error",
message: this.secrets.redact(error.message),
stack:
error.stack === undefined ? null : this.secrets.redact(error.stack),
});
});
if (this.enabled.has("network")) {
page.on("request", (request) => {
if (this.pageInScope(page)) this.request("request", request);
});
page.on("response", (response) => {
if (this.pageInScope(page)) this.response(response);
});
page.on("requestfailed", (request) => {
if (this.pageInScope(page)) this.request("request-failed", request);
});
page.on("request", (request) => this.request("request", request));
page.on("response", (response) => this.response(response));
page.on("requestfailed", (request) =>
this.request("request-failed", request),
);
}
if (this.enabled.has("websockets"))
page.on("websocket", (socket) => {
if (this.pageInScope(page)) this.webSocket(page, socket);
});
page.on("websocket", (socket) => this.webSocket(socket));
if (this.enabled.has("frames")) {
page.on("frameattached", (frame) => {
if (this.pageInScope(page)) this.frame("frame-attached", frame);
});
page.on("framedetached", (frame) => {
if (this.pageInScope(page)) this.frame("frame-detached", frame);
});
page.on("framenavigated", (frame) => {
if (this.pageInScope(page)) this.frame("frame-navigated", frame);
});
page.on("frameattached", (frame) => this.frame("frame-attached", frame));
page.on("framedetached", (frame) => this.frame("frame-detached", frame));
page.on("framenavigated", (frame) =>
this.frame("frame-navigated", frame),
);
}
if (this.enabled.has("workers"))
page.on("worker", (worker) => {
if (this.pageInScope(page)) this.worker(page, worker);
});
page.on("worker", (worker) => this.worker(worker));
if (this.enabled.has("popups"))
page.on("popup", (popup) => {
if (this.pageInScope(page)) this.popup(popup);
});
page.on("popup", (popup) => this.popup(popup));
if (this.enabled.has("downloads"))
page.on("download", (download) => {
if (this.pageInScope(page)) this.download(download);
});
}
private pageInScope(page: Page): boolean {
try {
return this.allowedOrigins.has(new URL(page.url()).origin);
} catch {
return false;
}
page.on("download", (download) => this.download(download));
}
private safeUrl(value: string) {
@@ -209,20 +177,16 @@ export class PlaywrightScenarioEvents {
});
}
private webSocket(page: Page, socket: WebSocket): void {
private webSocket(socket: WebSocket): void {
const url = this.safeUrl(socket.url());
this.push({ kind: "websocket-opened", url });
socket.on("framesent", ({ payload }) => {
if (this.pageInScope(page))
this.webSocketFrame("websocket-frame-sent", url, payload);
});
socket.on("framereceived", ({ payload }) => {
if (this.pageInScope(page))
this.webSocketFrame("websocket-frame-received", url, payload);
});
socket.on("close", () => {
if (this.pageInScope(page)) this.push({ kind: "websocket-closed", url });
});
socket.on("framesent", ({ payload }) =>
this.webSocketFrame("websocket-frame-sent", url, payload),
);
socket.on("framereceived", ({ payload }) =>
this.webSocketFrame("websocket-frame-received", url, payload),
);
socket.on("close", () => this.push({ kind: "websocket-closed", url }));
}
private webSocketFrame(
@@ -265,36 +229,27 @@ export class PlaywrightScenarioEvents {
});
}
private worker(page: Page, worker: Worker): void {
private worker(worker: Worker): void {
const details = {
url: this.safeUrl(worker.url()),
name: null,
};
this.push({ kind: "worker-created", ...details });
worker.on("close", () => {
if (this.pageInScope(page))
this.push({ kind: "worker-closed", ...details });
});
worker.on("close", () => this.push({ kind: "worker-closed", ...details }));
}
private popup(page: Page): void {
if (!this.pageInScope(page)) {
void page.close().catch(() => undefined);
return;
}
const opened = {
url: page.url() === "" ? null : this.safeUrl(page.url()),
name: null,
};
this.push({ kind: "popup-opened", ...opened });
page.on("close", () =>
this.pageInScope(page)
? this.push({
kind: "popup-closed",
url: page.url() === "" ? null : this.safeUrl(page.url()),
name: null,
})
: undefined,
this.push({
kind: "popup-closed",
url: page.url() === "" ? null : this.safeUrl(page.url()),
name: null,
}),
);
this.observePage(page);
}
src/browser/PlaywrightScenarioSession.ts
+6 -158
View File
@@ -1,9 +1,4 @@
import {
type BrowserContext,
type Page,
type Route,
type WebSocketRoute,
} from "playwright-core";
import { type BrowserContext, type Page } from "playwright-core";
import type {
BrowserScenario,
@@ -27,17 +22,6 @@ import { withPlaywrightExecutionBoundary } from "./PlaywrightExecutionBoundary.j
const OPERATION = "capture_browser_scenario" as const;
const originAllowed = (
value: string,
allowedOrigins: ReadonlySet<string>,
): boolean => {
try {
return allowedOrigins.has(new URL(value).origin);
} catch {
return false;
}
};
const storageSeeds = (
blocks: BrowserScenario["storage"]["local_storage"],
secrets: BrowserScenarioSecrets,
@@ -81,111 +65,6 @@ const installStorageSeeds = async (
})()`);
};
const installHttpPolicy = async (input: {
readonly context: BrowserContext;
readonly scenario: BrowserScenario;
readonly secrets: BrowserScenarioSecrets;
readonly ownedPages: Set<Page>;
readonly claimOwnedPage: (page: Page) => Promise<boolean>;
}): Promise<void> => {
const { context, scenario, secrets, ownedPages, claimOwnedPage } = input;
const routes =
scenario.request_replay.mode === "exact"
? new Map(
scenario.request_replay.routes.map((route) => [
`${route.method} ${secrets.url(route.request)}`,
route,
]),
)
: new Map();
await context.route("**/*", async (route, request) => {
let requestPage: Page | undefined;
try {
requestPage = request.frame().page();
} catch {
requestPage = undefined;
}
if (
scenario.browser.mode === "connect" &&
requestPage !== undefined &&
!ownedPages.has(requestPage)
)
await claimOwnedPage(requestPage);
if (
scenario.browser.mode === "connect" &&
(requestPage === undefined || !ownedPages.has(requestPage))
) {
await route.continue();
return;
}
if (!originAllowed(request.url(), new Set(scenario.allowed_origins))) {
await route.abort("blockedbyclient");
return;
}
const replay = routes.get(`${request.method()} ${request.url()}`);
if (replay !== undefined) {
await fulfillReplayRoute(route, replay.response, secrets);
return;
}
if (
scenario.request_replay.mode === "exact" &&
scenario.request_replay.unmatched === "abort"
) {
await route.abort("blockedbyclient");
return;
}
await route.continue();
});
};
type ReplayResponse = Extract<
BrowserScenario["request_replay"],
{ readonly mode: "exact" }
>["routes"][number]["response"];
const fulfillReplayRoute = async (
route: Route,
response: ReplayResponse,
secrets: BrowserScenarioSecrets,
): Promise<void> => {
if (response.kind === "redirect") {
await route.fulfill({
status: response.status,
headers: { location: secrets.url(response.destination) },
});
return;
}
await route.fulfill({
status: response.status,
headers: Object.fromEntries(
response.headers.map(({ name, value }) => [name, secrets.value(value)]),
),
...(response.body === undefined
? {}
: { body: secrets.value(response.body) }),
});
};
const installWebSocketPolicy = async (
owner: BrowserContext | Page,
scenario: BrowserScenario,
): Promise<void> => {
const allowed = new Set(scenario.allowed_origins);
await owner.routeWebSocket("**/*", async (route: WebSocketRoute) => {
const url = new URL(route.url());
const httpOrigin = `${url.protocol === "wss:" ? "https:" : "http:"}//${url.host}`;
if (
!allowed.has(httpOrigin) ||
(scenario.request_replay.mode === "exact" &&
scenario.request_replay.unmatched === "abort")
) {
await route.close({ code: 1008, reason: "outside scenario policy" });
return;
}
route.connectToServer();
});
};
const blockAttachedServiceWorkers = async (page: Page): Promise<void> => {
const controller = await page.evaluate(
"Boolean(navigator.serviceWorker?.controller)",
@@ -210,40 +89,12 @@ const initializePage = async (
page: Page,
scenario: BrowserScenario,
secrets: BrowserScenarioSecrets,
): Promise<Set<Page>> => {
): Promise<void> => {
context.setDefaultTimeout(0);
context.setDefaultNavigationTimeout(0);
const ownedPages = new Set([page]);
if (scenario.browser.mode === "connect")
await blockAttachedServiceWorkers(page);
await installWebSocketPolicy(
scenario.browser.mode === "launch" ? context : page,
scenario,
);
const claimOwnedPage = async (candidate: Page): Promise<boolean> => {
if (ownedPages.has(candidate)) return true;
const opener = await candidate.opener();
if (opener === null || !ownedPages.has(opener)) return false;
await installWebSocketPolicy(candidate, scenario);
ownedPages.add(candidate);
return true;
};
await installHttpPolicy({
context,
scenario,
secrets,
ownedPages,
claimOwnedPage,
});
page.on("popup", (popup) => {
if (scenario.browser.mode === "connect")
void claimOwnedPage(popup).catch(async () => {
await popup.close().catch(() => undefined);
});
else ownedPages.add(popup);
});
await installStorageSeeds(context, page, scenario, secrets);
return ownedPages;
};
export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
@@ -256,13 +107,12 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
private constructor(
private readonly opened: OpenedScenarioBrowser,
private readonly scenario: BrowserScenario,
mode: BrowserScenario["browser"]["mode"],
private readonly secrets: BrowserScenarioSecrets,
private readonly eventCapture: PlaywrightScenarioEvents,
) {
this.mode = scenario.browser.mode;
this.processOwnership =
scenario.browser.mode === "launch" ? "provider-owned" : "external";
this.mode = mode;
this.processOwnership = mode === "launch" ? "provider-owned" : "external";
this.version = opened.browser.version();
this.initialUrl = opened.page.url();
}
@@ -303,11 +153,10 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
page: opened.page,
enabled: new Set(scenario.capture.events),
secrets,
allowedOrigins: scenario.allowed_origins,
});
const session = new PlaywrightScenarioSession(
opened,
scenario,
scenario.browser.mode,
secrets,
events,
);
@@ -378,7 +227,6 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
capturePlaywrightStepArtifacts({
context: this.opened.context,
page: this.opened.page,
scenario: this.scenario,
secrets: this.secrets,
requested,
}),
src/cli.ts
+2 -2
View File
@@ -55,7 +55,7 @@ export const createCli = (
registerCoreAnalysisCommands(cli, logger);
registerUtilityCommands(cli, logger, environment);
registerArtifactCommands(cli, logger);
registerManagedCommands(cli, logger, environment);
registerManagedCommands(cli, logger);
registerEvidenceCommands(cli, logger);
registerProcessCommands(cli, logger, environment);
registerBrowserCommands(cli, logger);
@@ -63,6 +63,6 @@ export const createCli = (
registerBrowserScenarioCommands(cli, logger);
registerElectronCommands(cli, logger);
registerJavaScriptRuntimeObservationCommands(cli, logger);
registerApplicationCommands(cli, logger, environment);
registerApplicationCommands(cli, logger);
return cli;
};
src/cli/managedCommands.ts
-35
View File
@@ -3,9 +3,7 @@ import { z } from "incur";
import { compareManagedMemberPaths } from "../application/ManagedMemberComparisonService.js";
import { verifyManagedNativeBoundariesEvidence } from "../application/ManagedNativeVerificationService.js";
import { importManagedReconstructionEvidence } from "../application/ManagedReconstructionService.js";
import { planManagedRuntimeCorrelationEvidence } from "../application/ManagedRuntimeCorrelationService.js";
import { runProviderAnalysis } from "../application/DirectAnalysis.js";
import { parseConfig } from "../config.js";
import { parseCliJsonInput } from "../cliJsonInput.js";
import { projectAnalysisError } from "../domain/errors.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
@@ -17,7 +15,6 @@ import { registerProjectManagedApplicationGraph } from "./managedProjectGraphCom
export const registerManagedCommands = (
cli: CliInstance,
logger: Logger,
environment: Readonly<Record<string, string | undefined>> = process.env,
): void => {
registerInspectManagedArtifact(cli, logger);
registerInspectManagedMembers(cli, logger);
@@ -25,7 +22,6 @@ export const registerManagedCommands = (
registerCompareManagedMembers(cli, logger);
registerImportManagedReconstruction(cli, logger);
registerVerifyManagedNativeBoundaries(cli, logger);
registerPlanManagedRuntimeCorrelation(cli, logger, environment);
registerProjectManagedApplicationGraph(cli, logger);
};
@@ -162,34 +158,3 @@ const registerVerifyManagedNativeBoundaries = (
}),
});
};
const registerPlanManagedRuntimeCorrelation = (
cli: CliInstance,
logger: Logger,
environment: Readonly<Record<string, string | undefined>> = process.env,
): void => {
cli.command(CLI_COMMANDS.planManagedRuntimeCorrelation, {
description:
"Plan a managed runtime-correlation request without probing or executing a runtime",
args: z.object({
inputJson: z
.string()
.describe("Inline managed runtime-correlation JSON or JSON file path"),
}),
run: ({ args }) =>
logCliCommand(logger, "plan-managed-runtime-correlation", async () => {
const input = await parseCliJsonInput(
args.inputJson,
"plan-managed-runtime-correlation",
);
if (!input.ok) return input.error;
const config = parseConfig(environment);
if (!config.ok) return projectAnalysisError(config.error);
const result = await planManagedRuntimeCorrelationEvidence(
{ configuration: () => config.value.managedRuntimeConfiguration },
input.value,
);
return result.ok ? result.value : projectAnalysisError(result.error);
}),
});
};
src/cliApplicationCommands.ts
-120
View File
@@ -7,11 +7,6 @@ import {
traceApplicationFeatureEvidenceValidated,
} from "./application/JavaScriptApplicationWorkflowService.js";
import { traceJavaScriptSemanticsEvidenceValidated } from "./application/JavaScriptSemanticTraceService.js";
import { runControlledReplay } from "./application/JavaScriptReplayService.js";
import {
executeNodeCharacterization,
prepareNodeCharacterization,
} from "./application/NodeRuntimeCharacterizationService.js";
import {
evaluateReconstructionCoverage,
reconstructionCoverageEvaluationInputSchema,
@@ -27,10 +22,6 @@ import {
} from "./domain/errors.js";
import { jsonValueSchema, type JsonValue } from "./domain/jsonValue.js";
import type { Logger } from "./logger.js";
import { parseConfig } from "./config.js";
import { LinuxJavaScriptReplayRunner } from "./replay/LinuxJavaScriptReplayRunner.js";
import { SystemJavaScriptReplayHost } from "./replay/SystemJavaScriptReplayHost.js";
import type { AppConfig } from "./config.js";
import { traceApplicationFeatureInputSchema } from "./domain/javascriptFeatureTraceSchemas.js";
import { traceJavaScriptSemanticsInputSchema } from "./domain/javascriptSemanticTraceSchemas.js";
import { compareApplicationVersionsInputSchema } from "./domain/javascriptApplicationVersionComparisonSchemas.js";
@@ -45,7 +36,6 @@ type CliInstance = ReturnType<typeof Cli.create>;
export const registerApplicationCommands = (
cli: ReturnType<typeof Cli.create>,
logger: Logger,
environment: Readonly<Record<string, string | undefined>> = process.env,
): void => {
registerJsonCommand({
cli,
@@ -92,52 +82,6 @@ export const registerApplicationCommands = (
inputSchema: compareJavaScriptExportShapesInputSchema,
workflow: compareJavaScriptExportShapesEvidenceValidated,
});
cli.command(CLI_COMMANDS.runControlledReplay, {
description:
"Plan or execute an extracted-module replay in the isolated Linux sandbox",
args: z.object({
inputJson: z.string().describe("Inline replay JSON or JSON file path"),
}),
run: ({ args }) =>
logCliCommand(logger, CLI_COMMANDS.runControlledReplay, async () => {
const input = await parseCliJsonInput(
args.inputJson,
CLI_COMMANDS.runControlledReplay,
);
if (!input.ok) return input.error;
const config = parseConfig(environment);
if (!config.ok) return projectAnalysisError(config.error);
const result = await runControlledReplay(
{
configuration: () => config.value.javascriptReplayConfiguration,
host: new SystemJavaScriptReplayHost(),
runner: new LinuxJavaScriptReplayRunner(),
},
input.value,
);
return result.ok ? result.value : projectAnalysisError(result.error);
}),
});
registerConfiguredJsonCommand({
cli,
logger,
environment,
name: CLI_COMMANDS.prepareNodeCharacterization,
description:
"Prepare one exact Node/JavaScript characterization without execution",
workflow: (config, input) =>
prepareNodeCharacterization(replayDependencies(config), input),
});
registerConfiguredJsonCommand({
cli,
logger,
environment,
name: CLI_COMMANDS.executeNodeCharacterization,
description:
"Execute one exact Node/JavaScript characterization from its content-bound plan",
workflow: (config, input) =>
executeNodeCharacterization(replayDependencies(config), input),
});
registerObligationLedgerCommand(cli, logger);
registerCoverageCommand(cli, logger);
};
@@ -162,47 +106,6 @@ const registerObligationLedgerCommand = (
},
});
interface ConfiguredJsonCommandOptions {
readonly cli: CliInstance;
readonly logger: Logger;
/** Environment the configuration is read from; defaults to the process environment. */
readonly environment?: Readonly<Record<string, string | undefined>>;
readonly name: string;
readonly description: string;
readonly workflow: (
config: AppConfig,
input: unknown,
) => Promise<
| { readonly ok: true; readonly value: JsonValue }
| { readonly ok: false; readonly error: AnalysisError }
>;
}
const registerConfiguredJsonCommand = ({
cli,
logger,
environment = process.env,
name,
description,
workflow,
}: ConfiguredJsonCommandOptions): void => {
cli.command(name, {
description,
args: z.object({
inputJson: z.string().describe("Inline workflow JSON or JSON file path"),
}),
run: ({ args }) =>
logCliCommand(logger, name, async () => {
const input = await parseCliJsonInput(args.inputJson, name);
if (!input.ok) return input.error;
const configured = configuredConfig(environment);
if (!configured.ok) return configured.error;
const result = await workflow(configured.config, input.value);
return result.ok ? result.value : projectAnalysisError(result.error);
}),
});
};
const registerCoverageCommand = (cli: CliInstance, logger: Logger): void =>
registerJsonCommand({
cli,
@@ -218,29 +121,6 @@ const registerCoverageCommand = (cli: CliInstance, logger: Logger): void =>
},
});
const configuredConfig = (
environment: Readonly<Record<string, string | undefined>> = process.env,
):
| {
readonly ok: true;
readonly config: AppConfig;
}
| {
readonly ok: false;
readonly error: ReturnType<typeof projectAnalysisError>;
} => {
const configured = parseConfig(environment);
return configured.ok
? { ok: true, config: configured.value }
: { ok: false, error: projectAnalysisError(configured.error) };
};
const replayDependencies = (config: AppConfig) => ({
configuration: () => config.javascriptReplayConfiguration,
host: new SystemJavaScriptReplayHost(),
runner: new LinuxJavaScriptReplayRunner(),
});
interface JsonCommandOptions<Schema extends z.ZodType> {
readonly cli: CliInstance;
readonly logger: Logger;
src/cliCommandNames.ts
-5
View File
@@ -33,7 +33,6 @@ export const CLI_COMMANDS = Object.freeze({
compareManagedMembers: "compare-managed-members",
verifyManagedNativeBoundaries: "verify-managed-native-boundaries",
importManagedReconstruction: "import-managed-reconstruction",
planManagedRuntimeCorrelation: "plan-managed-runtime-correlation",
projectManagedApplicationGraph: "project-managed-application-graph",
observeNativeUi: "observe-native-ui",
captureNativeUiScenario: "capture-native-ui-scenario",
@@ -68,10 +67,6 @@ export const CLI_COMMANDS = Object.freeze({
compareApplicationVersions: "compare-application-versions",
compareSourceToBundle: "compare-source-to-bundle",
compareJavaScriptExportShapes: "compare-javascript-export-shapes",
runReplayMachine: "run-replay-machine",
runControlledReplay: "run-controlled-replay",
prepareNodeCharacterization: "prepare-node-characterization",
executeNodeCharacterization: "execute-node-characterization",
buildReconstructionObligationLedger: "build-reconstruction-obligation-ledger",
evaluateReconstructionCoverage: "evaluate-reconstruction-coverage",
});
Loaded 100 of 258 files, more files were not shown because too many files have changed in this diff. Show more