feat(artifacts): inspect and extract Godot PCK with retained facts (#3634)

* feat(artifacts): read Godot PCK packs

Add a Godot Engine PCK artifact reader for standalone packs and packs
embedded behind an executable trailer (formats 2 and 3). Members are
uncompressed byte ranges verified against their declared MD5 on read.

Encrypted directories, sparse bundles, patch removal entries, unknown
flags and format 1 fail closed as unavailable; encrypted members are
reported without being read. inspect-artifact and extract-artifact
recognize standalone packs by their GDPC magic.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(artifacts): bound Godot PCK metadata and own member streams

Cap member paths at 4096 bytes and the retained directory at 64 MiB
before allocation. Read members with positional chunk reads through
pipeline so cancellation is typed, abandoned consumers release their
source, and the shared descriptor stays reader-owned. Retain the root
descriptor for godot-pck inventory, as for ZIP and universal Mach-O.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(artifacts): complete Godot PCK facts, integrity and selected views

* fix(artifacts): preserve atomic-root inventory Evidence

* fix(artifacts): bound aggregate PCK member expansion

---------

Co-authored-by: Samucahub <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: gensokyo-dev <[email protected]>
This commit is contained in:
authored and GitHub committed 2026-10-20 09:53:53 +08:00
1 parent 833136507d
commit 5fa4defcbc
58 files changed
+3298 -361

No files matched your search

@@ -326,3 +326,22 @@ with two complete inspection Evidence records (or current-session retained
references) and explicit scope. Preserve unknown types/duplicate names; static
interface equality is not runtime compatibility. CLI accepts complete inline
Evidence in `rea compare-wasm-interfaces input.json --json`.
## Godot PCK
For a standalone PCK, `open_binary` then `inspect_artifact` with
`detail: "summary"` retains the complete static inventory. Use the returned
`parent_evidence_id` with `inspect_analysis_view` inventory occurrences filtered
by exact `logical_paths` or `occurrence_ids`, and join `artifact_id` to nodes
filtered by `artifact_ids`. Select only needed fields; identity links remain.
Prefer `inspect-artifact --detail summary --evidence_output PATH` for CLI tasks;
the same CLI discovers embedded packs via their EOF trailer without opening a
native analysis provider. Normal native opening remains unchanged.
Distinguish declared MD5 from observed SHA-256 and graph/page completeness from
`source_coverage`. Encrypted directories and sparse bundles have unknown entry
coverage; encrypted members have unknown content identity. An empty filtered
page cannot prove absence in unknown coverage. Retained views are historical,
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
decompilation or target execution is provided. See
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
+28 -27
View File
@@ -174,33 +174,34 @@ CLI commands use the same workflows.
REA requires Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+, plus npm.
Additional tools and host support depend on the target:
| Target | What REA returns | Requirements and guide |
| ---------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 29, 31 and 39; no Unity required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
| Target | What REA returns | Requirements and guide |
| ---------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 29, 31 and 39; no Unity required |
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
Static JavaScript and .NET inspection read the supplied files without running
the application. Runtime capture runs or interacts with the selected target
+11
View File
@@ -85,6 +85,17 @@ The shared CLI/MCP workflow reports compiler version, modules, replacements,
build settings, original bytes and SHA-256. See [Go binaries](go-binaries.md)
for supported containers and absent or malformed metadata.
Inspect a Godot PCK without running the game:
```sh
rea inspect-artifact game.pck --detail summary --evidence_output evidence.json --json
rea extract-artifact game.pck --json
```
Formats 2, 3 and 4 retain header, directory and integrity facts. Compact delivery
keeps complete Evidence in the requested bundle. See [Godot PCK](godot-pck.md)
for embedded packs, batch queries and unknown encrypted coverage.
Inspect Unity IL2CPP metadata names without Unity or a disassembler:
```bash
+47 -17
View File
@@ -389,24 +389,54 @@
"additionalProperties": false
},
"details": {
"type": "object",
"properties": {
"logical_path": { "type": "string" },
"declared_sha256": {
"anyOf": [{ "type": "string" }, { "type": "null" }]
"anyOf": [
{
"type": "object",
"properties": {
"logical_path": { "type": "string" },
"declared_sha256": {
"anyOf": [{ "type": "string" }, { "type": "null" }]
},
"calculated_sha256": {
"anyOf": [{ "type": "string" }, { "type": "null" }]
},
"unpacked": { "type": "boolean" }
},
"required": [
"logical_path",
"declared_sha256",
"calculated_sha256",
"unpacked"
],
"additionalProperties": false
},
"calculated_sha256": {
"anyOf": [{ "type": "string" }, { "type": "null" }]
},
"unpacked": { "type": "boolean" }
},
"required": [
"logical_path",
"declared_sha256",
"calculated_sha256",
"unpacked"
],
"additionalProperties": false
{
"type": "object",
"properties": {
"logical_path": { "type": "string" },
"algorithm": { "type": "string", "const": "md5" },
"declared_md5": {
"type": "string",
"pattern": "^[a-f0-9]{32}$"
},
"calculated_md5": {
"anyOf": [
{ "type": "string", "pattern": "^[a-f0-9]{32}$" },
{ "type": "null" }
]
},
"unpacked": { "type": "boolean" }
},
"required": [
"logical_path",
"algorithm",
"declared_md5",
"calculated_md5",
"unpacked"
],
"additionalProperties": false
}
]
}
},
"required": [
+108
View File
@@ -0,0 +1,108 @@
# Godot PCK archives
`inspect-artifact` and `extract-artifact` read ordinary Godot PCK formats 2, 3
and 4 without running the game or requiring Godot. Official Godot 4.4.1, 4.6.1
and 4.7.1 produced these respective formats in verification; other engine
releases have not been verified. Format 1 is unsupported.
Inspection retains the original header and directory facts, including engine
version, flags, declared file count, source paths, offsets and declared MD5.
Observed plaintext bytes receive content-addressed SHA-256 identities. MD5 is a
container integrity claim, separate from the observed SHA-256. A mismatch fails
by default; `--integrity-policy record-and-continue` retains the observed bytes
and an explicit untrusted contradiction. An all-zero MD5 is an unknown claim,
not a verified digest.
## CLI
Start with a compact summary and keep the complete Evidence bundle:
```sh
rea inspect-artifact /artifacts/game.pck --detail summary \
--evidence_output /evidence/game.json --json
rea extract-artifact /artifacts/game.pck --json
```
Summary delivery requires `--evidence_output` and never overwrites that file.
The response contains `summary` Evidence and an `evidence_bundle` receipt.
Without `--detail summary`, inspection keeps its complete inline delivery.
Extraction creates a new temporary directory and returns its path. It rejects
unsafe paths and verifies container identity before creating output.
The same CLI commands locate a PCK at the beginning of a file or through its
12-byte EOF trailer in an executable carrier. They do not scan arbitrary
payloads or discover packs behind additional trailing signatures. The root
content SHA-256 covers the whole carrier; its content classification remains
native, while the inventory records the PCK role and exact pack bounds.
Ordinary native `open_binary` detection remains unchanged.
## MCP and selected facts
For a standalone pack, call `open_binary` with its absolute path, then
`inspect_artifact` with `{"detail":"summary"}`. The complete inspection and its
nested inventory stay in the session. Use the summary's
`normalized_result.parent_evidence_id` as the source of subsequent views:
```json
{
"name": "inspect_analysis_view",
"arguments": {
"source": {
"kind": "retained-evidence",
"evidence_id": "<parent_evidence_id>"
},
"view": {
"kind": "inventory",
"collection": "occurrences",
"logical_paths": ["scripts/main.gd", "project.binary"],
"fields": [
"logical_path",
"declared_integrity",
"hash_status",
"format_metadata"
],
"offset": 0,
"limit": 32
}
}
}
```
Occurrences retain `occurrence_id`, `artifact_id` and `parent_occurrence_id`
even when `fields` is empty. Batch occurrence IDs with `occurrence_ids`, or
join returned content IDs to `collection: "nodes"`, `artifact_ids: [...]` and
`fields: ["sha256", "size"]`. Filters intersect; an omitted filter selects all
available rows and an empty array selects none. Pagination follows the filtered
canonical order. Multiple occurrences may share one content ID.
Views reuse authenticated immutable inventory facts and lazy lookup indexes;
they do not reopen the source. These are historical observations, including
after the source changes or disappears. Close the target with `close_binary`
when finished. To read an exported bundle through the CLI, select its complete
inspection record as `source: {kind: "inline", evidence: ...}` in a JSON request
file for `rea inspect-analysis-view request.json --json`.
## Coverage and limits
PCK summaries expose root format metadata, available graph counts,
`available_hash_status_counts`, and `source_coverage`. Inventory pages preserve
producer coverage alongside page coverage. `source_complete` means all declared
retained graph rows are available; it does not establish that an encrypted or
otherwise unknown directory was examined. Missing matches in unknown coverage
are not evidence of absence.
Encrypted directories retain their header and declared file count with unknown
entry coverage. Individually encrypted members retain directory metadata with
unknown content identity; REA does not decrypt them. Sparse bundles likewise
retain header facts with unknown member coverage. Extraction refuses an
unreadable directory or encrypted member. This capability does not recover
keys, decode Godot resources, decompile GDScript bytecode, analyze signals,
apply patch/removal entries or execute targets.
Readable logical member bytes, including repeated physical ranges, share the ZIP
archive expansion policy: at most the larger of 256 MiB or 500 times the PCK's
encoded byte span. Embedded carrier bytes do not enlarge that budget. The full
directory is checked before any member is yielded, hashed or extracted. Encrypted
members are not read and do not consume this plaintext-work allowance. Small
bounded aliases remain valid. This is a processing/output resource bound, separate
from the directory metadata retention budget and format validity.
+13 -13
View File
@@ -47,19 +47,19 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
## Choose a guide
| Task | Guides |
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
| Inspect assemblies, packages, or firmware | [.NET](managed-code-analysis.md), [Android](android-analysis.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
| Capture and compare a command | [Process capture](process-capture.md) |
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
| Task | Guides |
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
| Inspect assemblies, packages, or firmware | [Godot PCK](godot-pck.md), [.NET](managed-code-analysis.md), [Android](android-analysis.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
| Capture and compare a command | [Process capture](process-capture.md) |
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
For development, use [testing](testing.md), [client verification](client-verification.md),
[tool design](tool-design.md), and [releasing](releasing.md).
+5 -1
View File
@@ -38,6 +38,7 @@ import {
hasBinaryPlistSignature,
MAX_XML_PLIST_PROBE_BYTES,
} from "../domain/apple/appleArchiveFormat.js";
import { hasGodotPckHeader } from "../domain/godotPckFormat.js";
import {
hasZipSignature,
isZipPackageFormat,
@@ -325,6 +326,8 @@ const detectArtifactFormat = async (
if (lower.endsWith(".app")) return "app-pack";
return zipPackageFormatForPath(lower) ?? "zip";
}
if (hasGodotPckHeader(magic.subarray(0, observed.bytesRead)))
return "godot-pck";
// Binary plists, including keyed archives, often use other extensions.
if (hasBinaryPlistSignature(magic.subarray(0, observed.bytesRead)))
return "plist";
@@ -404,7 +407,8 @@ const isArchiveFormat = (
| "raw-binary"
>,
): format is Extract<BinaryTarget, { kind: "archive" }>["format"] =>
isZipPackageFormat(format) || ["asar", "dmg", "pkg"].includes(format);
isZipPackageFormat(format) ||
["asar", "godot-pck", "dmg", "pkg"].includes(format);
const sha256Handle = async (
handle: FileHandle,
+10 -1
View File
@@ -12,6 +12,7 @@ import {
export type HashResult = ArtifactClassificationEvidence & {
readonly sha256: string;
readonly bytes: number;
readonly md5?: string;
};
/** Preserve the inventory cancellation reason at a stream boundary. */
@@ -28,8 +29,10 @@ export const abortIfNeeded = (signal?: AbortSignal): void => {
export const hashReadable = async (
stream: Readable,
signal?: AbortSignal,
integrityAlgorithm?: "sha256" | "md5",
): Promise<HashResult> => {
const hash = createHash("sha256");
const md5 = integrityAlgorithm === "md5" ? createHash("md5") : undefined;
const probe = createArtifactClassificationProbe();
let bytes = 0;
for await (const raw of stream) {
@@ -37,8 +40,14 @@ export const hashReadable = async (
const chunk = streamChunkToBuffer(raw);
bytes += chunk.length;
hash.update(chunk);
md5?.update(chunk);
probe.write(chunk);
}
abortIfNeeded(signal);
return { sha256: hash.digest("hex"), bytes, ...probe.finish(bytes) };
return {
sha256: hash.digest("hex"),
...(md5 === undefined ? {} : { md5: md5.digest("hex") }),
bytes,
...probe.finish(bytes),
};
};
+54 -9
View File
@@ -1,3 +1,8 @@
import type {
ArtifactIntegrityClaim,
ArtifactIntegrityDetails,
} from "../domain/artifactIntegrity.js";
import type { ArtifactFormatMetadata } from "../domain/artifactFormatMetadata.js";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import type { AnalysisCleanupLease } from "./AnalysisCleanupLease.js";
import type { Readable } from "node:stream";
@@ -34,7 +39,8 @@ export interface ArtifactEntry {
readonly executable: boolean;
readonly encrypted: boolean;
readonly byteOffset: number | null;
readonly declaredSha256: string | null;
readonly declaredIntegrity: ArtifactIntegrityClaim | null;
readonly formatMetadata?: ArtifactFormatMetadata;
readonly unpacked: boolean;
readonly limitations: readonly string[];
readonly adapterKey: string;
@@ -73,6 +79,11 @@ export const sameArtifactSourceIdentity = (
export const copyArtifactEntry = (entry: ArtifactEntry): ArtifactEntry => ({
...entry,
limitations: [...entry.limitations],
declaredIntegrity:
entry.declaredIntegrity === null ? null : { ...entry.declaredIntegrity },
...(entry.formatMetadata === undefined
? {}
: { formatMetadata: { ...entry.formatMetadata } }),
...(entry.sourceIdentity === undefined
? {}
: { sourceIdentity: { ...entry.sourceIdentity } }),
@@ -90,7 +101,8 @@ export const sameArtifactEntry = (
entry.executable === produced.executable &&
entry.encrypted === produced.encrypted &&
entry.byteOffset === produced.byteOffset &&
entry.declaredSha256 === produced.declaredSha256 &&
sameDeclaredIntegrity(entry.declaredIntegrity, produced.declaredIntegrity) &&
sameFormatMetadata(entry.formatMetadata, produced.formatMetadata) &&
entry.unpacked === produced.unpacked &&
entry.adapterKey === produced.adapterKey &&
entry.limitations.length === produced.limitations.length &&
@@ -104,6 +116,32 @@ export const sameArtifactEntry = (
entry.sourceIdentity?.mtimeNs === produced.sourceIdentity?.mtimeNs &&
entry.sourceIdentity?.ctimeNs === produced.sourceIdentity?.ctimeNs;
const sameDeclaredIntegrity = (
left: ArtifactEntry["declaredIntegrity"],
right: ArtifactEntry["declaredIntegrity"],
): boolean =>
left === right ||
(left !== null &&
left !== undefined &&
right !== null &&
right !== undefined &&
left.algorithm === right.algorithm &&
left.digest === right.digest);
/** Format metadata contracts contain only primitive source fields. */
const sameFormatMetadata = (
left: ArtifactFormatMetadata | undefined,
right: ArtifactFormatMetadata | undefined,
): boolean =>
left === right ||
(left !== undefined &&
right !== undefined &&
Object.keys(left).length === Object.keys(right).length &&
Object.entries(left).every(
([key, value]) =>
Object.hasOwn(right, key) && Reflect.get(right, key) === value,
));
/** One entry or subtree deliberately excluded from a traversal. */
export interface ArtifactTraversalOmission {
readonly path: string;
@@ -119,17 +157,29 @@ export interface ArtifactReaderTraversal {
/** Per-iterator observation sink; adapters report intentional omissions without shared state. */
export interface ArtifactTraversalOptions {
readonly onOmission?: (omission: ArtifactTraversalOmission) => void;
readonly onCoverage?: (coverage: ArtifactReaderTraversal) => void;
}
/** Read-only adapter over one directory, archive, or virtual container. */
export interface ArtifactReader {
readonly format: "directory" | ZipPackageFormat | "asar" | "file";
readonly format:
| "directory"
| ZipPackageFormat
| "asar"
| "godot-pck"
| "file";
entries(
signal?: AbortSignal,
options?: ArtifactTraversalOptions,
): AsyncIterable<ArtifactEntry>;
open(entry: ArtifactEntry, signal?: AbortSignal): Promise<Readable>;
provenance(): readonly ArtifactCommand[];
formatMetadata?(): ArtifactFormatMetadata | undefined;
/** Bind materialization to the exact inventoried carrier before creating outputs. */
prepareContainer?(
expectedSha256: string,
signal?: AbortSignal,
): Promise<void>;
close(): Promise<void>;
}
@@ -153,12 +203,7 @@ export class ArtifactReaderFailure extends Error {
| "unavailable",
message: string,
options?: ArtifactReaderFailureOptions,
readonly details?: Readonly<{
logicalPath: string;
declaredSha256: string | null;
calculatedSha256: string | null;
unpacked: boolean;
}>,
readonly details?: ArtifactIntegrityDetails,
) {
super(message, options);
this.name = "ArtifactReaderFailure";
+10 -14
View File
@@ -1,3 +1,4 @@
import { artifactIntegrityDetails } from "../domain/artifactIntegrity.js";
import { constants, createWriteStream, type Stats } from "node:fs";
import {
lstat,
@@ -100,11 +101,11 @@ export class AsarArtifactReader implements ArtifactReader {
executable: metadata.executable === true,
encrypted: false,
byteOffset: null,
declaredSha256:
declaredIntegrity:
metadata.integrity !== undefined &&
metadata.integrity.algorithm === "SHA256" &&
/^[a-f0-9]{64}$/u.test(metadata.integrity.hash)
? metadata.integrity.hash
? { algorithm: "sha256", digest: metadata.integrity.hash }
: null,
unpacked: "unpacked" in metadata && metadata.unpacked === true,
limitations:
@@ -481,12 +482,12 @@ export class AsarArtifactReader implements ArtifactReader {
"integrity",
`ASAR unpacked entry changed before read: ${entry.path}`,
{},
{
logicalPath: entry.path,
declaredSha256: entry.declaredSha256,
calculatedSha256: null,
unpacked: true,
},
artifactIntegrityDetails(
entry.path,
entry.declaredIntegrity,
null,
true,
),
);
}
abortIfNeeded(signal);
@@ -636,12 +637,7 @@ const asarUnpackedEntryFailure = (
"unavailable",
`ASAR unpacked entry bytes are unavailable: ${entry.path}`,
{ cause },
{
logicalPath: entry.path,
declaredSha256: entry.declaredSha256,
calculatedSha256: null,
unpacked: true,
},
artifactIntegrityDetails(entry.path, entry.declaredIntegrity, null, true),
);
return asarFailure(archivePath, `read ${entry.path}`, cause);
};
+1 -1
View File
@@ -453,7 +453,7 @@ const artifactEntry = (
executable: hostExecutableMarking(absolute, metadata),
encrypted: false,
byteOffset: null,
declaredSha256: null,
declaredIntegrity: null,
unpacked: false,
limitations:
kind === "symlink"
@@ -0,0 +1,315 @@
import {
buildGodotPck,
buildAliasedGodotPck,
embedGodotPck,
pckU32,
pckU64,
type GodotPckMember,
} from "../../tests/fixtures/godotPck.js";
import { rename, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { text } from "node:stream/consumers";
import { describe, expect, it } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { openRegularFile } from "../filesystem/RegularFile.js";
import type { ArtifactEntry } from "./ArtifactReader.js";
import { GodotPckArtifactReader } from "./GodotPckArtifactReader.js";
const writePck = async (bytes: Buffer): Promise<string> => {
const path = join(await createTestTempDirectory("rea-godot-pck-"), "game");
await writeFile(path, bytes);
return path;
};
const list = async (
reader: GodotPckArtifactReader,
): Promise<ArtifactEntry[]> => {
const entries: ArtifactEntry[] = [];
for await (const entry of reader.entries()) entries.push(entry);
return entries;
};
const withReader = async <T>(
bytes: Buffer,
use: (reader: GodotPckArtifactReader) => Promise<T>,
): Promise<T> => {
const reader = new GodotPckArtifactReader(await writePck(bytes));
try {
return await use(reader);
} finally {
await reader.close();
}
};
const members: readonly GodotPckMember[] = [
{ path: "res://scenes/main.gd", data: "extends Node\n" },
{ path: "project.binary", data: "" },
{ path: "data/é.json", data: '{"session":"Session1"}' },
];
describe("GodotPckArtifactReader", () => {
it.each([
["format 4 standalone", buildGodotPck(4, members), false],
["format 4 embedded", embedGodotPck(buildGodotPck(4, members)), true],
["format 3 standalone", buildGodotPck(3, members), false],
["format 2 standalone", buildGodotPck(2, members), false],
["format 3 embedded", embedGodotPck(buildGodotPck(3, members)), true],
[
"format 2 embedded with relative file base",
embedGodotPck(buildGodotPck(2, members, { packFlags: 2 })),
true,
],
])(
"lists and reads every member of a %s pack",
async (_, bytes, embedded) => {
await withReader(bytes, async (reader) => {
const entries = await list(reader);
expect(entries.map(({ path }) => path)).toEqual([
"scenes/main.gd",
"project.binary",
"data/é.json",
]);
expect(reader.formatMetadata()).toMatchObject({
embedded,
engine_major: 4,
engine_minor: 6,
engine_patch: 1,
declared_file_count: 3,
});
const contents = await Promise.all(
entries.map(async (entry) => text(await reader.open(entry))),
);
expect(contents).toEqual(members.map(({ data }) => data));
expect(entries[1]?.limitations).toEqual([]);
});
},
);
it("preserves declared MD5 for the shared inventory integrity verifier", async () => {
const bytes = buildGodotPck(3, [
{ path: "a.gd", data: "real", md5: Buffer.alloc(16, 1) },
]);
await withReader(bytes, async (reader) => {
const [entry] = await list(reader);
if (entry === undefined) throw new Error("expected one entry");
expect(entry.declaredIntegrity).toEqual({
algorithm: "md5",
digest: "01".repeat(16),
});
expect(await text(await reader.open(entry))).toBe("real");
});
});
it("reports encrypted members without reading them", async () => {
const bytes = buildGodotPck(3, [
{ path: "secret.gd", data: "x", flags: 1 },
]);
await withReader(bytes, async (reader) => {
const [entry] = await list(reader);
if (entry === undefined) throw new Error("expected one entry");
expect(entry.encrypted).toBe(true);
await expect(reader.open(entry)).rejects.toMatchObject({
reason: "unavailable",
});
});
});
it.each([
["encrypted", 3],
["sparse-bundle", 6],
] as const)(
"preserves %s header facts with unknown directory coverage",
async (status, flags) => {
await withReader(
buildGodotPck(4, members, { packFlags: flags }),
async (reader) => {
const coverage: unknown[] = [];
const entries: ArtifactEntry[] = [];
for await (const entry of reader.entries(undefined, {
onCoverage: (value) => coverage.push(value),
}))
entries.push(entry);
expect(entries).toEqual([]);
expect(reader.formatMetadata()).toMatchObject({
directory_status: status,
declared_file_count: 3,
format_version: 4,
});
expect(coverage).toEqual([
expect.objectContaining({ status: "unknown" }),
]);
},
);
},
);
it("admits a bounded virtual path independently of the host PATH_MAX", async () => {
const path = "a".repeat(4097);
await withReader(buildGodotPck(4, [{ path, data: "" }]), async (reader) => {
expect((await list(reader))[0]?.path).toBe(path);
});
});
it("rejects oversized encoded path metadata before allocating its declared bytes", async () => {
const bytes = buildGodotPck(2, members);
bytes.writeUInt32LE(0xffff_ffff, 100);
await withReader(bytes, async (reader) => {
await expect(list(reader)).rejects.toMatchObject({
reason: "limit",
boundary: "memory",
});
});
});
it.each([
[
"a patch removal entry",
buildGodotPck(3, [{ path: "gone.gd", data: "", flags: 2 }]),
"unavailable",
],
[
"a member count that cannot fit",
buildGodotPck(3, members, { fileCount: 0xffff_ffff }),
"format",
],
[
"member data beyond the pack",
buildGodotPck(3, [{ path: "big.bin", data: "x", size: 1n << 40n }]),
"format",
],
[
"a Godot 3 format 1 pack",
Buffer.concat([Buffer.from("GDPC"), pckU32(1), Buffer.alloc(120)]),
"unavailable",
],
["bytes without a pack", Buffer.from("not a godot pack at all"), "format"],
[
"a trailer pointing outside the file",
Buffer.concat([
Buffer.from("engine"),
pckU64(1n << 40n),
Buffer.from("GDPC"),
]),
"format",
],
])("refuses %s", async (_, bytes, reason) => {
await withReader(bytes, async (reader) => {
await expect(list(reader)).rejects.toMatchObject({ reason });
});
});
});
describe("GodotPckArtifactReader member streams", () => {
it("types cancellation during a member read instead of crashing", async () => {
const data = "x".repeat(4 * 1024 * 1024);
await withReader(
buildGodotPck(3, [{ path: "big.bin", data }]),
async (reader) => {
const [entry] = await list(reader);
if (entry === undefined) throw new Error("expected one entry");
const controller = new AbortController();
const stream = await reader.open(entry, controller.signal);
const chunks = stream[Symbol.asyncIterator]();
await chunks.next();
controller.abort();
await expect(chunks.next()).rejects.toMatchObject({
reason: "cancelled",
});
},
);
});
it("keeps the reader usable after a consumer abandons a member", async () => {
const data = "x".repeat(4 * 1024 * 1024);
await withReader(
buildGodotPck(3, [{ path: "big.bin", data }]),
async (reader) => {
const [entry] = await list(reader);
if (entry === undefined) throw new Error("expected one entry");
const abandoned = await reader.open(entry);
await abandoned[Symbol.asyncIterator]().next();
abandoned.destroy();
expect((await text(await reader.open(entry))).length).toBe(data.length);
},
);
});
it("reads members from the admitted descriptor after the path is replaced", async () => {
const path = await writePck(buildGodotPck(3, members));
const handle = await openRegularFile(path, { symlinks: "reject" });
try {
const replacement = join(dirname(path), "replacement");
await writeFile(
replacement,
buildGodotPck(3, [{ path: "other.gd", data: "" }]),
);
await rename(replacement, path);
const reader = new GodotPckArtifactReader(path, {
handle,
initial: await handle.stat(),
});
const entries = await list(reader);
expect(entries.map(({ path: member }) => member)).toEqual([
"scenes/main.gd",
"project.binary",
"data/é.json",
]);
await reader.close();
} finally {
await handle.close();
}
});
});
describe("Godot PCK logical-byte admission", () => {
it.each([2, 3, 4] as const)(
"rejects total repeated plaintext work before yielding format %s members",
async (version) => {
await withReader(buildAliasedGodotPck(version, 1100), async (reader) => {
let emitted = 0;
let failure: unknown;
try {
for await (const _entry of reader.entries()) emitted++;
} catch (cause) {
failure = cause;
}
expect(failure).toMatchObject({ reason: "limit" });
expect(emitted).toBe(0);
});
},
);
it("uses the actual pack span rather than granting budget for an embedded carrier", async () => {
const pack = buildAliasedGodotPck(4, 1100);
const carrier = Buffer.alloc(1024 * 1024);
await withReader(
Buffer.concat([carrier, pack, pckU64(pack.length), Buffer.from("GDPC")]),
async (reader) => {
let failure: unknown;
try {
await list(reader);
} catch (cause) {
failure = cause;
}
expect(failure).toMatchObject({ reason: "limit" });
},
);
});
it("allows bounded aliases and never charges encrypted content that it will not read", async () => {
await withReader(buildAliasedGodotPck(4, 2, 4), async (reader) => {
const entries = await list(reader);
expect(await text(await reader.open(entries[0]!))).toBe("xxxx");
expect(await text(await reader.open(entries[1]!))).toBe("xxxx");
});
await withReader(
buildAliasedGodotPck(4, 1100, 256 * 1024, 1),
async (reader) => {
const entries = await list(reader);
expect(entries).toHaveLength(1100);
expect(entries.every((entry) => entry.encrypted)).toBe(true);
},
);
});
});
+564
View File
@@ -0,0 +1,564 @@
import { artifactExpansionBudgetBytes } from "../domain/artifactExpansionBudget.js";
import { hashReadable } from "./ArtifactHash.js";
import { artifactIntegrityDetails } from "../domain/artifactIntegrity.js";
import type { Stats } from "node:fs";
import type { FileHandle } from "node:fs/promises";
import { setImmediate } from "node:timers/promises";
import { Readable, Transform, pipeline } from "node:stream";
import {
ArtifactReaderFailure,
copyArtifactEntry,
sameArtifactEntry,
type ArtifactEntry,
type ArtifactReader,
type ArtifactTraversalOptions,
} from "./ArtifactReader.js";
import { ArtifactResourceScope } from "./ArtifactResourceScope.js";
import { assertArtifactReadActive } from "./ArtifactFileOperation.js";
import { locateGodotPck, readPckBytes } from "./GodotPckLocation.js";
import type { GodotPckMetadata } from "../domain/artifactFormatMetadata.js";
import { jsonValueByteLength } from "../domain/jsonValueByteLength.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { readFileHandleChunks } from "../filesystem/readFileHandleChunks.js";
import {
NonRegularFileReadError,
RegularFileAdmissionFailure,
RegularFileOperationCancelled,
openRegularFile,
sameRegularFileState,
type StableRegularFileDescriptor,
} from "../filesystem/RegularFile.js";
const PACK_DIR_ENCRYPTED = 1;
const PACK_REL_FILEBASE = 2;
const PACK_SPARSE_BUNDLE = 4;
const PACK_FILE_ENCRYPTED = 1;
const MINIMUM_ENTRY_BYTES = 40;
const DIRECTORY_CHUNK_BYTES = 64 * 1024;
/** Encoded retained entry metadata, including escaped paths and source facts. */
const MAX_DIRECTORY_METADATA_BYTES = 64 * 1024 * 1024;
const UTF8 = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
/** One descriptor, one validated directory, and bounded positional member reads. */
export class GodotPckArtifactReader implements ArtifactReader {
readonly format = "godot-pck" as const;
readonly #scope = new ArtifactResourceScope();
readonly #entries = new Map<string, ArtifactEntry>();
readonly #streams = new Set<Readable>();
#owner: OwnedFileHandle | undefined;
#initial: Stats | undefined;
#opening: Promise<FileHandle> | undefined;
#prepared: Promise<void> | undefined;
#header: GodotPckMetadata | undefined;
#closing = false;
#boundSha256: string | undefined;
constructor(
private readonly path: string,
private readonly admitted?: StableRegularFileDescriptor,
) {}
formatMetadata(): GodotPckMetadata | undefined {
return this.#header === undefined ? undefined : { ...this.#header };
}
async prepareContainer(
expectedSha256: string,
signal?: AbortSignal,
): Promise<void> {
await this.#scope.run(async () => {
this.#assertOpen(signal);
const handle = await this.#assertStable(signal);
const observed =
this.#boundSha256 ??
(
await hashReadable(
readFileHandleChunks(handle, {
signal,
cancellationFailure: (reason) =>
new ArtifactReaderFailure(
"cancelled",
"Godot PCK verification cancelled",
{ cause: reason },
),
}),
signal,
)
).sha256;
await this.#assertStable(signal);
if (observed !== expectedSha256)
throw new ArtifactReaderFailure(
"integrity",
"Godot PCK carrier changed after inventory",
undefined,
artifactIntegrityDetails(
".",
{ algorithm: "sha256", digest: expectedSha256 },
observed,
false,
),
);
this.#boundSha256 = observed;
});
}
async *entries(
signal?: AbortSignal,
options?: ArtifactTraversalOptions,
): AsyncIterable<ArtifactEntry> {
await this.#scope.run(async () => {
this.#assertOpen(signal);
this.#prepared ??= this.#prepare(signal);
await this.#prepared;
await this.#assertStable(signal);
});
if (this.#header?.directory_status !== "available") {
options?.onCoverage?.({
status: "unknown",
limitations: [
this.#header?.directory_status === "encrypted"
? "Godot PCK directory is encrypted; member identities and contents remain unknown."
: "Godot PCK sparse-bundle member bytes are external and were not traversed.",
],
});
return;
}
let index = 0;
for (const entry of this.#entries.values()) {
if (index++ % 256 === 0) await setImmediate();
this.#assertOpen(signal);
yield copyArtifactEntry(entry);
}
}
async open(entry: ArtifactEntry, signal?: AbortSignal): Promise<Readable> {
return this.#scope.run(async () => {
this.#assertOpen(signal);
const stored = this.#entries.get(entry.adapterKey);
if (stored === undefined || !sameArtifactEntry(entry, stored))
throw new ArtifactReaderFailure(
"integrity",
"Godot PCK entry does not match its owning inventory",
);
if (stored.encrypted)
throw new ArtifactReaderFailure(
"unavailable",
`Encrypted Godot PCK member is unsupported: ${stored.path}`,
);
const handle = await this.#assertStable(signal);
this.#assertOpen(signal);
if (stored.byteOffset === null || stored.declaredSize === null)
throw formatFailure("member has no physical byte range");
const source =
stored.declaredSize === 0
? Readable.from([])
: readFileHandleChunks(handle, {
start: stored.byteOffset,
end: stored.byteOffset + stored.declaredSize - 1,
signal,
cancellationFailure: (reason) =>
new ArtifactReaderFailure(
"cancelled",
"Godot PCK read cancelled",
{ cause: reason },
),
});
const output = this.#memberStream(stored, signal);
// The returned stream still reports failures to its consumer. Also handle
// retirement before a caller has attached its own error listener.
output.on("error", () => undefined);
this.#streams.add(output);
void this.#scope
.run(
() =>
new Promise<void>((resolve) => {
pipeline(source, output, () => {
this.#streams.delete(output);
resolve();
});
}),
)
.catch((cause: unknown) =>
output.destroy(
cause instanceof Error
? cause
: new Error("Godot PCK stream ownership failed", { cause }),
),
);
return output;
});
}
async close(): Promise<void> {
this.#closing = true;
for (const stream of this.#streams)
stream.destroy(
new ArtifactReaderFailure("unavailable", "Godot PCK reader closed"),
);
await this.#scope.close();
if (this.#owner !== undefined) {
await this.#owner.close();
this.#owner = undefined;
}
this.#entries.clear();
}
provenance(): readonly [] {
return [];
}
#assertOpen(signal?: AbortSignal): void {
assertArtifactReadActive(signal);
if (this.#closing)
throw new ArtifactReaderFailure(
"unavailable",
"Godot PCK reader is closed",
);
}
#open(signal?: AbortSignal): Promise<FileHandle> {
this.#opening ??= this.#admit(signal);
return this.#opening;
}
async #admit(signal?: AbortSignal): Promise<FileHandle> {
try {
const handle =
this.admitted?.handle ??
(await openRegularFile(this.path, { symlinks: "reject", signal }));
if (this.admitted === undefined)
this.#owner = new OwnedFileHandle(handle);
this.#initial = this.admitted?.initial ?? (await handle.stat());
return handle;
} catch (cause: unknown) {
if (cause instanceof RegularFileAdmissionFailure) {
this.#owner = cause.owner;
throw ArtifactReaderFailure.withCleanup(
admissionFailure(cause.cause, this.path),
ArtifactReaderFailure.cleanupObservation(
cause.cleanupCause,
this.path,
),
);
}
throw admissionFailure(cause, this.path);
}
}
async #assertStable(signal?: AbortSignal): Promise<FileHandle> {
assertArtifactReadActive(signal);
const handle = await this.#open(signal);
const observed = await handle.stat();
if (
this.#initial === undefined ||
!sameRegularFileState(this.#initial, observed)
)
throw new ArtifactReaderFailure(
"integrity",
`Godot PCK source identity changed: ${this.path}`,
);
assertArtifactReadActive(signal);
return handle;
}
async #prepare(signal?: AbortSignal): Promise<void> {
const handle = await this.#assertStable(signal);
const initial = this.#initial;
if (initial === undefined)
throw formatFailure("source identity was not admitted");
const header = await readGodotPckHeader(handle, initial.size, signal);
this.#header = header;
if (header.directory_status !== "available") return;
const cursor = new PckCursor(
handle,
header.directory_start + 4,
header.pack_end,
signal,
);
let retainedBytes = 0;
let declaredPlaintextBytes = 0n;
const expansionBudget = BigInt(
artifactExpansionBudgetBytes(header.pack_end - header.pack_start),
);
for (let index = 0; index < header.declared_file_count; index++) {
if (index % 256 === 0) await setImmediate();
assertArtifactReadActive(signal);
const pathBytes = await cursor.uint32();
// Check worst-case JSON escaping before decoding or allocating the path.
// Its source bytes, decoded string and retained metadata share one budget.
if (pathBytes * 12 > MAX_DIRECTORY_METADATA_BYTES - retainedBytes)
throw metadataLimit();
const rawPath = await cursor.bytes(pathBytes);
const offset = await cursor.uint64();
const size = await cursor.uint64();
const md5 = (await cursor.bytes(16)).toString("hex");
const flags = await cursor.uint32();
if ((flags & ~PACK_FILE_ENCRYPTED) !== 0)
throw new ArtifactReaderFailure(
"unavailable",
`Godot patch or unknown member flags are unsupported: 0x${flags.toString(16)}`,
);
const sourcePath = pckMemberPath(rawPath);
const start = toSafeOffset(
offset,
header.file_base,
header.pack_end,
sourcePath,
);
if (size > BigInt(header.pack_end - start))
throw formatFailure(`member exceeds the pack: ${sourcePath}`);
// Aliases are valid, but each path causes another hash/copy. Admit the
// complete directory before yielding anything, including all-zero claims.
if ((flags & PACK_FILE_ENCRYPTED) === 0) {
declaredPlaintextBytes += size;
if (declaredPlaintextBytes > expansionBudget)
throw new ArtifactReaderFailure(
"limit",
`Godot PCK readable entries declare ${String(declaredPlaintextBytes)} plaintext bytes, exceeding this pack's ${String(expansionBudget)}-byte expansion budget`,
);
}
const metadata = {
path: sourcePath.startsWith("res://")
? sourcePath.slice(6)
: sourcePath,
kind: "file" as const,
declaredSize: Number(size),
compressedSize: null,
executable: false,
encrypted: (flags & PACK_FILE_ENCRYPTED) !== 0,
byteOffset: start,
declaredIntegrity: /^0{32}$/u.test(md5)
? null
: { algorithm: "md5" as const, digest: md5 },
unpacked: false,
limitations: /^0{32}$/u.test(md5)
? ["Godot PCK declares an all-zero MD5; source integrity is unknown."]
: [],
adapterKey: `godot-pck-entry:${String(index)}`,
formatMetadata: {
kind: "godot-pck-entry" as const,
index,
source_path: sourcePath,
declared_offset: Number(offset),
flags,
md5_status: /^0{32}$/u.test(md5)
? ("all-zero" as const)
: ("declared" as const),
},
} satisfies ArtifactEntry;
retainedBytes += jsonValueByteLength(
metadata,
MAX_DIRECTORY_METADATA_BYTES - retainedBytes,
);
if (retainedBytes > MAX_DIRECTORY_METADATA_BYTES) throw metadataLimit();
this.#entries.set(metadata.adapterKey, metadata);
}
await this.#assertStable(signal);
}
#memberStream(entry: ArtifactEntry, signal?: AbortSignal): Transform {
let bytes = 0;
const check = async (): Promise<void> => {
if (bytes !== entry.declaredSize)
throw formatFailure(`member ended early: ${entry.path}`);
await this.#assertStable(signal);
};
return new Transform({
transform(chunk: Buffer, _encoding, callback) {
bytes += chunk.length;
callback(null, chunk);
},
flush(callback) {
void check().then(
() => callback(),
(cause: unknown) =>
callback(
cause instanceof Error
? cause
: new Error("Godot PCK read failed", { cause }),
),
);
},
});
}
}
/** Decode header fields once; unsupported content does not erase observed headers. */
export const readGodotPckHeader = async (
handle: FileHandle,
size: number,
signal?: AbortSignal,
): Promise<GodotPckMetadata> => {
const location = await locateGodotPck(handle, size, signal);
if (location === undefined)
throw formatFailure("no standalone header or executable trailer");
const { start, end, embedded } = location;
const fixed = await readPckBytes(handle, start, 40, end, signal);
const version = fixed.readUInt32LE(4);
if (![2, 3, 4].includes(version))
throw new ArtifactReaderFailure(
"unavailable",
`Godot PCK format ${String(version)} is unsupported; plain formats 2, 3 and 4 are supported`,
);
const flags = fixed.readUInt32LE(20);
if (
(flags & ~(PACK_DIR_ENCRYPTED | PACK_REL_FILEBASE | PACK_SPARSE_BUNDLE)) !==
0
)
throw new ArtifactReaderFailure(
"unavailable",
`Godot PCK declares unknown pack flags 0x${flags.toString(16)}`,
);
const fileBase = toSafeOffset(
fixed.readBigUInt64LE(24),
version >= 3 || (flags & PACK_REL_FILEBASE) !== 0 ? start : 0,
end,
"file base",
);
const directory =
version >= 3
? toSafeOffset(fixed.readBigUInt64LE(32), start, end, "directory")
: start + 96;
const count = (
await readPckBytes(handle, directory, 4, end, signal)
).readUInt32LE(0);
const status =
(flags & PACK_DIR_ENCRYPTED) !== 0
? "encrypted"
: (flags & PACK_SPARSE_BUNDLE) !== 0
? "sparse-bundle"
: "available";
if (
status === "available" &&
count * MINIMUM_ENTRY_BYTES > end - directory - 4
)
throw formatFailure("declared member count cannot fit in the directory");
return {
kind: "godot-pck",
format_version: version,
engine_major: fixed.readUInt32LE(8),
engine_minor: fixed.readUInt32LE(12),
engine_patch: fixed.readUInt32LE(16),
pack_start: start,
pack_end: end,
embedded,
pack_flags: flags,
file_base: fileBase,
directory_start: directory,
declared_file_count: count,
directory_status: status,
};
};
const pckMemberPath = (raw: Buffer): string => {
let end = raw.length;
while (end > 0 && raw[end - 1] === 0) end--;
let path: string;
try {
path = UTF8.decode(raw.subarray(0, end));
} catch (cause: unknown) {
throw new ArtifactReaderFailure(
"format",
"Godot PCK path is not valid UTF-8",
{ cause },
);
}
if (path.length === 0 || path === "res://" || path.includes("\0"))
throw formatFailure("empty or NUL-containing member path");
return path;
};
/** One bounded read window; no whole-pack buffer or per-member source reopening. */
class PckCursor {
#buffer: Buffer = Buffer.alloc(0);
#bufferStart = 0;
constructor(
private readonly handle: FileHandle,
private position: number,
private readonly end: number,
private readonly signal?: AbortSignal,
) {}
async bytes(length: number): Promise<Buffer> {
if (length > this.end - this.position)
throw formatFailure("directory entry exceeds the pack");
if (length > MAX_DIRECTORY_METADATA_BYTES / 12) throw metadataLimit();
if (length > DIRECTORY_CHUNK_BYTES) {
// A large valid virtual path is bounded by retained representation, not
// the host filesystem's PATH_MAX. Only this path receives a larger read.
const value = await readPckBytes(
this.handle,
this.position,
length,
this.end,
this.signal,
);
this.position += length;
return value;
}
const offset = this.position - this.#bufferStart;
if (offset < 0 || offset + length > this.#buffer.length) {
this.#buffer = await readPckBytes(
this.handle,
this.position,
Math.min(DIRECTORY_CHUNK_BYTES, this.end - this.position),
this.end,
this.signal,
);
this.#bufferStart = this.position;
}
const start = this.position - this.#bufferStart;
this.position += length;
return this.#buffer.subarray(start, start + length);
}
async uint32(): Promise<number> {
return (await this.bytes(4)).readUInt32LE(0);
}
async uint64(): Promise<bigint> {
return (await this.bytes(8)).readBigUInt64LE(0);
}
}
const toSafeOffset = (
offset: bigint,
base: number,
end: number,
subject: string,
): number => {
if (base > end || offset > BigInt(end - base))
throw formatFailure(`${subject} offset exceeds the pack`);
return base + Number(offset);
};
const formatFailure = (detail: string): ArtifactReaderFailure =>
new ArtifactReaderFailure("format", `Godot PCK is malformed: ${detail}`);
const metadataLimit = (): ArtifactReaderFailure =>
new ArtifactReaderFailure(
"limit",
"Godot PCK directory exceeds the 64 MiB encoded metadata budget",
{ boundary: "memory" },
);
const admissionFailure = (
cause: unknown,
path: string,
): ArtifactReaderFailure => {
if (cause instanceof ArtifactReaderFailure) return cause;
if (cause instanceof RegularFileOperationCancelled)
return new ArtifactReaderFailure(
"cancelled",
"Godot PCK admission cancelled",
{ cause: cause.reason },
);
if (cause instanceof NonRegularFileReadError)
return new ArtifactReaderFailure(
"format",
`Godot PCK source is not a regular file: ${path}`,
{ cause },
);
return new ArtifactReaderFailure(
"io",
cause instanceof Error
? cause.message
: `Cannot admit Godot PCK source: ${path}`,
{ cause },
);
};
+87
View File
@@ -0,0 +1,87 @@
import type { FileHandle } from "node:fs/promises";
import { GODOT_PCK_MAGIC } from "../domain/godotPckFormat.js";
import { ArtifactReaderFailure } from "./ArtifactReader.js";
import { assertArtifactReadActive } from "./ArtifactFileOperation.js";
export interface GodotPckLocation {
readonly start: number;
readonly end: number;
readonly embedded: boolean;
}
/** Positional exact reads preserve short-read, cancellation and I/O semantics. */
export const readPckBytes = async (
handle: Pick<FileHandle, "read">,
position: number,
length: number,
end: number,
signal?: AbortSignal,
): Promise<Buffer> => {
if (
!Number.isSafeInteger(position) ||
!Number.isSafeInteger(length) ||
length < 0 ||
position < 0 ||
length > end - position
)
throw new ArtifactReaderFailure(
"format",
"Godot PCK read exceeds its source range",
);
const buffer = Buffer.alloc(length);
let consumed = 0;
while (consumed < length) {
assertArtifactReadActive(signal);
const { bytesRead } = await handle.read(
buffer,
consumed,
length - consumed,
position + consumed,
);
assertArtifactReadActive(signal);
if (bytesRead === 0)
throw new ArtifactReaderFailure(
"format",
"Godot PCK source is truncated",
);
consumed += bytesRead;
}
return buffer;
};
/** Recognize a standalone header or the exact engine trailer; never scan payloads. */
export const locateGodotPck = async (
handle: Pick<FileHandle, "read">,
size: number,
signal?: AbortSignal,
prefix?: Uint8Array,
): Promise<GodotPckLocation | undefined> => {
assertArtifactReadActive(signal);
if (!Number.isSafeInteger(size) || size < 4) return undefined;
const magic =
prefix !== undefined && prefix.length >= 4
? Buffer.from(prefix.buffer, prefix.byteOffset, 4)
: await readPckBytes(handle, 0, 4, size, signal);
if (magic.readUInt32LE(0) === GODOT_PCK_MAGIC)
return { start: 0, end: size, embedded: false };
if (size < 12) return undefined;
const trailer = await readPckBytes(handle, size - 12, 12, size, signal);
if (trailer.readUInt32LE(8) !== GODOT_PCK_MAGIC) return undefined;
const end = size - 12;
const bytes = trailer.readBigUInt64LE(0);
if (bytes < 4n || bytes > BigInt(end))
throw new ArtifactReaderFailure(
"format",
"Godot PCK trailer declares an invalid pack range",
);
const start = end - Number(bytes);
if (
(await readPckBytes(handle, start, 4, end, signal)).readUInt32LE(0) !==
GODOT_PCK_MAGIC
)
throw new ArtifactReaderFailure(
"format",
"Godot PCK trailer does not point at a pack header",
);
return { start, end, embedded: true };
};
+1 -1
View File
@@ -186,7 +186,7 @@ export class MachOSliceArtifactReader implements ArtifactReader {
executable: true,
encrypted: false,
byteOffset: offset,
declaredSha256: null,
declaredIntegrity: null,
unpacked: false,
limitations: [
...(structural.status === "not-mach-o"
+4 -23
View File
@@ -14,11 +14,7 @@ import { describe, expect, it } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { ZipArtifactReader } from "./ZipArtifactReader.js";
import {
ARTIFACT_DECOMPRESSION_RATIO,
artifactDecompressionBudgetBytes,
MINIMUM_ARTIFACT_DECOMPRESSION_BYTES,
} from "../domain/artifactDecompressionBudget.js";
import { MINIMUM_ARTIFACT_EXPANDED_BYTES } from "../domain/artifactExpansionBudget.js";
/** Small real ZIP carriers with forged sizes; metadata-only controls, never decompressed. */
const writeZipWithDeclaredSize = async (
@@ -75,7 +71,7 @@ describe("ZipArtifactReader decompression budget", () => {
it("charges all declarations in one traversal before yielding an over-budget member", async () => {
const directory = await createTestTempDirectory("rea-zip-budget-total-");
const half = MINIMUM_ARTIFACT_DECOMPRESSION_BYTES / 2;
const half = MINIMUM_ARTIFACT_EXPANDED_BYTES / 2;
const path = await writeZipWithDeclaredSize(directory, [half, half + 1]);
const reader = new ZipArtifactReader(path, "zip");
const iterator = reader.entries()[Symbol.asyncIterator]();
@@ -93,8 +89,8 @@ describe("ZipArtifactReader decompression budget", () => {
"rea-zip-budget-iterators-",
);
const path = await writeZipWithDeclaredSize(directory, [
MINIMUM_ARTIFACT_DECOMPRESSION_BYTES / 2,
MINIMUM_ARTIFACT_DECOMPRESSION_BYTES / 2,
MINIMUM_ARTIFACT_EXPANDED_BYTES / 2,
MINIMUM_ARTIFACT_EXPANDED_BYTES / 2,
]);
const reader = new ZipArtifactReader(path, "zip");
const first = reader.entries()[Symbol.asyncIterator]();
@@ -177,21 +173,6 @@ describe("ZipArtifactReader decompression budget", () => {
});
});
describe("artifactDecompressionBudgetBytes", () => {
it("grants at least the floor and grows with the compressed size", () => {
expect(artifactDecompressionBudgetBytes(0)).toBe(
MINIMUM_ARTIFACT_DECOMPRESSION_BYTES,
);
expect(artifactDecompressionBudgetBytes(1024)).toBe(
MINIMUM_ARTIFACT_DECOMPRESSION_BYTES,
);
const large = MINIMUM_ARTIFACT_DECOMPRESSION_BYTES * 4;
expect(artifactDecompressionBudgetBytes(large)).toBe(
large * ARTIFACT_DECOMPRESSION_RATIO,
);
});
});
it.each([null, { caller: "opaque ZIP interruption" }])(
"preserves the exact ZIP cancellation reason at enumeration and open: %j",
async (reason) => {
+7 -7
View File
@@ -36,10 +36,10 @@ import {
} from "./ArtifactReader.js";
import type { ZipPackageFormat } from "../domain/zipPackageFormat.js";
import {
ARTIFACT_DECOMPRESSION_RATIO,
artifactDecompressionBudgetBytes,
MINIMUM_ARTIFACT_DECOMPRESSION_BYTES,
} from "../domain/artifactDecompressionBudget.js";
ARTIFACT_EXPANSION_RATIO,
artifactExpansionBudgetBytes,
MINIMUM_ARTIFACT_EXPANDED_BYTES,
} from "../domain/artifactExpansionBudget.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import {
NonRegularFileReadError,
@@ -209,11 +209,11 @@ export class ZipArtifactReader implements ArtifactReader {
metadata.declaredSize >= 0
) {
declaredFileBytes += metadata.declaredSize;
const budget = artifactDecompressionBudgetBytes(this.#source.size);
const budget = artifactExpansionBudgetBytes(this.#source.size);
if (declaredFileBytes > budget)
throw new ArtifactReaderFailure(
"limit",
`ZIP entries declare ${String(declaredFileBytes)} uncompressed bytes, exceeding this archive's ${String(budget)}-byte decompression budget (${String(MINIMUM_ARTIFACT_DECOMPRESSION_BYTES)}-byte minimum, ${String(ARTIFACT_DECOMPRESSION_RATIO)}x archive size); a small archive cannot expand this far`,
`ZIP entries declare ${String(declaredFileBytes)} uncompressed bytes, exceeding this archive's ${String(budget)}-byte decompression budget (${String(MINIMUM_ARTIFACT_EXPANDED_BYTES)}-byte minimum, ${String(ARTIFACT_EXPANSION_RATIO)}x archive size); a small archive cannot expand this far`,
);
}
const stored = this.#entries.get(metadata.adapterKey);
@@ -289,7 +289,7 @@ const projectZipEntry = (
executable: entry.executable,
encrypted: entry.encrypted,
byteOffset: null,
declaredSha256: null,
declaredIntegrity: null,
unpacked: false,
limitations: [
...(kind === "symlink" ? ["Archive symlink target was not read."] : []),
+45 -9
View File
@@ -1,9 +1,12 @@
import { locateGodotPck } from "../GodotPckLocation.js";
import { withArtifactRegularFile } from "../ArtifactFileOperation.js";
import type { ArtifactCleanupAttempt } from "../ArtifactResourceScope.js";
import { lstat, realpath } from "node:fs/promises";
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
import { digestCanonicalValue } from "../../domain/canonicalDigest.js";
import { AsarArtifactReader } from "../AsarArtifactReader.js";
import { GodotPckArtifactReader } from "../GodotPckArtifactReader.js";
import {
ArtifactPathRegistry,
normalizeArtifactPath,
@@ -65,7 +68,7 @@ const extractArtifactInScope = async (
// Cancellation wins over every later refusal, including unsupported formats.
abortIfNeeded(signal);
const sourcePath = await realpath(input.inputPath);
await requireExtractableFormat(sourcePath, input);
await requireExtractableFormat(sourcePath, input, signal);
const snapshot = await scanArtifactInventoryInScope(sourcePath, {
resourceScope: input.resourceScope,
signal,
@@ -99,6 +102,18 @@ const materializeArtifactInventoryInScope = async (
signal?: AbortSignal,
): Promise<ArtifactExtractionResult> => {
abortIfNeeded(signal);
const containerMetadata = snapshot.occurrences.find(
(row) => row.logical_path === ".",
)?.format_metadata;
if (
containerMetadata?.kind === "godot-pck" &&
containerMetadata.directory_status !== "available"
)
throw new AnalysisUnsupportedTargetError(
"extract_artifact",
input.inputPath,
`Godot PCK directory is ${containerMetadata.directory_status}; its complete contents cannot be extracted`,
);
const selectedOccurrences = snapshot.occurrences.filter(
(occurrence) =>
(occurrence.entry_kind === "file" || occurrence.entry_kind === "slice") &&
@@ -225,7 +240,11 @@ const materializeSelection = async ({
);
byPath.set(path, item);
}
const reader = await createReader(sourcePath, input);
const reader = await createReader(
sourcePath,
input,
inventory.manifest.root_format,
);
const readerOwner: ArtifactResourceOwner = {
kind: "reader" as const,
reader,
@@ -236,6 +255,7 @@ const materializeSelection = async ({
let completed: ArtifactExtractionResult | undefined;
const extracted: ExtractedOccurrence[] = [];
try {
await reader.prepareContainer?.(inventory.manifest.root_sha256, signal);
try {
output = await SafeOutputTree.create(input.outputRoot);
} catch (cause: unknown) {
@@ -405,21 +425,35 @@ interface ExtractionInventory {
*/
const requireExtractableFormat = async (
path: string,
input: Pick<ArtifactExtractionInput, "inputPath" | "inputFormat">,
input: Pick<
ArtifactExtractionInput,
"inputPath" | "inputFormat" | "resourceScope"
>,
signal?: AbortSignal,
): Promise<boolean> => {
const format = input.inputFormat;
const directory = (await lstat(path)).isDirectory();
if (
!directory &&
format !== "asar" &&
format !== "godot-pck" &&
!isMachOFormat(format) &&
!isZipPackageFormat(format)
)
throw new AnalysisUnsupportedTargetError(
"extract_artifact",
input.inputPath,
`Artifact format has no extraction reader: ${format}`,
) {
const pack = await withArtifactRegularFile(
path,
input.resourceScope,
signal,
async (handle) =>
locateGodotPck(handle, (await handle.stat()).size, signal),
);
if (pack === undefined)
throw new AnalysisUnsupportedTargetError(
"extract_artifact",
input.inputPath,
`Artifact format has no extraction reader: ${format}`,
);
}
return directory;
};
@@ -427,10 +461,12 @@ const createReader = async (
path: string,
input: Pick<
ArtifactExtractionInput,
"inputPath" | "inputFormat" | "environment"
"inputPath" | "inputFormat" | "environment" | "resourceScope"
>,
containerFormat: ArtifactGraphManifest["root_format"],
): Promise<ArtifactReader> => {
const format = input.inputFormat;
if (containerFormat === "godot-pck") return new GodotPckArtifactReader(path);
if (await requireExtractableFormat(path, input))
return new DirectoryArtifactReader(path);
if (format === "asar") return new AsarArtifactReader(path);
@@ -20,7 +20,7 @@ const entry = (path: string, kind: ArtifactEntry["kind"]): ArtifactEntry => ({
executable: false,
encrypted: false,
byteOffset: null,
declaredSha256: null,
declaredIntegrity: null,
unpacked: false,
limitations: [],
adapterKey: path,
@@ -1,3 +1,4 @@
import { hasGodotPckHeader } from "../../domain/godotPckFormat.js";
import type { ArtifactClassificationEvidence } from "../ArtifactClassificationProbe.js";
import { ArtifactReaderFailure } from "../ArtifactReader.js";
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
@@ -49,6 +50,8 @@ export interface MutableOccurrence {
encrypted: boolean;
hash_status: ArtifactOccurrence["hash_status"];
source_location: ArtifactOccurrence["source_location"];
declared_integrity?: NonNullable<ArtifactOccurrence["declared_integrity"]>;
format_metadata?: NonNullable<ArtifactOccurrence["format_metadata"]>;
limitations: string[];
}
@@ -107,10 +110,18 @@ export const createOccurrence = (
encrypted: entry.encrypted,
hash_status: entry.encrypted ? "unavailable" : "not-hashed",
source_location:
entry.byteOffset === null || entry.declaredSize === null
entry.encrypted ||
entry.byteOffset === null ||
entry.declaredSize === null
? null
: { offset: entry.byteOffset, length: entry.declaredSize },
limitations: [...entry.limitations],
...(entry.declaredIntegrity === null
? {}
: { declared_integrity: { ...entry.declaredIntegrity } }),
...(entry.formatMetadata === undefined
? {}
: { format_metadata: { ...entry.formatMetadata } }),
};
};
@@ -231,6 +242,7 @@ export const rootOccurrenceFor = (
readonly executable: boolean;
readonly format: ArtifactOccurrence["artifact_format"];
readonly path: string;
readonly formatMetadata?: ArtifactOccurrence["format_metadata"];
},
): MutableOccurrence => ({
occurrence_id: occurrenceIdForLocation({
@@ -245,16 +257,21 @@ export const rootOccurrenceFor = (
artifact_kind:
metadata.format === "directory" ||
isZipPackageFormat(metadata.format) ||
["asar", "dmg", "pkg"].includes(metadata.format)
["asar", "godot-pck", "dmg", "pkg"].includes(metadata.format)
? "container"
: artifactRoleForFormat(metadata.path, metadata.format).kind,
artifact_format: metadata.format,
declared_size: metadata.size,
compressed_size: null,
executable: metadata.executable,
encrypted: false,
encrypted:
metadata.formatMetadata?.kind === "godot-pck" &&
metadata.formatMetadata.directory_status === "encrypted",
hash_status: "verified",
source_location: null,
...(metadata.formatMetadata === undefined
? {}
: { format_metadata: metadata.formatMetadata }),
limitations: [],
});
@@ -520,6 +537,7 @@ const artifactRoleForFormat = (
format: zipPackageFormatForPath(path) ?? format,
};
if (format === "plist") return { kind: "plist", format };
if (format === "godot-pck") return { kind: "container", format };
return {
kind: ["native-addon", "dynamic-library"].includes(byPath.kind)
? byPath.kind
@@ -534,6 +552,7 @@ export const classifyArtifactBytes = (
fileSize = prefix.length,
): ArtifactNode["format"] => {
if (hasZipSignature(prefix)) return "zip";
if (hasGodotPckHeader(prefix)) return "godot-pck";
if (hasBinaryPlistSignature(prefix)) return "plist";
if (hasXmlPlistSignature(prefix)) return "plist";
const fatFormat = classifyFatArtifact(prefix, fileSize);
+6 -1
View File
@@ -1,3 +1,4 @@
import { locateGodotPck } from "../GodotPckLocation.js";
import { sameRegularFileState } from "../../filesystem/RegularFile.js";
import type { FileHandle } from "node:fs/promises";
import type { Stats } from "node:fs";
@@ -109,7 +110,9 @@ export const classifyAndHashRootForInventory = async (
options.signal,
);
const retainSource =
isZipPackageFormat(format) || format === "mach-o-universal";
isZipPackageFormat(format) ||
format === "mach-o-universal" ||
format === "godot-pck";
outcome = {
kind: "completed",
value: {
@@ -166,6 +169,8 @@ const classifyRootFormat = async (
(await hasKolyTrailer(handle, archiveBytes, signal))
)
return "dmg";
if (await locateGodotPck(handle, archiveBytes, signal, evidence.prefix))
return "godot-pck";
return classifyArtifactContent(path, evidence, archiveBytes).format;
};
+14 -1
View File
@@ -4,6 +4,7 @@ import type {
ArtifactReaderTraversal,
} from "../ArtifactReader.js";
import { DirectoryArtifactReader } from "../DirectoryArtifactReader.js";
import { GodotPckArtifactReader } from "../GodotPckArtifactReader.js";
import { MachOSliceArtifactReader } from "../MachOSliceArtifactReader.js";
import { NativeDmgArtifactReader } from "../NativeDmgArtifactReader.js";
import { ZipArtifactReader } from "../ZipArtifactReader.js";
@@ -30,6 +31,8 @@ export const createReader = (
return new DirectoryArtifactReader(path);
case "asar":
return new AsarArtifactReader(path);
case "godot-pck":
return new GodotPckArtifactReader(path, rootSource);
case "mach-o-universal":
return process.platform === "darwin"
? new MachOSliceArtifactReader(path, environment, undefined, rootSource)
@@ -58,6 +61,11 @@ export const inventoryCoverage = (
...(isZipPackageFormat(reader.format)
? [ZIP_NON_ENTRY_LIMITATION]
: []),
...(reader.format === "godot-pck"
? [
"PCK inventory establishes packaged bytes and metadata; scenes, binary resources, GDScript bytecode and runtime signal behavior were not decoded.",
]
: []),
...traversal.limitations,
],
};
@@ -76,7 +84,12 @@ export const inventoryCoverage = (
"Universal Mach-O slices require the native macOS lipo adapter.",
],
};
if (isZipPackageFormat(format) || format === "directory" || format === "asar")
if (
isZipPackageFormat(format) ||
format === "directory" ||
format === "asar" ||
format === "godot-pck"
)
return {
coverage: { scope: "container-entries", status: "unknown" },
limitations: [`${format} child inventory reader is unavailable.`],
+3 -4
View File
@@ -220,6 +220,7 @@ const buildInventorySnapshot = async (
executable: hostExecutableMarking(path, metadata),
format: rootFormat,
path,
formatMetadata: reader?.formatMetadata?.(),
});
for (const occurrence of input.occurrences)
if (
@@ -312,14 +313,12 @@ const buildIntegrityContradictions = (
contradiction_id: artifactContradictionId({
rootArtifactId: rootNode.artifact_id,
logicalPath: occurrence.logical_path,
declaredSha256: contradiction.declaredSha256,
observedSha256: contradiction.observedSha256,
integrity: contradiction.integrity,
}),
occurrence_id: occurrence.occurrence_id,
parent_artifact_id: parentArtifactId,
logical_path: occurrence.logical_path,
declared_sha256: contradiction.declaredSha256,
observed_sha256: contradiction.observedSha256,
...contradiction.integrity,
entry_kind: contradiction.entryKind,
unpacked: contradiction.unpacked,
trust: "observed-untrusted",
+25 -14
View File
@@ -1,3 +1,8 @@
import {
artifactIntegrityPair,
artifactIntegrityDetails,
type ArtifactIntegrityPair,
} from "../../domain/artifactIntegrity.js";
import { isAbsolute } from "node:path";
import {
@@ -33,8 +38,7 @@ import { hashReadable } from "../ArtifactHash.js";
export interface PendingIntegrityContradiction {
readonly occurrence: MutableOccurrence;
readonly declaredSha256: string;
readonly observedSha256: string;
readonly integrity: ArtifactIntegrityPair;
readonly entryKind: "file" | "slice";
readonly unpacked: boolean;
}
@@ -60,6 +64,7 @@ interface ScanContext {
readonly expandedContainerIds: Set<string>;
readonly ownedReaders: ArtifactReader[];
readonly omissions: ArtifactTraversalOmission[];
readonly coverage: ArtifactReaderTraversal[];
}
export const scanReader = async (
@@ -97,6 +102,7 @@ export const scanReader = async (
expandedContainerIds: new Set<string>(),
ownedReaders,
omissions: [],
coverage: [],
};
await visitArtifactEntries(context, reader, "");
// Archive directories may appear after their children. Resolve containment
@@ -116,7 +122,9 @@ export const scanReader = async (
unknownParentOccurrences: parentResolution.unknownParentOccurrences,
traversal: {
status:
context.omissions.length === 0 && structuralLimitations.length === 0
context.omissions.length === 0 &&
structuralLimitations.length === 0 &&
context.coverage.every((value) => value.status === "complete")
? "complete"
: "unknown",
limitations: [
@@ -126,6 +134,7 @@ export const scanReader = async (
`Artifact traversal omitted ${kind} at ${JSON.stringify(path)}.`,
),
...structuralLimitations,
...context.coverage.flatMap((value) => value.limitations),
]),
].sort(compareUnicodeCodePoints),
},
@@ -139,6 +148,7 @@ const inventoryEntries = (
): AsyncIterator<ArtifactEntry> =>
reader
.entries(context.signal, {
onCoverage: (coverage) => context.coverage.push(coverage),
onOmission: (omission) =>
context.omissions.push({
...omission,
@@ -290,26 +300,27 @@ const digestArtifactEntry = async (
const digest = await hashReadable(
await currentReader.open(entry, context.signal),
context.signal,
entry.declaredIntegrity?.algorithm,
);
const mismatched =
entry.declaredSha256 !== null && entry.declaredSha256 !== digest.sha256;
if (mismatched && entry.declaredSha256 !== null) {
const claim = entry.declaredIntegrity;
const observed = claim?.algorithm === "md5" ? digest.md5 : digest.sha256;
if (claim !== null && observed === undefined)
throw new ArtifactReaderFailure(
"integrity",
"The selected integrity digest was not computed",
);
const mismatched = claim !== null && claim.digest !== observed;
if (mismatched && claim !== null && observed !== undefined) {
if (context.integrity.mode === "fail")
throw new ArtifactReaderFailure(
"integrity",
`Artifact integrity metadata disagrees with content: ${logicalPath}`,
undefined,
{
logicalPath,
declaredSha256: entry.declaredSha256,
calculatedSha256: digest.sha256,
unpacked: entry.unpacked,
},
artifactIntegrityDetails(logicalPath, claim, observed, entry.unpacked),
);
context.pendingContradictions.push({
occurrence,
declaredSha256: entry.declaredSha256,
observedSha256: digest.sha256,
integrity: artifactIntegrityPair(claim, observed),
entryKind: entry.kind,
unpacked: entry.unpacked,
});
+59 -23
View File
@@ -1,7 +1,8 @@
import { z } from "incur";
import type { Evidence } from "../domain/evidence.js";
import { parseEvidence, type Evidence } from "../domain/evidence.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { AnalysisErrorProjection } from "../domain/analysisErrorProjection.js";
import type { Result } from "../domain/result.js";
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
import { createEvidenceBundle } from "../domain/evidenceBundle.js";
@@ -25,20 +26,42 @@ export const analysisEvidenceDeliveryOptions = {
),
};
const isEvidence = (
value: Evidence | AnalysisErrorProjection,
): value is Evidence => "evidence_id" in value && "normalized_result" in value;
const isEvidence = (value: JsonValue): value is Evidence =>
typeof value === "object" &&
value !== null &&
!Array.isArray(value) &&
"evidence_id" in value &&
"normalized_result" in value;
/** One-shot delivery owns report publication; analysis keeps its own cleanup owner. */
export const deliverCliAnalysisEvidence = async (input: {
readonly run: () => Promise<Result<Evidence, AnalysisError>>;
readonly service: { close(): Promise<void> };
interface CliEvidenceDelivery {
readonly providerId: string;
readonly operation: string;
readonly signal: AbortSignal;
readonly detail: "complete" | "summary";
readonly evidenceOutput: string | undefined;
}) => {
}
export const cliEvidenceDeliveryInputError = (
input: Pick<CliEvidenceDelivery, "detail" | "evidenceOutput" | "operation">,
): AnalysisInputError | undefined =>
input.detail === "summary" && input.evidenceOutput === undefined
? new AnalysisInputError(input.operation, undefined, [
{
path: ["evidence_output"],
reason: "invalid_value",
message:
"Summary delivery requires --evidence_output so complete source Evidence remains available.",
},
])
: undefined;
/** One-shot delivery owns report publication; analysis keeps its own cleanup owner. */
export const deliverCliAnalysisEvidence = async (
input: CliEvidenceDelivery & {
readonly run: () => Promise<Result<Evidence, AnalysisError>>;
readonly service: { close(): Promise<void> };
},
) => {
const {
run,
service,
@@ -48,19 +71,10 @@ export const deliverCliAnalysisEvidence = async (input: {
detail,
evidenceOutput,
} = input;
if (detail === "summary" && evidenceOutput === undefined)
const inputError = cliEvidenceDeliveryInputError(input);
if (inputError !== undefined)
return finishAnalysisCommand(
async () => ({
ok: false,
error: new AnalysisInputError(operation, undefined, [
{
path: ["evidence_output"],
reason: "invalid_value",
message:
"Summary delivery requires --evidence_output so complete source Evidence remains available.",
},
]),
}),
async () => ({ ok: false, error: inputError }),
service,
providerId,
operation,
@@ -73,7 +87,29 @@ export const deliverCliAnalysisEvidence = async (input: {
operation,
signal,
);
if (!isEvidence(complete) || evidenceOutput === undefined) return complete;
return publishCliAnalysisEvidence(complete, {
providerId,
operation,
signal,
detail,
evidenceOutput,
});
};
/** Publish already finalized CLI analysis without opening a second provider/session. */
export const publishCliAnalysisEvidence = async (
value: JsonValue,
input: CliEvidenceDelivery,
) => {
const { providerId, operation, signal, detail, evidenceOutput } = input;
if (!isEvidence(value)) return value;
const complete = parseEvidence(value);
const inputError = cliEvidenceDeliveryInputError(input);
if (inputError !== undefined) {
inputError.retainPartialObservation(complete);
return projectAnalysisError(inputError);
}
if (evidenceOutput === undefined) return complete;
const resources = new ArtifactResourceScope();
let receipt: { readonly path: string; readonly bytes: number } | undefined;
const delivered = await finishAnalysisCommand(
+28 -8
View File
@@ -1,3 +1,11 @@
import {
analysisEvidenceDeliveryOptions,
cliEvidenceDeliveryInputError,
publishCliAnalysisEvidence,
} from "./analysisEvidenceDelivery.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { ARTIFACT_GRAPH_PROVIDER } from "../application/InvestigationProviders.js";
import { z } from "incur";
import type { DirectAnalysis } from "../composition/directAnalysis.js";
@@ -228,6 +236,7 @@ const registerInspectionCommand = (
path: z.string().min(1).describe("Application or package path"),
}),
options: z.object({
...analysisEvidenceDeliveryOptions,
integrityPolicy: z
.enum(["fail", "record-and-continue"])
.default("fail")
@@ -235,14 +244,25 @@ const registerInspectionCommand = (
}),
run: ({ args, options }) =>
logCliCommand(logger, "inspect-artifact", () =>
runProviderAnalysis(
args.path,
"inspect_artifact",
{
integrity_policy: options.integrityPolicy,
},
logger,
),
withCommandCancellation(async (signal) => {
const delivery = {
providerId: ARTIFACT_GRAPH_PROVIDER.id,
operation: "inspect_artifact",
signal,
detail: options.detail,
evidenceOutput: options.evidence_output,
};
const inputError = cliEvidenceDeliveryInputError(delivery);
if (inputError !== undefined) return projectAnalysisError(inputError);
const complete = await runProviderAnalysis(
args.path,
"inspect_artifact",
{ integrity_policy: options.integrityPolicy },
logger,
signal,
);
return publishCliAnalysisEvidence(complete, delivery);
}),
),
});
};
+1 -1
View File
@@ -17,7 +17,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
...toolContractMetadata("inspect_analysis_view"),
kind: "application",
description:
'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_macho, decode_interface_builder, inspect_evm_interface, inspect_managed_artifact, inspect_managed_members, inspect_android_package, inspect_dart_aot, inspect_il2cpp_metadata or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. For a retained record use source: {kind: "retained-evidence", evidence_id: "<parent_evidence_id>"}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: <complete Evidence>}. For layout, JavaScript, inventory, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Typed module references use view {kind: "javascript", collection: "modules", projection: "references", where: {path: "main.js", kind: "javascript-module"}, offset: 0, limit: 64}; where matches exact retained strings (null selects unavailable paths). Rows include canonical identity, retained source-range count and observation support states. Source coverage and selection remain explicit; exhausting matching rows does not prove runtime loading or complete analysis. For exact file/native facts and processing gaps, use view {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"]}. Rows page original observations with canonical identities and Evidence; missing properties remain omitted. Paths match recorded artifact-path/source-range locations; optional kinds filter nodes. To discover field names without values, opt into include_available_property_keys: true with property_keys: []. Canonical identity and Evidence are separate from properties; keep each node_id/observation_id pair together when composing results. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paLine truncated
'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_macho, decode_interface_builder, inspect_evm_interface, inspect_managed_artifact, inspect_managed_members, inspect_android_package, inspect_dart_aot, inspect_il2cpp_metadata or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. For a retained record use source: {kind: "retained-evidence", evidence_id: "<parent_evidence_id>"}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: <complete Evidence>}. For layout, JavaScript, inventory, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Typed module references use view {kind: "javascript", collection: "modules", projection: "references", where: {path: "main.js", kind: "javascript-module"}, offset: 0, limit: 64}; where matches exact retained strings (null selects unavailable paths). Rows include canonical identity, retained source-range count and observation support states. Source coverage and selection remain explicit; exhausting matching rows does not prove runtime loading or complete analysis. For exact file/native facts and processing gaps, use view {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"]}. Rows page original observations with canonical identities and Evidence; missing properties remain omitted. Paths match recorded artifact-path/source-range locations; optional kinds filter nodes. To discover field names without values, opt into include_available_property_keys: true with property_keys: []. Canonical identity and Evidence are separate from properties; keep each node_id/observation_id pair together when composing results. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions. Occurrences accept exLine truncated
inputSchema: inspectAnalysisViewInputSchema,
outputSchema: evidenceResultOf(analysisViewResultSchema),
examples: [
+2 -2
View File
@@ -40,7 +40,7 @@ export const artifactExtractionExecutionSchema =
const exampleInputSchema = z.record(z.string(), jsonValueSchema);
const examples: Readonly<Record<string, Readonly<Record<string, unknown>>>> = {
inspect_artifact: {},
inspect_artifact: { detail: "summary" },
extract_artifact: {},
decode_interface_builder: {},
inspect_asset_catalog: { offset: 0, limit: 1000 },
@@ -80,7 +80,7 @@ const artifact = <
export const ARTIFACT_TOOL_CONTRACTS = [
artifact(
"inspect_artifact",
"Inspect an active archive or application package in one call. By default returns the complete content-addressed artifact graph once, as the typed nested inventory Evidence, with root and integrity observations, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Optional detail summary retains the complete inspection and returns its derived summary view; read exact paths or inventory pages with inspect_analysis_view using parent_evidence_id. This changes delivery, not inventory coverage. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
"Inspect an active archive or application package in one call. By default returns the complete content-addressed artifact graph once, as the typed nested inventory Evidence, with root and integrity observations, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. For PCK and large containers prefer detail summary: it retains the complete inspection and returns its derived summary view; read exact paths, batch occurrence/content IDs and selected fields with inspect_analysis_view using parent_evidence_id. This changes delivery, not inventory coverage. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
artifactInspectionInputSchema,
),
artifact(
+14 -4
View File
@@ -47,14 +47,24 @@ export const analysisErrorProjectionSchema = z.discriminatedUnion("code", [
.object({
code: z.literal("artifact_integrity_mismatch"),
...common,
details: z
.object({
details: z.union([
z.strictObject({
logical_path: z.string(),
declared_sha256: z.string().nullable(),
calculated_sha256: z.string().nullable(),
unpacked: z.boolean(),
})
.strict(),
}),
z.strictObject({
logical_path: z.string(),
algorithm: z.literal("md5"),
declared_md5: z.string().regex(/^[a-f0-9]{32}$/u),
calculated_md5: z
.string()
.regex(/^[a-f0-9]{32}$/u)
.nullable(),
unpacked: z.boolean(),
}),
]),
})
.strict(),
generic("artifact_operation_failed"),
@@ -62,6 +62,7 @@ export const targetFormatSchema = z.enum([
"raw-binary",
...ZIP_PACKAGE_FORMATS,
"asar",
"godot-pck",
"dmg",
"pkg",
"plist",
+3 -1
View File
@@ -440,7 +440,9 @@ const artifactMessage = ({
return "Artifact processing is unavailable for the current target or host. Check artifact support and required tools.";
if (reason === "integrity" && INTEGRITY_POLICY_OPERATIONS.has(operation))
return `${
details?.unpacked === true && details.calculatedSha256 !== null
details?.unpacked === true &&
"calculatedSha256" in details &&
details.calculatedSha256 !== null
? `Declared ASAR integrity for unpacked entry ${details.logicalPath} contradicts its companion file; packaging tools commonly sign or strip unpacked native binaries after writing the archive.`
: "Artifact bytes contradict declared integrity."
} If expected, rerun ${operation === "inventory_artifact" ? "inspect_artifact" : operation} with integrity_policy=record-and-continue (CLI: --integrity-policy record-and-continue) to retain observed bytes as untrusted; otherwise get a fresh copy.`;
+2 -6
View File
@@ -1,3 +1,4 @@
import { projectArtifactIntegrityDetails } from "./artifactIntegrity.js";
import { jsonValueSchema, type JsonValue } from "./jsonValue.js";
import { isImmutableJsonSnapshot } from "./immutableJson.js";
@@ -335,12 +336,7 @@ const artifactStateErrorDetails = (
error: AnalysisError,
): Readonly<Record<string, JsonValue>> | undefined => {
if (error instanceof ArtifactOperationError && error.artifactDetails)
return {
logical_path: error.artifactDetails.logicalPath,
declared_sha256: error.artifactDetails.declaredSha256,
calculated_sha256: error.artifactDetails.calculatedSha256,
unpacked: error.artifactDetails.unpacked,
};
return projectArtifactIntegrityDetails(error.artifactDetails);
if (error instanceof ArtifactOperationError)
return {
operation: error.operation,
+1
View File
@@ -70,6 +70,7 @@ const formatSchema = z.enum([
"hsp",
"app-pack",
"asar",
"godot-pck",
"dmg",
"pkg",
"plist",
+13 -65
View File
@@ -32,10 +32,7 @@ import { safeParseOwnedBinaryLayout } from "../native/binaryLayout.js";
import { functionDossierSchema } from "../hopperValues.js";
import {
artifactGraphManifestSchema,
artifactNodeSchema,
artifactOccurrenceSchema,
artifactEdgeSchema,
integrityContradictionSchema,
artifactInventoryCoverageSchema,
} from "../artifactGraph.js";
import { artifactInspectionResultSchema } from "../artifactInspection.js";
import {
@@ -57,7 +54,12 @@ import {
evmViewItemSchema,
evmSummarySchema,
} from "./evmInterfaceView.js";
import { projectArtifactInventoryView } from "./artifactInventoryView.js";
import {
projectArtifactInventoryView,
artifactInventoryViewRequestSchema,
artifactInventoryItemsSchema,
artifactInventorySummarySchema,
} from "./artifactInventoryView.js";
import {
projectManagedArtifactView,
managedViewRequestSchema,
@@ -214,28 +216,6 @@ const nativeViewSchema = z.strictObject({
limit: z.number().int().positive().default(64),
});
/** Original inventory records in stable order, optionally matching exact paths. */
const inventoryViewSchema = z.discriminatedUnion("collection", [
z.strictObject({
kind: z.literal("inventory"),
collection: z.literal("occurrences"),
logical_paths: z
.array(z.string())
.optional()
.describe(
"Exact recorded logical paths. Omitted selects all available occurrences; an empty array selects none. This does not resolve filesystem paths.",
),
offset: z.number().int().nonnegative().default(0),
limit: z.number().int().positive().default(64),
}),
z.strictObject({
kind: z.literal("inventory"),
collection: z.enum(["nodes", "edges", "integrity_contradictions"]),
offset: z.number().int().nonnegative().default(0),
limit: z.number().int().positive().default(64),
}),
]);
/** Caller-selected projection of already completed analysis Evidence. */
export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
summaryViewSchema,
@@ -246,7 +226,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
javascriptModuleReferencesRequestSchema,
javascriptFactsViewRequestSchema,
nativeViewSchema,
inventoryViewSchema,
artifactInventoryViewRequestSchema,
wasmViewRequestSchema,
machoViewRequestSchema,
interfaceBuilderViewRequestSchema,
@@ -337,32 +317,6 @@ const javascriptSummarySchema = z.strictObject({
limitation_count: z.number().int().nonnegative(),
});
const inventorySummarySchema = z.strictObject({
manifest: artifactGraphManifestSchema,
available_counts: z.strictObject({
nodes: z.number().int().nonnegative(),
occurrences: z.number().int().nonnegative(),
edges: z.number().int().nonnegative(),
integrity_contradictions: z.number().int().nonnegative(),
}),
observed_entry_kind_counts: z.strictObject({
file: z.number().int().nonnegative(),
directory: z.number().int().nonnegative(),
symlink: z.number().int().nonnegative(),
slice: z.number().int().nonnegative(),
}),
source_complete: z.boolean(),
inventory_evidence_ids: z.array(prefixedDigestSchema("ev")).min(1),
inspection: z
.strictObject({
coverage: artifactInspectionResultSchema.shape.coverage,
unexplored_branches:
artifactInspectionResultSchema.shape.unexplored_branches,
})
.nullable(),
limitation_count: z.number().int().nonnegative(),
});
const viewResultBase = {
parent_evidence_id: prefixedDigestSchema("ev"),
parent_operation: z.enum([
@@ -436,7 +390,7 @@ const javascriptFactsViewResultSchema = z.strictObject({
export const artifactInventorySummaryViewResultSchema = z.strictObject({
...summaryViewResultBase,
parent_operation: z.enum(["inspect_artifact", "inventory_artifact"]),
summary: inventorySummarySchema,
summary: artifactInventorySummarySchema,
});
export const desktopApplicationSummaryViewResultSchema = z.strictObject({
@@ -550,7 +504,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
layoutSummarySchema,
desktopSummarySchema,
javascriptSummarySchema,
inventorySummarySchema,
artifactInventorySummarySchema,
wasmSummarySchema,
machoSummarySchema,
interfaceBuilderSummarySchema,
@@ -692,18 +646,12 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
}),
z.strictObject({
kind: z.literal("inventory"),
view: inventoryViewSchema,
view: artifactInventoryViewRequestSchema,
source_coverage: artifactInventoryCoverageSchema.exactOptional(),
manifest: artifactGraphManifestSchema,
inventory_evidence_ids: z.array(prefixedDigestSchema("ev")).min(1),
source_complete: z.boolean(),
items: z.array(
z.union([
artifactNodeSchema,
artifactOccurrenceSchema,
artifactEdgeSchema,
integrityContradictionSchema,
]),
),
items: artifactInventoryItemsSchema,
...viewResultBase,
}),
]);
@@ -0,0 +1,306 @@
import {
artifactIdForContent,
occurrenceIdForLocation,
artifactEdgeId,
artifactGraphDigest,
artifactManifestId,
} from "../artifactIdentity.js";
import { afterEach, expect, it, vi } from "vitest";
import { ARTIFACT_COMPARISON_EXAMPLE } from "../../contracts/artifactComparisonExample.js";
import { artifactInventoryResultSchema } from "../artifactGraph.js";
import { freezeJsonSnapshot } from "../immutableJson.js";
import {
artifactInventoryViewRequestSchema,
projectArtifactInventoryView,
} from "./artifactInventoryView.js";
import type { AnalysisViewParent } from "./analysisView.js";
const carrier = ARTIFACT_COMPARISON_EXAMPLE.left;
const freshInventory = () => {
const inventory = artifactInventoryResultSchema.parse(
carrier.normalized_result,
);
const root = inventory.occurrences[0]!;
const rootNode = inventory.nodes[0]!;
root.artifact_kind = "container";
root.artifact_format = "zip";
rootNode.format = "zip";
const sha256 = "1".repeat(64);
const child = {
...root,
occurrence_id: occurrenceIdForLocation({
rootArtifactId: rootNode.artifact_id,
logicalPath: "child.txt",
entryKind: "file",
}),
artifact_id: artifactIdForContent(sha256),
parent_occurrence_id: root.occurrence_id,
logical_path: "child.txt",
artifact_kind: "resource" as const,
artifact_format: "file" as const,
declared_size: 2,
limitations: [...root.limitations],
};
inventory.nodes.push({
...rootNode,
artifact_id: child.artifact_id,
sha256,
format: "file",
size: 2,
content_state: "embedded",
limitations: [...rootNode.limitations],
});
inventory.nodes.sort((left, right) =>
left.artifact_id < right.artifact_id
? -1
: left.artifact_id > right.artifact_id
? 1
: 0,
);
inventory.occurrences.push(child);
const edge = {
ordinal: 0,
parent_artifact_id: rootNode.artifact_id,
relation: "contains" as const,
occurrence_id: child.occurrence_id,
producer: null,
};
inventory.edges.push({ ...edge, edge_id: artifactEdgeId(edge, child) });
const graph = artifactGraphDigest({
nodes: inventory.nodes,
occurrences: inventory.occurrences,
edges: inventory.edges,
contradictions: inventory.integrity_contradictions,
});
inventory.manifest = {
...inventory.manifest,
root_format: "zip",
node_count: 2,
occurrence_count: 2,
edge_count: 1,
graph_sha256: graph,
manifest_id: artifactManifestId(rootNode.artifact_id, graph),
};
return inventory;
};
const parent = (normalizedResult: unknown): AnalysisViewParent => ({
evidenceId: carrier.evidence_id,
operation: "inventory_artifact",
subject:
carrier.subject === null
? null
: { ...carrier.subject, format: "zip", architecture: null },
normalizedResult,
sealedResult: true,
limitations: [],
});
const page = (source: AnalysisViewParent, view: unknown) => {
const result = projectArtifactInventoryView(
source,
artifactInventoryViewRequestSchema.parse(view),
);
if (!result.ok) throw result.error;
if (result.value.kind !== "inventory")
throw new Error("Expected an inventory page");
return result.value;
};
afterEach(() => vi.restoreAllMocks());
it("batches exact occurrence IDs in source order while retaining linkage with no selected fields", () => {
const inventory = freshInventory();
const selected = inventory.occurrences.slice(0, 2);
expect(selected).toHaveLength(2);
const result = page(parent(inventory), {
kind: "inventory",
collection: "occurrences",
occurrence_ids: [...selected]
.reverse()
.flatMap((row) => [row.occurrence_id, row.occurrence_id]),
fields: [],
});
expect(result.items).toEqual(
selected.map((row) => ({
occurrence_id: row.occurrence_id,
artifact_id: row.artifact_id,
parent_occurrence_id: row.parent_occurrence_id,
})),
);
expect(result.coverage.total).toBe(2);
});
it("intersects exact IDs and paths, then pages the filtered order", () => {
const inventory = freshInventory();
const selected = inventory.occurrences.slice(0, 2);
const result = page(parent(inventory), {
kind: "inventory",
collection: "occurrences",
occurrence_ids: selected.map((row) => row.occurrence_id),
logical_paths: selected.map((row) => row.logical_path),
fields: ["logical_path"],
offset: 1,
limit: 1,
});
expect(result.items).toEqual([
expect.objectContaining({
occurrence_id: selected[1]?.occurrence_id,
logical_path: selected[1]?.logical_path,
}),
]);
expect(result.coverage).toMatchObject({
total: 2,
examined: 1,
next_offset: null,
});
expect(
page(parent(inventory), {
kind: "inventory",
collection: "occurrences",
occurrence_ids: [selected[0]!.occurrence_id],
logical_paths: [selected[1]!.logical_path],
}).items,
).toEqual([]);
});
it.each([{ occurrence_ids: [] }, { logical_paths: [] }])(
"keeps an explicitly empty occurrence selection empty",
(filter) => {
expect(
page(parent(freshInventory()), {
kind: "inventory",
collection: "occurrences",
...filter,
}).items,
).toEqual([]);
},
);
it("fetches only selected content identities and fields, in canonical node order", () => {
const inventory = freshInventory();
const selected = inventory.nodes.slice(0, 2);
const result = page(parent(inventory), {
kind: "inventory",
collection: "nodes",
artifact_ids: [...selected].reverse().map((row) => row.artifact_id),
fields: ["sha256", "size"],
});
expect(result.items).toEqual(
selected.map((row) => ({
artifact_id: row.artifact_id,
sha256: row.sha256,
size: row.size,
})),
);
expect(
page(parent(inventory), {
kind: "inventory",
collection: "nodes",
artifact_ids: [selected[0]!.artifact_id],
fields: [],
}).items,
).toEqual([{ artifact_id: selected[0]!.artifact_id }]);
expect(
page(parent(inventory), {
kind: "inventory",
collection: "nodes",
artifact_ids: [],
}).items,
).toEqual([]);
});
it("does not fabricate a selected field absent from the owning report", () => {
const inventory = freshInventory();
expect(
page(parent(inventory), {
kind: "inventory",
collection: "occurrences",
fields: ["format_metadata"],
limit: 1,
}).items,
).toEqual([
{
occurrence_id: inventory.occurrences[0]!.occurrence_id,
artifact_id: inventory.occurrences[0]!.artifact_id,
parent_occurrence_id: inventory.occurrences[0]!.parent_occurrence_id,
},
]);
expect(() =>
artifactInventoryViewRequestSchema.parse({
kind: "inventory",
collection: "nodes",
fields: ["logical_path"],
}),
).toThrow();
});
it("borrows authenticated immutable facts and validates once across retained queries", () => {
const inventory = freezeJsonSnapshot(freshInventory());
const source = parent(inventory);
const parse = vi.spyOn(artifactInventoryResultSchema, "parse");
const first = page(source, {
kind: "inventory",
collection: "occurrences",
limit: 1,
});
expect(first.items[0]).toBe(inventory.occurrences[0]);
expect(parse).toHaveBeenCalledOnce();
page(source, {
kind: "inventory",
collection: "nodes",
artifact_ids: [inventory.nodes[0]!.artifact_id],
});
expect(parse).toHaveBeenCalledOnce();
});
it("does not reuse validation for a mutable object carrying a forged sealing flag", () => {
const inventory = freshInventory();
const source = parent(inventory);
page(source, { kind: "inventory", collection: "nodes", fields: [] });
inventory.manifest.graph_sha256 = "0".repeat(64);
expect(() =>
page(source, { kind: "inventory", collection: "nodes", fields: [] }),
).toThrow();
});
it("keeps a missing match unknown when retained inventory pages are partial", () => {
const inventory = freshInventory();
inventory.occurrences = inventory.occurrences.slice(0, 1);
const result = page(parent(inventory), {
kind: "inventory",
collection: "occurrences",
logical_paths: ["not-recorded"],
});
expect(result.source_complete).toBe(false);
expect(result.items).toEqual([]);
expect(result.unknowns.join(" ")).toContain(
"Missing matches do not establish absence",
);
});
it("binds reused facts to the current Evidence ID and validates its subject on every projection", () => {
const inventory = freezeJsonSnapshot(freshInventory());
const first = parent(inventory);
page(first, { kind: "inventory", collection: "nodes", fields: [] });
const next = { ...first, evidenceId: `ev_${"f".repeat(64)}` };
expect(
page(next, { kind: "inventory", collection: "nodes", fields: [] })
.inventory_evidence_ids,
).toEqual([next.evidenceId]);
const subject = next.subject;
if (subject === null || subject === undefined)
throw new Error("Expected a subject");
expect(() =>
page(
{
...next,
subject: { ...subject, digest: { sha256: "f".repeat(64) } },
},
{ kind: "inventory", collection: "nodes" },
),
).toThrow();
expect(() =>
page(
{ ...next, operation: "inspect_artifact" },
{ kind: "inventory", collection: "nodes" },
),
).toThrow();
});
+361 -26
View File
@@ -1,9 +1,26 @@
import {
artifactGraphManifestSchema,
artifactInventoryCoverageSchema,
artifactNodeSchema,
artifactOccurrenceSchema,
artifactEdgeSchema,
integrityContradictionSchema,
} from "../artifactGraph.js";
import { artifactInspectionResultSchema } from "../artifactInspection.js";
import {
artifactFormatMetadataSchema,
type ArtifactFormatMetadata,
} from "../artifactFormatMetadata.js";
import { prefixedDigestSchema } from "../digests.js";
import type { ArtifactOccurrence } from "../artifactGraph.js";
import { isImmutableJsonSnapshot } from "../immutableJson.js";
import { z } from "zod";
import { AnalysisInputError } from "../analysisErrorCore.js";
import type { AnalysisError } from "../analysisErrorBase.js";
import { uniqueSorted } from "../canonicalOrdering.js";
import { err, ok, type Result } from "../result.js";
import {
validateArtifactInventorySubject,
parseArtifactInspectionInventoryResult,
parseArtifactInventoryResult,
type InventorySet,
@@ -19,6 +36,87 @@ import {
pageViewCoverage,
} from "./analysisViewCoverage.js";
const page = {
kind: z.literal("inventory"),
offset: z.number().int().nonnegative().default(0),
limit: z.number().int().positive().default(64),
};
/** Exact selection intersects filters; record identities always survive projection. */
export const artifactInventoryViewRequestSchema = z.discriminatedUnion(
"collection",
[
z.strictObject({
...page,
collection: z.literal("occurrences"),
occurrence_ids: z.array(prefixedDigestSchema("occ")).optional(),
logical_paths: z.array(z.string()).optional(),
fields: z.array(artifactOccurrenceSchema.keyof()).optional(),
}),
z.strictObject({
...page,
collection: z.literal("nodes"),
artifact_ids: z.array(prefixedDigestSchema("art")).optional(),
fields: z.array(artifactNodeSchema.keyof()).optional(),
}),
z.strictObject({
...page,
collection: z.enum(["edges", "integrity_contradictions"]),
}),
],
);
export const artifactInventoryItemsSchema = z.array(
z.union([
artifactNodeSchema.partial().required({ artifact_id: true }).strict(),
artifactOccurrenceSchema
.partial()
.required({
occurrence_id: true,
artifact_id: true,
parent_occurrence_id: true,
})
.strict(),
artifactEdgeSchema,
integrityContradictionSchema,
]),
);
export const artifactInventorySummarySchema = z.strictObject({
source_coverage: artifactInventoryCoverageSchema.exactOptional(),
available_hash_status_counts: z
.strictObject({
verified: z.number().int().nonnegative(),
mismatched: z.number().int().nonnegative(),
unavailable: z.number().int().nonnegative(),
"not-hashed": z.number().int().nonnegative(),
})
.exactOptional(),
manifest: artifactGraphManifestSchema,
root_format_metadata: artifactFormatMetadataSchema.exactOptional(),
available_counts: z.strictObject({
nodes: z.number().int().nonnegative(),
occurrences: z.number().int().nonnegative(),
edges: z.number().int().nonnegative(),
integrity_contradictions: z.number().int().nonnegative(),
}),
observed_entry_kind_counts: z.strictObject({
file: z.number().int().nonnegative(),
directory: z.number().int().nonnegative(),
symlink: z.number().int().nonnegative(),
slice: z.number().int().nonnegative(),
}),
source_complete: z.boolean(),
inventory_evidence_ids: z.array(prefixedDigestSchema("ev")).min(1),
inspection: z
.strictObject({
coverage: artifactInspectionResultSchema.shape.coverage,
unexplored_branches:
artifactInspectionResultSchema.shape.unexplored_branches,
})
.nullable(),
limitation_count: z.number().int().nonnegative(),
});
const inputError = (message: string): AnalysisInputError =>
new AnalysisInputError("inspect_analysis_view", undefined, [
{
@@ -28,15 +126,62 @@ const inputError = (message: string): AnalysisInputError =>
},
]);
interface InventoryCounts {
readonly kinds: Readonly<Record<ArtifactOccurrence["entry_kind"], number>>;
readonly hashes: Readonly<Record<ArtifactOccurrence["hash_status"], number>>;
}
interface InventorySource {
readonly inventory: InventorySet;
readonly evidenceIds: readonly string[];
readonly inspection: ArtifactInspectionResult | null;
rootMetadata?: ArtifactFormatMetadata | null;
nodeIndex?: ReadonlyMap<string, number>;
index?: {
byId: ReadonlyMap<string, number>;
byPath: ReadonlyMap<string, readonly number[]>;
};
counts?: InventoryCounts;
}
/** Reuse the owning inventory integrity/completeness parser for both carriers. */
const checkedSources = new WeakMap<
object,
{ readonly operation: string; readonly source: InventorySource }
>();
const parseSource = (
parent: AnalysisViewParent,
): Result<InventorySource, AnalysisError> => {
const key = parent.normalizedResult;
const reusable =
parent.sealedResult === true &&
typeof key === "object" &&
key !== null &&
isImmutableJsonSnapshot(key);
const cached = reusable ? checkedSources.get(key) : undefined;
if (cached !== undefined && cached.operation === parent.operation) {
try {
validateArtifactInventorySubject(
parent.subject ?? null,
cached.source.inventory.manifest,
);
} catch (cause: unknown) {
return sourceFailure(cause);
}
return ok(cached.source);
}
const parsed = parseSourceUncached(parent);
if (parsed.ok && reusable)
checkedSources.set(key, {
operation: parent.operation,
source: parsed.value,
});
return parsed;
};
const parseSourceUncached = (
parent: AnalysisViewParent,
): Result<InventorySource, AnalysisError> => {
try {
if (parent.operation === "inventory_artifact")
@@ -45,7 +190,7 @@ const parseSource = (
parent.normalizedResult,
parent.subject ?? null,
),
evidenceIds: [parent.evidenceId],
evidenceIds: [],
inspection: null,
});
const parsed = parseArtifactInspectionInventoryResult(
@@ -58,12 +203,16 @@ const parseSource = (
inspection: parsed.inspection,
});
} catch (cause: unknown) {
if (!(cause instanceof TypeError) && !(cause instanceof z.ZodError))
throw cause;
return err(inputError(cause.message));
return sourceFailure(cause);
}
};
const sourceFailure = (cause: unknown): Result<never, AnalysisError> => {
if (!(cause instanceof TypeError) && !(cause instanceof z.ZodError))
throw cause;
return err(inputError(cause.message));
};
/** Project exact source records; exhausted pages cover available records, not runtime behavior. */
export const projectArtifactInventoryView = (
parent: AnalysisViewParent,
@@ -82,7 +231,24 @@ export const projectArtifactInventoryView = (
);
const parsed = parseSource(parent);
if (!parsed.ok) return parsed;
const { inventory, evidenceIds, inspection } = parsed.value;
const source = parsed.value;
const { inventory, inspection } = source;
const evidenceIds =
inspection === null ? [parent.evidenceId] : source.evidenceIds;
if (source.rootMetadata === undefined)
source.rootMetadata =
inventory.occurrences.find((row) => row.logical_path === ".")
?.format_metadata ?? null;
const rootMetadata = source.rootMetadata;
const includeCoverage =
view.kind === "inventory" || inventory.manifest.root_format === "godot-pck";
const sourceCoverage = includeCoverage
? { source_coverage: inventory.coverage }
: {};
const counts =
includeCoverage || view.kind === "summary"
? inventoryCounts(source)
: undefined;
const limitations = uniqueSorted([
...parent.limitations,
...inventory.limitations,
@@ -99,6 +265,18 @@ export const projectArtifactInventoryView = (
limitations,
unknowns: [
"Inventory does not establish application runtime behavior or complete format-specific analysis.",
...(includeCoverage && inventory.coverage.status === "unknown"
? [
"Producer traversal coverage is unknown; missing matches do not prove source absence.",
]
: []),
...(includeCoverage &&
counts !== undefined &&
(counts.hashes.unavailable > 0 || counts.hashes["not-hashed"] > 0)
? [
"Content hashes are unknown for some recorded files; missing node IDs do not prove content absence.",
]
: []),
...(inventory.complete
? []
: [
@@ -115,9 +293,7 @@ export const projectArtifactInventoryView = (
| "unknowns"
>;
if (view.kind === "summary") {
const counts = { file: 0, directory: 0, symlink: 0, slice: 0 };
for (const occurrence of inventory.occurrences)
counts[occurrence.entry_kind]++;
const counts = inventoryCounts(source);
const examined =
inventory.nodes.length +
inventory.occurrences.length +
@@ -133,9 +309,16 @@ export const projectArtifactInventoryView = (
edges: inventory.edges.length,
integrity_contradictions: inventory.integrityContradictions.length,
},
observed_entry_kind_counts: counts,
observed_entry_kind_counts: counts.kinds,
...(includeCoverage
? { available_hash_status_counts: counts.hashes }
: {}),
source_complete: inventory.complete,
...sourceCoverage,
inventory_evidence_ids: [...evidenceIds],
...(rootMetadata === null
? {}
: { root_format_metadata: rootMetadata }),
inspection:
inspection === null
? null
@@ -149,29 +332,181 @@ export const projectArtifactInventoryView = (
...shared,
});
}
const paths =
view.collection === "occurrences" && view.logical_paths !== undefined
? new Set(view.logical_paths)
: undefined;
const rows =
view.collection === "occurrences" && paths !== undefined
? inventory.occurrences.filter((row) => paths.has(row.logical_path))
: view.collection === "occurrences"
? inventory.occurrences
: view.collection === "nodes"
? inventory.nodes
: view.collection === "edges"
? inventory.edges
: inventory.integrityContradictions;
const items = rows.slice(view.offset, view.offset + view.limit);
const selected =
view.collection === "occurrences"
? occurrencePage(source, view)
: view.collection === "nodes"
? nodePage(source, view)
: {
items: (view.collection === "edges"
? inventory.edges
: inventory.integrityContradictions
).slice(view.offset, view.offset + view.limit),
total:
view.collection === "edges"
? inventory.edges.length
: inventory.integrityContradictions.length,
};
return ok({
kind: "inventory",
view,
manifest: inventory.manifest,
inventory_evidence_ids: [...evidenceIds],
source_complete: inventory.complete,
items,
coverage: pageViewCoverage(view.offset, items.length, rows.length),
...sourceCoverage,
items: selected.items,
coverage: pageViewCoverage(
view.offset,
selected.items.length,
selected.total,
),
...shared,
});
};
const occurrencePage = (source: InventorySource, view: OccurrenceView) => {
const indices = occurrenceIndices(source, view);
const rows = source.inventory.occurrences;
const total = indices?.length ?? rows.length;
const items = [];
for (
let at = view.offset;
at < Math.min(total, view.offset + view.limit);
at++
)
items.push(selectOccurrenceFields(rows[indices?.[at] ?? at]!, view.fields));
return { items, total };
};
const nodePage = (
source: InventorySource,
view: Extract<
AnalysisViewRequest,
{ kind: "inventory"; collection: "nodes" }
>,
) => {
const indices = nodeIndices(source, view.artifact_ids);
const rows = source.inventory.nodes;
const total = indices?.length ?? rows.length;
const items = [];
for (
let at = view.offset;
at < Math.min(total, view.offset + view.limit);
at++
) {
const row = rows[indices?.[at] ?? at]!;
items.push(
view.fields === undefined
? row
: { ...selectFields(row, view.fields), artifact_id: row.artifact_id },
);
}
return { items, total };
};
const inventoryCounts = (source: InventorySource): InventoryCounts => {
if (source.counts !== undefined) return source.counts;
const kinds = { file: 0, directory: 0, symlink: 0, slice: 0 };
const hashes = {
verified: 0,
mismatched: 0,
unavailable: 0,
"not-hashed": 0,
};
for (const occurrence of source.inventory.occurrences) {
kinds[occurrence.entry_kind]++;
if (occurrence.entry_kind === "file" || occurrence.entry_kind === "slice")
hashes[occurrence.hash_status]++;
}
source.counts = Object.freeze({
kinds: Object.freeze(kinds),
hashes: Object.freeze(hashes),
});
return source.counts;
};
type OccurrenceView = Extract<
AnalysisViewRequest,
{ kind: "inventory"; collection: "occurrences" }
>;
/** Index authenticated facts only when an exact lookup is requested. */
const occurrenceIndices = (
source: InventorySource,
view: OccurrenceView,
): number[] | undefined => {
if (view.occurrence_ids === undefined && view.logical_paths === undefined)
return undefined;
if (view.occurrence_ids?.length === 0 || view.logical_paths?.length === 0)
return [];
if (source.index === undefined) {
const byId = new Map<string, number>();
const byPath = new Map<string, number[]>();
for (const [index, row] of source.inventory.occurrences.entries()) {
byId.set(row.occurrence_id, index);
const existing = byPath.get(row.logical_path);
if (existing === undefined) byPath.set(row.logical_path, [index]);
else existing.push(index);
}
source.index = { byId, byPath };
}
const lookup = source.index;
const paths =
view.logical_paths === undefined ? undefined : new Set(view.logical_paths);
const selected =
view.occurrence_ids === undefined
? [...(paths ?? [])].flatMap((path) => lookup.byPath.get(path) ?? [])
: [...new Set(view.occurrence_ids)].flatMap((id) => {
const index = lookup.byId.get(id);
return index === undefined ||
(paths !== undefined &&
!paths.has(source.inventory.occurrences[index]!.logical_path))
? []
: [index];
});
return selected.sort((left, right) => left - right);
};
const nodeIndices = (
source: InventorySource,
ids: string[] | undefined,
): number[] | undefined => {
if (ids === undefined) return undefined;
if (ids.length === 0) return [];
const byId = (source.nodeIndex ??= new Map(
source.inventory.nodes.map((row, index) => [row.artifact_id, index]),
));
return [...new Set(ids)]
.flatMap((id) => {
const index = byId.get(id);
return index === undefined ? [] : [index];
})
.sort((left, right) => left - right);
};
const selectFields = <Row extends object>(
row: Row,
fields: (keyof Row)[] | undefined,
): Partial<Row> => {
if (fields === undefined) return row;
const selected: Partial<Row> = {};
for (const field of fields) {
const value = row[field];
if (value !== undefined) selected[field] = value;
}
return selected;
};
/** Parent linkage and content identity survive even an explicitly empty field set. */
const selectOccurrenceFields = (
row: ArtifactOccurrence,
fields: OccurrenceView["fields"],
) =>
fields === undefined
? row
: {
...selectFields(row, fields),
occurrence_id: row.occurrence_id,
artifact_id: row.artifact_id,
parent_occurrence_id: row.parent_occurrence_id,
};
-35
View File
@@ -1,35 +0,0 @@
/**
* Aggregate decompression ceiling for one archive-backed artifact operation.
*
* A ZIP/APK/IPA/MSIX member is attacker-controlled and can expand far beyond the
* archive that carries it, so a small input can drive unbounded decompression
* work during inventory hashing and extraction. A fixed per-entry byte cap is
* the wrong shape: it rejects legitimately large members inside large packages
* and cannot distinguish a bomb from a large file.
*
* The ceiling instead scales with the archive's own compressed size, so a large
* package keeps a proportionally large budget while a small archive cannot
* amplify into unbounded CPU time or disk writes. The floor keeps ordinary
* small archives usable, and the ratio sits above real-world artifact
* compression (measured ~3:1-15:1, up to ~350:1 for pure text) but below the
* DEFLATE ceiling (~1030:1) that a single-entry bomb relies on.
*/
/** Allow a whole archive to expand by this factor before decompression is refused. */
export const ARTIFACT_DECOMPRESSION_RATIO = 500;
/**
* Smallest aggregate budget granted to any archive, so small packages stay usable.
* This floor keeps ordinary archives extractable and stays well above the largest
* single members real packages carry; it is not a per-entry cap.
*/
export const MINIMUM_ARTIFACT_DECOMPRESSION_BYTES = 256 * 1024 * 1024;
/** Largest total decompressed bytes allowed for an archive of this compressed size. */
export const artifactDecompressionBudgetBytes = (
compressedBytes: number,
): number =>
Math.max(
MINIMUM_ARTIFACT_DECOMPRESSION_BYTES,
Math.floor(compressedBytes) * ARTIFACT_DECOMPRESSION_RATIO,
);
@@ -0,0 +1,25 @@
import { describe, expect, it } from "vitest";
import {
ARTIFACT_EXPANSION_RATIO,
artifactExpansionBudgetBytes,
MINIMUM_ARTIFACT_EXPANDED_BYTES,
} from "./artifactExpansionBudget.js";
describe("artifactExpansionBudgetBytes", () => {
it("grants at least the floor and grows with the encoded archive size", () => {
expect(artifactExpansionBudgetBytes(0)).toBe(
MINIMUM_ARTIFACT_EXPANDED_BYTES,
);
expect(artifactExpansionBudgetBytes(1024)).toBe(
MINIMUM_ARTIFACT_EXPANDED_BYTES,
);
const large = MINIMUM_ARTIFACT_EXPANDED_BYTES * 4;
expect(artifactExpansionBudgetBytes(large)).toBe(
large * ARTIFACT_EXPANSION_RATIO,
);
});
it("keeps cumulative byte accounting within the exact number domain", () => {
expect(artifactExpansionBudgetBytes(Number.MAX_SAFE_INTEGER)).toBe(
Number.MAX_SAFE_INTEGER,
);
});
});
+23
View File
@@ -0,0 +1,23 @@
/**
* Shared aggregate plaintext-work ceiling for archive-backed operations.
* ZIP compression and PCK aliases can both multiply hashing and output bytes.
* Charge every readable logical occurrence, including repeated physical ranges.
*
* Keep the existing ZIP policy: a 256 MiB floor and 500 times the encoded archive
* size. Large real packages retain proportionally large budgets without granting
* a fixed per-member allowance. PCK callers use the actual pack span, excluding
* an embedded executable carrier. The safe-integer ceiling preserves exact byte
* accounting at the JSON/number boundary.
*/
export const ARTIFACT_EXPANSION_RATIO = 500;
export const MINIMUM_ARTIFACT_EXPANDED_BYTES = 256 * 1024 * 1024;
/** Largest total readable plaintext bytes for this encoded archive span. */
export const artifactExpansionBudgetBytes = (archiveBytes: number): number =>
Math.max(
MINIMUM_ARTIFACT_EXPANDED_BYTES,
Math.min(
Number.MAX_SAFE_INTEGER,
Math.floor(archiveBytes) * ARTIFACT_EXPANSION_RATIO,
),
);
+38
View File
@@ -0,0 +1,38 @@
import { z } from "zod";
const offset = z.number().int().min(0).max(Number.MAX_SAFE_INTEGER);
const uint32 = z.number().int().min(0).max(0xffff_ffff);
/** Parsed source fields owned by the PCK adapter, independent of content hashes. */
export const godotPckMetadataSchema = z.strictObject({
kind: z.literal("godot-pck"),
format_version: uint32,
engine_major: uint32,
engine_minor: uint32,
engine_patch: uint32,
pack_start: offset,
pack_end: offset,
embedded: z.boolean(),
pack_flags: uint32,
file_base: offset,
directory_start: offset,
declared_file_count: uint32,
directory_status: z.enum(["available", "encrypted", "sparse-bundle"]),
});
export type GodotPckMetadata = z.infer<typeof godotPckMetadataSchema>;
export const godotPckEntryMetadataSchema = z.strictObject({
kind: z.literal("godot-pck-entry"),
index: uint32,
source_path: z.string(),
declared_offset: offset,
flags: uint32,
md5_status: z.enum(["declared", "all-zero"]),
});
export const artifactFormatMetadataSchema = z.discriminatedUnion("kind", [
godotPckMetadataSchema,
godotPckEntryMetadataSchema,
]);
export type ArtifactFormatMetadata = z.infer<
typeof artifactFormatMetadataSchema
>;
+17 -3
View File
@@ -1,3 +1,9 @@
import {
artifactIntegrityClaimSchema,
artifactSha256PairSchema,
artifactMd5PairSchema,
} from "./artifactIntegrity.js";
import { artifactFormatMetadataSchema } from "./artifactFormatMetadata.js";
import { z } from "zod";
import { prefixedDigestSchema } from "./../domain/digests.js";
@@ -54,6 +60,7 @@ const artifactFormatSchema = z.enum([
"zip",
"pkg",
"asar",
"godot-pck",
"javascript-bundle",
"source-map",
"mach-o-universal",
@@ -82,6 +89,7 @@ const artifactFormatSchema = z.enum([
/** Formats established from bytes or a virtual directory tree, independent of path roles. */
const artifactContentFormatSchema = artifactFormatSchema.extract([
"zip",
"godot-pck",
"mach-o-universal",
"mach-o",
"elf",
@@ -149,17 +157,18 @@ export const artifactOccurrenceSchema = z.object({
encrypted: z.boolean(),
hash_status: z.enum(["verified", "mismatched", "not-hashed", "unavailable"]),
source_location: artifactByteRangeSchema.nullable(),
declared_integrity: artifactIntegrityClaimSchema.exactOptional(),
format_metadata: artifactFormatMetadataSchema.exactOptional(),
limitations: z.array(z.string()),
});
/** Observed bytes that contradict a container's declared integrity identity. */
export const integrityContradictionSchema = z.object({
const integrityContradictionBaseSchema = z.object({
contradiction_id: contradictionIdSchema,
occurrence_id: occurrenceIdSchema,
parent_artifact_id: artifactIdSchema,
logical_path: boundedRelativePathSchema,
declared_sha256: sha256Schema,
observed_sha256: sha256Schema,
entry_kind: z.enum(["file", "slice"]),
unpacked: z.boolean(),
trust: z.literal("observed-untrusted"),
@@ -167,6 +176,11 @@ export const integrityContradictionSchema = z.object({
limitations: z.array(z.string()).min(1),
});
export const integrityContradictionSchema = z.union([
integrityContradictionBaseSchema.extend(artifactSha256PairSchema.shape),
integrityContradictionBaseSchema.extend(artifactMd5PairSchema.shape),
]);
/**
* Parent-child derivation relation between two content-addressed artifacts.
* The child artifact and its logical path are those of the edge's occurrence.
+19 -8
View File
@@ -1,3 +1,7 @@
import {
artifactIntegrityPairOf,
type ArtifactIntegrityPair,
} from "./artifactIntegrity.js";
import { posix } from "node:path";
import { digestCanonicalValue } from "./canonicalDigest.js";
@@ -82,18 +86,25 @@ export const artifactManifestId = (
)}`;
/** Identify one integrity disagreement at its observed logical path. */
export const artifactContradictionId = (input: {
readonly rootArtifactId: string;
readonly logicalPath: string;
readonly declaredSha256: string;
readonly observedSha256: string;
}): string =>
export const artifactContradictionId = (
input: {
readonly rootArtifactId: string;
readonly logicalPath: string;
} & (
| { readonly declaredSha256: string; readonly observedSha256: string }
| { readonly integrity: ArtifactIntegrityPair }
),
): string =>
`ic_${digestCanonicalValue(
{
root_artifact_id: input.rootArtifactId,
logical_path: input.logicalPath,
declared_sha256: input.declaredSha256,
observed_sha256: input.observedSha256,
...("integrity" in input
? artifactIntegrityPairOf(input.integrity)
: {
declared_sha256: input.declaredSha256,
observed_sha256: input.observedSha256,
}),
},
"Artifact",
)}`;
+13 -5
View File
@@ -1,3 +1,8 @@
import {
artifactIntegrityPairOf,
artifactSha256PairSchema,
artifactMd5PairSchema,
} from "./artifactIntegrity.js";
import { z } from "zod";
import {
@@ -30,15 +35,19 @@ const inspectionHypothesisSchema = z.strictObject({
limitation: textSchema,
});
const inspectionContradictionSchema = z.strictObject({
const inspectionContradictionBaseSchema = z.strictObject({
contradiction_id: prefixedDigestSchema("aic"),
statement: textSchema,
occurrence_id: prefixedDigestSchema("occ"),
declared_sha256: z.string().regex(/^[a-f0-9]{64}$/u),
observed_sha256: z.string().regex(/^[a-f0-9]{64}$/u),
evidence_id: evidenceIdSchema,
});
const inspectionContradictionSchema = z.union([
inspectionContradictionBaseSchema.extend(artifactSha256PairSchema.shape),
inspectionContradictionBaseSchema.extend(artifactMd5PairSchema.shape),
]);
const nextProbeSchema = z.strictObject({
operation: textSchema,
rationale: textSchema,
@@ -269,8 +278,7 @@ const contradictions = (
const semantic = {
statement: `Declared integrity for ${value.logical_path} contradicts observed bytes.`,
occurrence_id: value.occurrence_id,
declared_sha256: value.declared_sha256,
observed_sha256: value.observed_sha256,
...artifactIntegrityPairOf(value),
evidence_id: evidenceId,
};
return {
+101
View File
@@ -0,0 +1,101 @@
import { z } from "zod";
const sha256 = z.string().regex(/^[a-f0-9]{64}$/u);
const md5 = z.string().regex(/^[a-f0-9]{32}$/u);
/** A container's claim is distinct from the SHA-256 identity of observed bytes. */
export const artifactIntegrityClaimSchema = z.discriminatedUnion("algorithm", [
z.strictObject({ algorithm: z.literal("sha256"), digest: sha256 }),
z.strictObject({ algorithm: z.literal("md5"), digest: md5 }),
]);
export type ArtifactIntegrityClaim = z.infer<
typeof artifactIntegrityClaimSchema
>;
// Keep existing SHA-256 Evidence and its identities unchanged.
export const artifactSha256PairSchema = z.object({
declared_sha256: sha256,
observed_sha256: sha256,
});
export const artifactMd5PairSchema = z.object({
algorithm: z.literal("md5"),
declared_md5: md5,
observed_md5: md5,
});
export type ArtifactIntegrityPair =
| z.infer<typeof artifactSha256PairSchema>
| z.infer<typeof artifactMd5PairSchema>;
/** The canonical portable fields shared by inventory and inspection findings. */
export const artifactIntegrityPair = (
claim: ArtifactIntegrityClaim,
observed: string,
): ArtifactIntegrityPair =>
claim.algorithm === "sha256"
? { declared_sha256: claim.digest, observed_sha256: observed }
: { algorithm: "md5", declared_md5: claim.digest, observed_md5: observed };
/** Select only digest facts from an authenticated inventory contradiction. */
export const artifactIntegrityPairOf = (
pair: ArtifactIntegrityPair,
): ArtifactIntegrityPair =>
"declared_sha256" in pair
? {
declared_sha256: pair.declared_sha256,
observed_sha256: pair.observed_sha256,
}
: {
algorithm: pair.algorithm,
declared_md5: pair.declared_md5,
observed_md5: pair.observed_md5,
};
export type ArtifactIntegrityDetails = {
readonly logicalPath: string;
readonly unpacked: boolean;
} & (
| {
readonly declaredSha256: string | null;
readonly calculatedSha256: string | null;
}
| {
readonly algorithm: "md5";
readonly declaredMd5: string;
readonly calculatedMd5: string | null;
}
);
/** Preserve the existing error carrier while naming non-SHA digest algorithms. */
export const artifactIntegrityDetails = (
logicalPath: string,
claim: ArtifactIntegrityClaim | null,
observed: string | null,
unpacked: boolean,
): ArtifactIntegrityDetails => ({
logicalPath,
unpacked,
...(claim?.algorithm === "md5"
? {
algorithm: "md5" as const,
declaredMd5: claim.digest,
calculatedMd5: observed,
}
: { declaredSha256: claim?.digest ?? null, calculatedSha256: observed }),
});
export const projectArtifactIntegrityDetails = (
details: ArtifactIntegrityDetails,
) => ({
logical_path: details.logicalPath,
...("declaredSha256" in details
? {
declared_sha256: details.declaredSha256,
calculated_sha256: details.calculatedSha256,
}
: {
algorithm: details.algorithm,
declared_md5: details.declaredMd5,
calculated_md5: details.calculatedMd5,
}),
unpacked: details.unpacked,
});
+14 -1
View File
@@ -1,8 +1,12 @@
import { freezeJsonSnapshot } from "./immutableJson.js";
import { expect, it } from "vitest";
import { ARTIFACT_COMPARISON_EXAMPLE } from "../contracts/artifactComparisonExample.js";
import { createArtifactInspection } from "./artifactInspection.js";
import { parseArtifactInventoryEvidence } from "./artifactInventoryEvidence.js";
import {
parseArtifactInventoryEvidence,
parseArtifactInspectionInventoryResult,
} from "./artifactInventoryEvidence.js";
import { createEvidence } from "./evidence.js";
import { jsonValueSchema } from "./jsonValue.js";
@@ -73,3 +77,12 @@ it("rejects a substep evidence ID that disagrees with its authenticated nested E
"Artifact inspection substep evidence ID does not match nested Evidence",
);
});
it("retains the authenticated parent's nested facts without a second inventory copy", () => {
const owned = freezeJsonSnapshot(createArtifactInspection(inventoryEvidence));
const parsed = parseArtifactInspectionInventoryResult(owned, target);
expect(parsed.inspection).toBe(owned);
expect(parsed.inventory.nodes[0]).toBe(
owned.substeps[0]!.evidence.normalized_result.nodes[0],
);
});
+32 -6
View File
@@ -1,3 +1,9 @@
import {
canBorrowParsedJsonSnapshot,
isImmutableJsonSnapshot,
freezeJsonSnapshot,
immutableJsonSnapshotMember,
} from "./immutableJson.js";
import {
artifactInventoryResultSchema,
type ArtifactInventoryCoverage,
@@ -100,13 +106,29 @@ const parseInspectionInventoryPayload = (
readonly pages: readonly ArtifactInventoryResult[];
readonly evidenceIds: readonly string[];
} => {
const inspection = artifactInspectionResultSchema.parse(input);
const parsedInspection = artifactInspectionResultSchema.parse(input);
const inspection = canBorrowParsedJsonSnapshot(input, parsedInspection)
? input
: isImmutableJsonSnapshot(input)
? freezeJsonSnapshot(parsedInspection)
: parsedInspection;
const sealedSubsteps = immutableJsonSnapshotMember(inspection, "substeps");
if (inspection.coverage.substeps_completed !== inspection.substeps.length)
throw new TypeError(
"Artifact inspection substep count does not match its coverage declaration",
);
const substeps = inspection.substeps.map((step) => {
const evidence = parseEvidence(step.evidence);
const substeps = inspection.substeps.map((step, index) => {
const sealedStep = immutableJsonSnapshotMember(
sealedSubsteps,
String(index),
);
const sourceEvidence =
immutableJsonSnapshotMember(sealedStep, "evidence") ?? step.evidence;
const parsedEvidence = parseEvidence(sourceEvidence);
const evidence = canBorrowParsedJsonSnapshot(sourceEvidence, parsedEvidence)
? sourceEvidence
: parsedEvidence;
immutableJsonSnapshotMember(evidence, "normalized_result");
if (step.evidence_id !== evidence.evidence_id)
throw new TypeError(
"Artifact inspection substep evidence ID does not match nested Evidence",
@@ -164,7 +186,12 @@ const parseInventoryPage = (
input: unknown,
subject: Evidence["subject"],
): ArtifactInventoryResult => {
const inventory = artifactInventoryResultSchema.parse(input);
const parsed = artifactInventoryResultSchema.parse(input);
const inventory = canBorrowParsedJsonSnapshot(input, parsed)
? input
: isImmutableJsonSnapshot(input)
? freezeJsonSnapshot(parsed)
: parsed;
validateArtifactInventorySubject(subject, inventory.manifest);
validateInventoryPage(inventory);
return inventory;
@@ -391,8 +418,7 @@ const validateCompleteInventory = (inventory: InventorySet): void => {
const expectedId = artifactContradictionId({
rootArtifactId: inventory.manifest.root_artifact_id,
logicalPath: contradiction.logical_path,
declaredSha256: contradiction.declared_sha256,
observedSha256: contradiction.observed_sha256,
integrity: contradiction,
});
if (contradiction.contradiction_id !== expectedId)
throw new TypeError(
+18 -7
View File
@@ -1,3 +1,7 @@
import {
projectArtifactIntegrityDetails,
type ArtifactIntegrityDetails,
} from "./artifactIntegrity.js";
import {
AnalysisError,
type AnalysisErrorOptions,
@@ -30,12 +34,7 @@ export class ArtifactOperationError extends AnalysisError {
| "path"
| "unavailable"
| "io",
readonly artifactDetails?: Readonly<{
logicalPath: string;
declaredSha256: string | null;
calculatedSha256: string | null;
unpacked: boolean;
}>,
readonly artifactDetails?: ArtifactIntegrityDetails,
/** The specific constraint that failed, such as the colliding path. */
readonly detail?: string,
options?: AnalysisErrorOptions,
@@ -43,8 +42,20 @@ export class ArtifactOperationError extends AnalysisError {
super(
artifactDetails === undefined
? `Artifact ${operation} failed: ${reason}`
: `Artifact ${operation} failed: ${reason} at ${artifactDetails.logicalPath} (declared_sha256=${artifactDetails.declaredSha256 ?? "unavailable"}, calculated_sha256=${artifactDetails.calculatedSha256 ?? "unavailable"}, unpacked=${String(artifactDetails.unpacked)})`,
: artifactDetailsMessage(operation, reason, artifactDetails),
options,
);
}
}
const artifactDetailsMessage = (
operation: string,
reason: string,
details: ArtifactIntegrityDetails,
): string => {
const { logical_path, ...facts } = projectArtifactIntegrityDetails(details);
const description = Object.entries(facts)
.map(([key, value]) => `${key}=${String(value ?? "unavailable")}`)
.join(", ");
return `Artifact ${operation} failed: ${reason} at ${logical_path} (${description})`;
};
+6 -1
View File
@@ -232,7 +232,12 @@ export type BinaryTarget =
| (BinaryTargetIdentity &
NonExecutableMetadata & {
readonly kind: "archive";
readonly format: ZipPackageFormat | "asar" | "dmg" | "pkg";
readonly format:
| ZipPackageFormat
| "asar"
| "godot-pck"
| "dmg"
| "pkg";
})
| (BinaryTargetIdentity &
NonExecutableMetadata & {
+22 -1
View File
@@ -6,7 +6,12 @@ import {
RAW_ONLY_ARCHITECTURES,
type BinaryTarget,
} from "./binaryTargetTypes.js";
import { createEvidence, evidenceSchema, parseEvidence } from "./evidence.js";
import {
createEvidence,
evidenceSchema,
parseEvidence,
immutableEvidence,
} from "./evidence.js";
import { MAX_JSON_DEPTH, type JsonValue } from "./jsonValue.js";
import { createEvidenceBundle } from "./evidenceBundle.js";
import { freezeJsonSnapshot } from "./immutableJson.js";
@@ -604,3 +609,19 @@ it("preserves prototype-named raw and normalized results with aligned identity",
});
expect(stripped.evidence_id).not.toBe(evidence.evidence_id);
});
it("borrows a previously sealed envelope only after authenticating all semantic fields", () => {
const original = freezeJsonSnapshot(
createEvidence(TARGET, PROVIDER, {
operation: "inspect",
parameters: {},
result: { fact: 1 },
}),
);
expect(immutableEvidence(original)).toBe(original);
const changed = freezeJsonSnapshot({
...original,
normalized_result: { fact: 2 },
});
expect(() => immutableEvidence(changed)).toThrow();
});
+8 -2
View File
@@ -24,6 +24,7 @@ import {
} from "./jsonValue.js";
import {
freezeJsonSnapshot,
canBorrowParsedJsonSnapshot,
isImmutableJsonSnapshot,
} from "./immutableJson.js";
import { digestSchema } from "./../domain/digests.js";
@@ -57,6 +58,7 @@ const subjectFormatSchema = z.enum([
"hsp",
"app-pack",
"asar",
"godot-pck",
"dmg",
"pkg",
"plist",
@@ -288,8 +290,12 @@ export const parseEvidence = (input: unknown): Evidence => {
};
/** Authenticate and seal a ledger-owned snapshot; external mutable values are copied first. */
export const immutableEvidence = (input: unknown): Evidence =>
rememberImmutableEvidence(parseEvidence(input));
export const immutableEvidence = (input: unknown): Evidence => {
const parsed = parseEvidence(input);
return rememberImmutableEvidence(
canBorrowParsedJsonSnapshot(input, parsed) ? input : parsed,
);
};
/** Rebind display path metadata when an authenticated record names this target. */
export const rebindEvidenceToTargetPath = (
+10
View File
@@ -0,0 +1,10 @@
/** "GDPC" read as a little-endian uint32: the Godot Engine pack magic. */
export const GODOT_PCK_MAGIC = 0x43504447;
/** Whether bytes start with a standalone Godot PCK header. */
export const hasGodotPckHeader = (bytes: Uint8Array): boolean =>
bytes.length >= 4 &&
bytes[0] === 0x47 &&
bytes[1] === 0x44 &&
bytes[2] === 0x50 &&
bytes[3] === 0x43;
+17 -6
View File
@@ -1,3 +1,7 @@
import {
canBorrowParsedJsonSnapshot,
immutableJsonSnapshotMember,
} from "../domain/immutableJson.js";
import type { EvidenceMcpServer } from "./EvidenceMcpServer.js";
import {
@@ -42,11 +46,18 @@ export const registerArtifactTools = (
: decodeWithoutDelivery(operation, parameters);
},
parametersForEvidence: decodeWithoutDelivery,
sourceEvidence: (operation, result) =>
operation === "inspect_artifact"
? artifactInspectionResultSchema
.parse(result)
.substeps.map(({ evidence }) => evidence)
: [],
sourceEvidence: (operation, result) => {
if (operation !== "inspect_artifact") return [];
const parsed = artifactInspectionResultSchema.parse(result);
const inspection = canBorrowParsedJsonSnapshot(result, parsed)
? result
: parsed;
const substeps = immutableJsonSnapshotMember(inspection, "substeps");
return inspection.substeps.map((step, index) => {
const ownedStep = immutableJsonSnapshotMember(substeps, String(index));
immutableJsonSnapshotMember(ownedStep, "evidence");
return step.evidence;
});
},
});
};
+339
View File
@@ -0,0 +1,339 @@
import { createHash } from "node:crypto";
import { readFile, readdir, unlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect } from "vitest";
import { cliTest } from "../support/cli/cliFixture.js";
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
import {
buildAliasedGodotPck,
buildGodotPck,
pckU64,
} from "../fixtures/godotPck.js";
import { elf } from "../../src/domain/binaryTarget.fixture.js";
import { readEvidenceBundle } from "../../src/application/EvidenceBundleFiles.js";
import { analysisViewResultSchema } from "../../src/domain/analysisView/analysisView.js";
import { parseEvidence } from "../../src/domain/evidence.js";
import { artifactInspectionResultSchema } from "../../src/domain/artifactInspection.js";
import {
artifactExtractionResultSchema,
artifactInventoryResultSchema,
} from "../../src/domain/artifactGraph.js";
const members = [
{ path: "res://scripts/main.gd", data: "extends Node\n" },
{ path: "res://data/玩家.txt", data: "玩家数据\n" },
];
const graphFrom = (reply: unknown) => {
const evidence = parseEvidence(reply);
const inspection = artifactInspectionResultSchema.parse(
evidence.normalized_result,
);
return artifactInventoryResultSchema.parse(
inspection.substeps[0]?.evidence.normalized_result,
);
};
cliTest.for([2, 3, 4] as const)(
"inspects and safely extracts format %s PCK through the compiled CLI",
async (version, { cli }) => {
const root = await createTestTempDirectory("rea-pck-cli-");
const source = join(root, "game.pck");
await writeFile(source, buildGodotPck(version, members));
const inspected = await cli.run({
arguments: ["inspect-artifact", source, "--json"],
});
expect(inspected.exitCode, inspected.stdout + inspected.stderr).toBe(0);
const graph = graphFrom(inspected.json);
expect(graph.coverage).toEqual({
scope: "container-entries",
status: "complete",
});
const rootRow = graph.occurrences.find((row) => row.logical_path === ".");
expect(rootRow?.format_metadata).toMatchObject({
kind: "godot-pck",
format_version: version,
declared_file_count: 2,
});
for (const [index, member] of members.entries()) {
const row = graph.occurrences.find(
(row) => row.logical_path === member.path.slice(6),
);
expect(row?.declared_integrity).toEqual({
algorithm: "md5",
digest: createHash("md5").update(member.data).digest("hex"),
});
expect(row?.format_metadata).toMatchObject({
index,
source_path: member.path,
flags: 0,
});
expect(row?.hash_status).toBe("verified");
}
const extracted = await cli.run({
arguments: ["extract-artifact", source, "--json"],
environment: { TMPDIR: root, TMP: root, TEMP: root },
});
expect(extracted.exitCode, extracted.stdout + extracted.stderr).toBe(0);
const result = artifactExtractionResultSchema.parse(
parseEvidence(extracted.json).normalized_result,
);
expect(result.containment_verified).toBe(true);
for (const member of members)
expect(
await readFile(join(result.output_root, member.path.slice(6)), "utf8"),
).toBe(member.data);
},
);
cliTest(
"reads an embedded pack while preserving the native carrier's content classification",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-pck-embedded-cli-");
const source = join(root, "game.x86_64");
const pack = buildGodotPck(4, members);
const carrier = elf(2, 1, 62);
const bytes = Buffer.concat([
carrier,
pack,
pckU64(pack.length),
Buffer.from("GDPC"),
]);
await writeFile(source, bytes);
const reply = await cli.run({
arguments: ["inspect-artifact", source, "--json"],
});
expect(reply.exitCode, reply.stdout + reply.stderr).toBe(0);
const graph = graphFrom(reply.json);
expect(graph.manifest.root_sha256).toBe(
createHash("sha256").update(bytes).digest("hex"),
);
expect(
graph.nodes.find(
(row) => row.artifact_id === graph.manifest.root_artifact_id,
)?.format,
).toBe("elf");
expect(
graph.occurrences.find((row) => row.logical_path === ".")
?.format_metadata,
).toMatchObject({
embedded: true,
pack_start: carrier.length,
pack_end: carrier.length + pack.length,
});
},
);
cliTest(
"applies strict and record-and-continue MD5 integrity to inspection and extraction",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-pck-integrity-cli-");
const source = join(root, "contradiction.pck");
await writeFile(
source,
buildGodotPck(4, [
{ path: "res://main.gd", data: "real", md5: Buffer.alloc(16, 1) },
]),
);
const rejected = await cli.run({
arguments: ["inspect-artifact", source, "--json"],
});
expect(rejected.exitCode).toBe(1);
expect(rejected.json).toMatchObject({
code: "artifact_integrity_mismatch",
details: {
algorithm: "md5",
logical_path: "main.gd",
declared_md5: "01".repeat(16),
calculated_md5: createHash("md5").update("real").digest("hex"),
},
});
const inspected = await cli.run({
arguments: [
"inspect-artifact",
source,
"--integrity-policy",
"record-and-continue",
"--json",
],
});
expect(inspected.exitCode, inspected.stdout + inspected.stderr).toBe(0);
const graph = graphFrom(inspected.json);
expect(graph.integrity_contradictions).toEqual([
expect.objectContaining({
algorithm: "md5",
logical_path: "main.gd",
declared_md5: "01".repeat(16),
observed_md5: createHash("md5").update("real").digest("hex"),
trust: "observed-untrusted",
}),
]);
expect(
graph.occurrences.find((row) => row.logical_path === "main.gd")
?.hash_status,
).toBe("mismatched");
const extracted = await cli.run({
arguments: [
"extract-artifact",
source,
"--integrity-policy",
"record-and-continue",
"--json",
],
environment: { TMPDIR: root, TMP: root, TEMP: root },
});
expect(extracted.exitCode, extracted.stdout + extracted.stderr).toBe(0);
const result = artifactExtractionResultSchema.parse(
parseEvidence(extracted.json).normalized_result,
);
expect(result.integrity_contradictions).toHaveLength(1);
expect(await readFile(join(result.output_root, "main.gd"), "utf8")).toBe(
"real",
);
},
);
cliTest(
"retains encrypted-directory declarations without claiming an empty package",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-pck-encrypted-cli-");
const source = join(root, "encrypted.pck");
const bytes = buildGodotPck(4, members, { packFlags: 3 });
const directory = Number(bytes.readBigUInt64LE(32));
bytes.fill(0xff, directory + 4);
await writeFile(source, bytes);
const inspected = await cli.run({
arguments: ["inspect-artifact", source, "--json"],
});
expect(inspected.exitCode, inspected.stdout + inspected.stderr).toBe(0);
const graph = graphFrom(inspected.json);
expect(graph.coverage).toEqual({
scope: "container-entries",
status: "unknown",
});
expect(graph.occurrences).toHaveLength(1);
expect(graph.occurrences[0]).toMatchObject({
encrypted: true,
format_metadata: {
directory_status: "encrypted",
declared_file_count: 2,
},
});
const extracted = await cli.run({
arguments: ["extract-artifact", source, "--json"],
environment: { TMPDIR: root, TMP: root, TEMP: root },
});
expect(extracted.exitCode).toBe(1);
},
);
cliTest(
"exports compact CLI delivery and queries its retained complete Evidence after unlink",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-pck-export-");
const source = join(root, "game.pck");
const bundle = join(root, "evidence.json");
const refused = await cli.run({
arguments: ["inspect-artifact", source, "--detail", "summary", "--json"],
});
expect(refused.exitCode).toBe(1);
expect(refused.json).toMatchObject({
code: "invalid_request",
details: { issues: [{ path: ["evidence_output"] }] },
});
await writeFile(source, buildGodotPck(4, members));
const args = [
"inspect-artifact",
source,
"--detail",
"summary",
"--evidence_output",
bundle,
"--json",
];
const reply = await cli.run({ arguments: args });
expect(reply.exitCode, reply.stdout + reply.stderr).toBe(0);
expect(reply.json).toMatchObject({
evidence_bundle: { path: bundle, bytes: expect.any(Number) },
summary: {
normalized_result: {
kind: "summary",
summary: { root_format_metadata: { format_version: 4 } },
},
},
});
const saved = await readEvidenceBundle(bundle);
if (!saved.ok) throw saved.error;
const evidence = saved.value.records[0];
if (evidence === undefined) throw new Error("Missing complete inspection");
expect(graphFrom(evidence).occurrences).toHaveLength(3);
const before = await readFile(bundle);
expect((await cli.run({ arguments: args })).exitCode).toBe(1);
expect(await readFile(bundle)).toEqual(before);
await unlink(source);
const query = join(root, "query.json");
await writeFile(
query,
JSON.stringify({
source: { kind: "inline", evidence },
view: {
kind: "inventory",
collection: "occurrences",
logical_paths: ["scripts/main.gd"],
fields: ["logical_path"],
},
}),
);
const selected = await cli.run({
arguments: ["inspect-analysis-view", query, "--json"],
});
expect(selected.exitCode, selected.stdout + selected.stderr).toBe(0);
const view = analysisViewResultSchema.parse(
parseEvidence(selected.json).normalized_result,
);
if (view.kind !== "inventory") throw new Error("Expected inventory view");
expect(view.parent_evidence_id).toBe(evidence.evidence_id);
expect(view.items).toEqual([
expect.objectContaining({
logical_path: "scripts/main.gd",
occurrence_id: expect.any(String),
artifact_id: expect.any(String),
parent_occurrence_id: expect.any(String),
}),
]);
},
30000,
);
cliTest(
"rejects repeated-range expansion before hashing members or creating extraction output",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-pck-expansion-");
const source = join(root, "aliased.pck");
await writeFile(source, buildAliasedGodotPck(4, 1100));
for (const operation of ["inspect-artifact", "extract-artifact"]) {
for (const policy of ["fail", "record-and-continue"]) {
const reply = await cli.run({
arguments: [
operation,
source,
"--integrity-policy",
policy,
"--json",
],
environment: { TMPDIR: root, TMP: root, TEMP: root },
});
expect(reply.exitCode, reply.stdout + reply.stderr).toBe(1);
expect(reply.json).toMatchObject({
code: "truncated",
details: {
reason: "limit",
truncated: true,
detail: expect.stringContaining("expansion budget"),
},
});
expect(await readdir(root)).toEqual(["aliased.pck"]);
}
}
},
30000,
);
@@ -279,8 +279,8 @@ describe("ASAR captured-byte identity", () => {
const replacementMember = await right.next();
if (originalMember.done || replacementMember.done)
throw new Error("Expected second members");
expect(originalMember.value.declaredSha256).not.toBe(
replacementMember.value.declaredSha256,
expect(originalMember.value.declaredIntegrity?.digest).not.toBe(
replacementMember.value.declaredIntegrity?.digest,
);
expect(
(await buffer(await first.open(originalMember.value))).toString(),
@@ -0,0 +1,63 @@
import { writeFile, stat } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { buildGodotPck, pckU64 } from "../../fixtures/godotPck.js";
import {
scanArtifactInventory,
materializeArtifactInventory,
} from "../../fixtures/artifactInventory.js";
import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver.js";
import { elf } from "../../../src/domain/binaryTarget.fixture.js";
it("binds extraction to the original carrier even if a changed directory names identical payloads", async () => {
const root = await createTestTempDirectory("rea-pck-extraction-identity-");
const source = join(root, "game.pck");
const bytes = buildGodotPck(4, [{ path: "res://a.txt", data: "unchanged" }]);
await writeFile(source, bytes);
const snapshot = await scanArtifactInventory(source);
// A version field changes the carrier identity without changing any member's
// path, length or bytes. Per-member SHA-256 checks alone cannot detect this.
bytes.writeUInt32LE(7, 12);
await writeFile(source, bytes);
const output = join(root, "out");
await expect(
materializeArtifactInventory(
{
inputPath: source,
inputFormat: "godot-pck",
outputRoot: output,
environment: {},
integrityPolicy: "fail",
},
source,
snapshot,
),
).rejects.toMatchObject({ reason: "integrity" });
await expect(stat(output)).rejects.toMatchObject({ code: "ENOENT" });
});
it("preserves native defaults and explicit native selection for embedded carriers", async () => {
const root = await createTestTempDirectory("rea-pck-native-selection-");
const source = join(root, "game.x86_64");
const pack = buildGodotPck(4, [{ path: "res://a.txt", data: "a" }]);
await writeFile(
source,
Buffer.concat([
elf(2, 1, 62),
pack,
pckU64(pack.length),
Buffer.from("GDPC"),
]),
);
expect(await parseBinaryTarget(source)).toMatchObject({
ok: true,
value: { kind: "executable", format: "elf" },
});
expect(
await parseBinaryTarget(source, { targetKind: "executable" }),
).toMatchObject({
ok: true,
value: { kind: "executable", format: "elf", architecture: "x86_64" },
});
});
@@ -433,7 +433,7 @@ class TraversalReader implements ArtifactReader {
executable: false,
encrypted: false,
byteOffset: null,
declaredSha256: null,
declaredIntegrity: null,
unpacked: false,
limitations: [],
adapterKey: "/tmp/escape.js",
+234
View File
@@ -0,0 +1,234 @@
import { writeFile, unlink } from "node:fs/promises";
import { join } from "node:path";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { Ajv2020 } from "ajv/dist/2020.js";
import { expect, it } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { buildGodotPck } from "../../fixtures/godotPck.js";
import { ArtifactProvider } from "../../../src/artifacts/ArtifactProvider.js";
import { createServer } from "../../../src/server/createServer.js";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { artifactInspectionResultSchema } from "../../../src/domain/artifactInspection.js";
import {
artifactInventorySummaryViewResultSchema,
analysisViewResultSchema,
} from "../../../src/domain/analysisView/analysisView.js";
import {
canBorrowParsedJsonSnapshot,
immutableJsonSnapshotMember,
} from "../../../src/domain/immutableJson.js";
const viewValidators = async (client: Client) => {
const catalog = await client.listTools();
const contract = catalog.tools.find(
(tool) => tool.name === "inspect_analysis_view",
);
if (contract?.outputSchema === undefined)
throw new Error("Missing named view output contract");
const ajv = new Ajv2020({ strict: false, validateFormats: false });
const inputSchema: Record<string, unknown> = contract.inputSchema;
const outputSchema: Record<string, unknown> = contract.outputSchema;
expect(ajv.validateSchema(inputSchema)).toBe(true);
expect(ajv.validateSchema(outputSchema)).toBe(true);
const validateInput = ajv.compile(inputSchema);
const validateOutput = ajv.compile(outputSchema);
return { validateInput, validateOutput };
};
it("validates advertised schemas, batches connected facts and reuses one retained inventory after unlink", async () => {
const root = await createTestTempDirectory("rea-pck-mcp-");
const path = join(root, "game.pck");
await writeFile(
path,
buildGodotPck(4, [
{ path: "res://a.txt", data: "a" },
{ path: "res://b.txt", data: "b" },
]),
);
const session = createTestBinarySession(new ArtifactProvider({}));
const server = createServer({ kind: "session", session });
const client = new Client({ name: "pck-contract-test", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
const { validateInput, validateOutput } = await viewValidators(client);
expect(
(await client.callTool({ name: "open_binary", arguments: { path } }))
.isError,
).not.toBe(true);
const inspected = await client.callTool({
name: "inspect_artifact",
arguments: { detail: "summary" },
});
expect(inspected.isError).not.toBe(true);
const summaryEvidence = parseEvidence(inspected.structuredContent);
const summary = artifactInventorySummaryViewResultSchema.parse(
summaryEvidence.normalized_result,
);
expect(summary.summary.root_format_metadata).toMatchObject({
kind: "godot-pck",
format_version: 4,
declared_file_count: 2,
});
const parent = session.evidenceForAnalysis(summary.parent_evidence_id);
if (parent === undefined)
throw new Error("Missing retained complete inspection");
immutableJsonSnapshotMember(parent, "normalized_result");
const parsed = artifactInspectionResultSchema.parse(
parent.normalized_result,
);
if (!canBorrowParsedJsonSnapshot(parent.normalized_result, parsed))
throw new Error("Expected an owned parse-equivalent inspection");
const inspection = parent.normalized_result;
const nested = inspection.substeps[0]!.evidence;
expect(
session.evidenceForAnalysis(nested.evidence_id)?.normalized_result,
).toBe(nested.normalized_result);
const graph = nested.normalized_result;
const source = {
kind: "retained-evidence",
evidence_id: parent.evidence_id,
};
const view = async (query: Record<string, unknown>) => {
const arguments_ = { source, view: { kind: "inventory", ...query } };
expect(
validateInput(arguments_),
JSON.stringify(validateInput.errors),
).toBe(true);
const reply = await client.callTool({
name: "inspect_analysis_view",
arguments: arguments_,
});
expect(reply.isError, JSON.stringify(reply.structuredContent)).not.toBe(
true,
);
expect(
validateOutput(reply.structuredContent),
JSON.stringify(validateOutput.errors),
).toBe(true);
const evidence = parseEvidence(reply.structuredContent);
expect(evidence.evidence_links).toContain(parent.evidence_id);
return analysisViewResultSchema.parse(evidence.normalized_result);
};
expect(
validateInput({
source,
view: {
kind: "inventory",
collection: "nodes",
fields: ["logical_path"],
},
}),
).toBe(false);
const wanted = graph.occurrences.filter((row) => row.logical_path !== ".");
const occurrences = await view({
collection: "occurrences",
logical_paths: wanted.map((row) => row.logical_path),
fields: ["logical_path", "declared_integrity", "format_metadata"],
});
expect(occurrences.kind).toBe("inventory");
if (occurrences.kind !== "inventory")
throw new Error("Expected inventory view");
expect(occurrences.items).toEqual(
wanted.map((row) => ({
occurrence_id: row.occurrence_id,
artifact_id: row.artifact_id,
parent_occurrence_id: row.parent_occurrence_id,
logical_path: row.logical_path,
declared_integrity: row.declared_integrity,
format_metadata: row.format_metadata,
})),
);
const ids = wanted.flatMap((row) =>
row.artifact_id === null ? [] : [row.artifact_id],
);
const nodes = await view({
collection: "nodes",
artifact_ids: ids,
fields: ["sha256", "size"],
});
if (nodes.kind !== "inventory") throw new Error("Expected inventory nodes");
expect(nodes.items).toEqual(
graph.nodes
.filter((row) => ids.includes(row.artifact_id))
.map((row) => ({
artifact_id: row.artifact_id,
sha256: row.sha256,
size: row.size,
})),
);
await unlink(path);
const historical = await view({
collection: "occurrences",
occurrence_ids: wanted.map((row) => row.occurrence_id),
fields: [],
});
if (historical.kind !== "inventory")
throw new Error("Expected historical inventory");
expect(historical.source_coverage).toEqual({
scope: "container-entries",
status: "complete",
});
expect(historical.items).toEqual(
wanted.map((row) => ({
occurrence_id: row.occurrence_id,
artifact_id: row.artifact_id,
parent_occurrence_id: row.parent_occurrence_id,
})),
);
} finally {
await client.close();
await server.close();
await session.close();
}
});
it("carries unknown encrypted-directory coverage through compact MCP delivery", async () => {
const root = await createTestTempDirectory("rea-pck-encrypted-mcp-");
const path = join(root, "game.pck");
await writeFile(
path,
buildGodotPck(4, [{ path: "res://secret.gd", data: "x" }], {
packFlags: 3,
}),
);
const session = createTestBinarySession(new ArtifactProvider({}));
const server = createServer({ kind: "session", session });
const client = new Client({ name: "pck-unknown-test", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
await client.callTool({ name: "open_binary", arguments: { path } });
const reply = await client.callTool({
name: "inspect_artifact",
arguments: { detail: "summary" },
});
expect(reply.isError, JSON.stringify(reply.structuredContent)).not.toBe(
true,
);
const summary = artifactInventorySummaryViewResultSchema.parse(
parseEvidence(reply.structuredContent).normalized_result,
);
expect(summary.summary.source_coverage).toEqual({
scope: "container-entries",
status: "unknown",
});
expect(summary.summary.root_format_metadata).toMatchObject({
directory_status: "encrypted",
declared_file_count: 1,
});
expect(summary.summary.available_counts.occurrences).toBe(1);
expect(summary.unknowns.join(" ")).toContain(
"missing matches do not prove source absence",
);
} finally {
await client.close();
await server.close();
await session.close();
}
});
+105
View File
@@ -0,0 +1,105 @@
import { createHash } from "node:crypto";
export interface GodotPckMember {
readonly path: string;
readonly data: string;
readonly flags?: number;
readonly md5?: Buffer;
readonly size?: bigint;
readonly offset?: bigint;
}
export const pckU32 = (value: number): Buffer => {
const bytes = Buffer.alloc(4);
bytes.writeUInt32LE(value);
return bytes;
};
export const pckU64 = (value: bigint | number): Buffer => {
const bytes = Buffer.alloc(8);
bytes.writeBigUInt64LE(BigInt(value));
return bytes;
};
const pad4 = (bytes: Buffer): Buffer =>
Buffer.concat([bytes, Buffer.alloc((4 - (bytes.length % 4)) % 4)]);
/**
* Build a pack with the layout Godot's exporter writes (and GDRE Tools
* reproduces): format 2 places the directory after a 96-byte header and
* members after it; format 3 stores members first and a pack-relative
* directory offset.
*/
export const buildGodotPck = (
version: 2 | 3 | 4,
members: readonly GodotPckMember[],
options: { packFlags?: number; fileCount?: number } = {},
): Buffer => {
const data: Buffer[] = [];
const offsets: number[] = [];
let dataBytes = 0;
for (const member of members) {
offsets.push(dataBytes);
const bytes = pad4(Buffer.from(member.data));
data.push(bytes);
dataBytes += bytes.length;
}
const directory = Buffer.concat([
pckU32(options.fileCount ?? members.length),
...members.flatMap((member, index) => {
const path = pad4(Buffer.from(member.path));
return [
pckU32(path.length),
path,
pckU64(member.offset ?? BigInt(offsets[index] ?? 0)),
pckU64(member.size ?? BigInt(Buffer.byteLength(member.data))),
member.md5 ?? createHash("md5").update(member.data).digest(),
pckU32(member.flags ?? 0),
];
}),
]);
const headerBytes = 96;
const fileBase = version === 2 ? headerBytes + directory.length : headerBytes;
const header = Buffer.concat([
Buffer.from("GDPC"),
pckU32(version),
pckU32(4),
pckU32(6),
pckU32(1),
pckU32(options.packFlags ?? (version >= 3 ? 2 : 0)),
pckU64(fileBase),
pckU64(version >= 3 ? headerBytes + dataBytes : 0),
]);
const paddedHeader = Buffer.concat([
header,
Buffer.alloc(headerBytes - header.length),
]);
return version === 2
? Buffer.concat([paddedHeader, directory, ...data])
: Buffer.concat([paddedHeader, ...data, directory]);
};
export const embedGodotPck = (pack: Buffer): Buffer =>
Buffer.concat([
Buffer.from("\x7fELF fake engine bytes"),
pack,
pckU64(pack.length),
Buffer.from("GDPC"),
]);
/** Repeated physical ranges are valid aliases but multiply plaintext I/O per path. */
export const buildAliasedGodotPck = (
version: 2 | 3 | 4,
count: number,
entryBytes = 256 * 1024,
flags = 0,
): Buffer =>
buildGodotPck(
version,
Array.from({ length: count }, (_, index) => ({
path: `res://aliases/${String(index)}.bin`,
data: index === 0 ? "x".repeat(entryBytes) : "",
offset: 0n,
size: BigInt(entryBytes),
md5: Buffer.alloc(16),
flags,
})),
);