merge: consolidate upstream identity and representation owners
This commit is contained in:
commit
53601d0bdf
114 files changed
+8444
-813
No files matched your search
@@ -104,3 +104,14 @@ each plan states why in its `reason`. It replays the capture instead with
|
||||
`--replay-only` or when the state is unavailable (not derivable, or a catalog
|
||||
over the discovery budget), and its stopped document's `state` says which
|
||||
applied.
|
||||
|
||||
For a PE/ELF export question, `inspect_native_exports` accepts `detail: "summary"`
|
||||
and retains the complete table. Query `inspect_analysis_view` with
|
||||
`view: {kind: "native-exports", collection: "exports", names: ["selected_name"],
|
||||
fields: ["name"], offset: 0, limit: 32}` and the reported parent Evidence ID.
|
||||
All same-name versions and aliases survive; use `source_index` with the parent ID
|
||||
for stable identity. `collection: "issues"` pages interpretation failures. Page
|
||||
coverage describes selected retained rows; `summary.coverage` or `exports.coverage`
|
||||
still describes the source table, so an empty partial page does not prove absence.
|
||||
CLI `inspect-native-exports --detail summary --evidence_output PATH` saves the
|
||||
complete bundle for later `inspect-analysis-view --evidence-bundle PATH` requests.
|
||||
@@ -777,11 +777,27 @@ two collections can reuse a live id, and an equal pair stays in that collection.
|
||||
environment answers, a path parameter by the collection the live path names (a nested collection keeps its parent values, each decoded once, and a literal route segment stays), except a query value that pages a list, such as `limit`
|
||||
or `page`. Each call is checked against the served tool's input schema first, as the MCP server checks it. After mapping, two clock values compare by format only (ISO-8601 strings, and epoch numbers under
|
||||
`at`, `time`, `date`, `timestamp`, `created`, `updated` or `modified` field names, including snake-case and camel-case forms).
|
||||
Numeric array elements have no time-named field and remain literal measurements.
|
||||
Different string values compare as independently generated only at entity fields
|
||||
whose executed served write plan declares a unique token or UUID/token identity
|
||||
generator. UUID-like or token-like spelling alone does not establish generation;
|
||||
other business strings compare literally. The report counts each kind of normalised value.
|
||||
Numeric arrays retain that containing field's clock context through nested
|
||||
arrays; an object's own fields establish their own context. Numeric values at
|
||||
the body root, in root arrays, or under measurement fields remain literal.
|
||||
|
||||
Generated strings normalize only where this confirmed, served write's entity
|
||||
response rules identify generation. A `unique` rule or UUID identity rule must
|
||||
still compare two UUIDs; a `renewed` rule applies only at its recorded pointers,
|
||||
with `*` matching actual array items. Token/sequence strings retain the existing
|
||||
format check: equal length of at least eight, the same character classes, and
|
||||
a digit. Numeric sequences continue to compare literally unless an ID mapping
|
||||
applies. String shape and names such as `token` or `sku` alone do not establish
|
||||
generation. Read and replay responses without these facts stay literal except
|
||||
for supported clocks and learned IDs. A missing write target replays an absence
|
||||
response and does not inherit the successful entity response's generation rules.
|
||||
Admitted generated values must also preserve aliases and distinct identities
|
||||
within that response. UUID identity uses the generator's case-insensitive
|
||||
matching; token and sequence strings retain their literal identity. This
|
||||
correspondence ends with the answer and is not inferred across calls.
|
||||
The report counts each kind of normalised
|
||||
value; the generation rules remain model-derived facts, rather than new
|
||||
observations of the application.
|
||||
ISO clock offsets are accepted only within RFC 3339's `00`–`23` hour and `00`–`59` minute ranges.
|
||||
Probe ordering retains every recorded fractional-second digit in ISO timestamps
|
||||
with a four-digit year, `T`, and a `Z` or `±HH:MM` suffix. Equal instants use ordinal order. Other native-Date
|
||||
|
||||
+14
-5
@@ -221,9 +221,9 @@ Not yet read: v4 `.apk.idsig` (a sibling file), verity (`0x0421`/`0x0423`/`0x042
|
||||
## Bounds and errors
|
||||
|
||||
The archive is read once, bounded at 512 MiB, without following symlinks and
|
||||
with change detection. Every size and offset in the ZIP tail and signing block
|
||||
is checked against its containing section before use, including the complete
|
||||
central-directory extent before ZIP end records. A process-wide queue permits
|
||||
with change detection. ZIP tail offsets, the central-directory size and signing
|
||||
block spans used by the reader are checked against their containing sections,
|
||||
including the complete central-directory extent before ZIP end records. A process-wide queue permits
|
||||
one active whole-file inspection and eight pending scans across provider
|
||||
instances; pending scans retain no APK bytes and can be cancelled with their
|
||||
original reason. Further requests receive a capacity error. The active slot
|
||||
@@ -250,8 +250,17 @@ A file that is not a ZIP is refused as an unsupported target. A damaged ZIP
|
||||
(local headers but no end-of-central-directory), a central directory that
|
||||
cannot be walked or whose declared record count does not account for all of
|
||||
its bytes, or a malformed signing block or signer is an input error naming the
|
||||
scheme and reason. Signer framing includes signature records, the public-key
|
||||
field and matching signed/outer v3 SDK ranges. Suffix bytes beyond known
|
||||
offset or scheme and reason. Zip64 tail damage includes a missing required
|
||||
locator, an impossible locator disk/count pair, a fixed end record that does
|
||||
not fit before its locator, or an incorrect end-record signature. These are
|
||||
input errors, not evidence of an unsupported format. Coherent multi-disk
|
||||
locator metadata and 64-bit offsets/counts/sizes beyond the reader's safe
|
||||
integer range remain unsupported. The locator offset is disk-relative, so
|
||||
multi-disk metadata is checked before applying local file bounds. This follows
|
||||
the record fields in [PKWARE APPNOTE 6.3.10, sections 4.3.14–4.3.15](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT);
|
||||
it does not establish full validation of Zip64 versions or extensible data.
|
||||
Signer framing includes signature records, the public-key field and matching
|
||||
signed/outer v3 SDK ranges. Suffix bytes beyond known
|
||||
fields are counted as uninterpreted, as Android readers may ignore extensions;
|
||||
their presence does not establish malformed signing.
|
||||
When the central directory is not immediately followed by
|
||||
|
||||
@@ -38,7 +38,11 @@ graph TD
|
||||
IDA["src/ida/<br/>Upstream MCP contract adaptation<br/>live GUI binding or owned headless database"]
|
||||
KUNA["src/kuna/<br/>BYO Kuna deep analysis candidate<br/>discovery reads metadata; version probe on target open"]
|
||||
IDA_MCP["External ida-pro-mcp<br/>GUI proxy or database supervisor<br/>upstream installation and engine lifecycle"]
|
||||
|
||||
KUNA["src/kuna/<br/>BYO Kuna decompiler, a deep analysis candidate<br/>discovery reads metadata only; version probe when a target opens"]
|
||||
AUX["src/artifacts/ + src/native/<br/>artifact inventory, bounded directory triage and extraction<br/>format readers and native capabilities"]
|
||||
|
||||
|
||||
BROWSER["src/browser/<br/>CDP / Playwright capture<br/>captured scripts and native module resolution<br/>execution/: coverage, listener and source attribution"]
|
||||
JS_CODEC["src/javascript/sourceMaps/<br/>pinned upstream map codec<br/>owned heap-limited Node process, complete point evidence"]
|
||||
INSPECTOR["src/inspector/<br/>passive V8 Inspector discovery and capture<br/>script and target locations"]
|
||||
@@ -50,12 +54,19 @@ graph TD
|
||||
FIRMWARE["src/firmware/<br/>Binwalk / Unblob adapters<br/>owned processes, report normalization, private extraction"]
|
||||
ANDROID["src/android/<br/>Static APK adapter<br/>owned SDK stdio, pinned headless JADX, normalization"]
|
||||
REFERENCE["src/reference/<br/>Filesystem reference-source reader<br/>bounded traversal, validation, errors"]
|
||||
|
||||
ANDROID_TOOLS["src/adb/ + src/apktool/ + src/jeb/<br/>BYO device, resource-decoding and JEB adapters<br/>owned adb and Apktool processes; JEB engine client over MCP"]
|
||||
CAPTURE_TOOLS["src/captures/ + src/pcap/ + src/signals/ + src/sqlite/<br/>BYO tshark, pcap, sigrok and SQLite child inspection<br/>selected captures, recordings and databases; owned processes"]
|
||||
NATIVE_READERS["src/go/ + src/rust/ + src/il2cpp/ + src/flutter/ + src/gpu/<br/>Bundled in-process binary metadata readers<br/>Go build info, Rust dependencies, IL2CPP, Dart build ID, GPU code objects<br/>src/compression/: shared codec-result helper for rust and gpu"]
|
||||
FORMAT_READERS["src/model/ + src/python/ + src/erlang/ + src/electron/<br/>Bundled in-process format readers<br/>GGUF and safetensors headers, Python .pyc, Erlang BEAM, Electron fuses"]
|
||||
RUNTIME["src/frida/ + src/cutter/<br/>Frida instrumentation sessions and live Cutter bridge client"]
|
||||
end
|
||||
|
||||
subgraph "Provider Process Foundation"
|
||||
PROCESS["src/process/<br/>owned process groups, private roots<br/>deadlines, bounded file reads and diagnostics, cleanup"]
|
||||
WINDOWS["src/windows/<br/>Windows host boundary adapters<br/>package loader, opaque handles, owned jobs and private runtimes"]
|
||||
WINDOWS_NATIVE["native/windows/src/<br/>Node-API implementation<br/>NTFS admission, protected DACLs and atomic Job Object assignment"]
|
||||
FILESYSTEM["src/filesystem/<br/>owned file and directory handles<br/>stable reads, staged output, caller paths"]
|
||||
end
|
||||
|
||||
subgraph "Contracts Layer"
|
||||
@@ -181,3 +192,23 @@ graph TD
|
||||
APP -->|EVM interface port| EVM
|
||||
EVM --> PROCESS
|
||||
EVM --> DOMAIN
|
||||
COMPOSITION --> KUNA
|
||||
KUNA --> PROCESS
|
||||
KUNA --> FILESYSTEM
|
||||
KUNA --> DOMAIN
|
||||
COMPOSITION --> ANDROID_TOOLS
|
||||
ANDROID_TOOLS --> PROCESS
|
||||
ANDROID_TOOLS --> DOMAIN
|
||||
COMPOSITION --> CAPTURE_TOOLS
|
||||
CAPTURE_TOOLS --> PROCESS
|
||||
CAPTURE_TOOLS --> DOMAIN
|
||||
COMPOSITION --> NATIVE_READERS
|
||||
NATIVE_READERS --> DOMAIN
|
||||
COMPOSITION --> FORMAT_READERS
|
||||
FORMAT_READERS --> DOMAIN
|
||||
SERVER --> RUNTIME
|
||||
CLI --> RUNTIME
|
||||
RUNTIME --> DOMAIN
|
||||
PROCESS --> FILESYSTEM
|
||||
AUX --> FILESYSTEM
|
||||
APP --> FILESYSTEM
|
||||
@@ -340,7 +340,48 @@ rea inspect-native-exports ./runtimes/linux-x64/native/libexample.so --json
|
||||
```
|
||||
|
||||
The complete Evidence identifies the stable artifact snapshot by path, byte
|
||||
count and SHA-256.
|
||||
count and SHA-256. Complete delivery remains the default. For a small initial
|
||||
MCP reply, select `detail: "summary"`; this retains the complete Evidence and
|
||||
returns artifact identity, table coverage and row counts. Use its
|
||||
`normalized_result.parent_evidence_id` with `inspect_analysis_view`:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_<parent digest>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "native-exports",
|
||||
"collection": "exports",
|
||||
"names": ["selected_name"],
|
||||
"fields": ["name"],
|
||||
"offset": 0,
|
||||
"limit": 32
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Names match exactly and preserve every same-name version or alias. Optional
|
||||
`indices` selects original row positions, intersecting a name filter when both
|
||||
are supplied; empty filters select nothing. `source_index` always remains and,
|
||||
with the parent ID, identifies the original export or issue row. Fields default
|
||||
to the complete row; an empty field list returns only its identity. ELF `name`
|
||||
selection also returns raw-byte and encoding qualifications. Fields from the
|
||||
other format are rejected. `collection: "issues"` pages the complete diagnostic
|
||||
records, with optional original indices. Offset and limit default to 0 and 64
|
||||
and apply to matching rows in original order.
|
||||
|
||||
View `coverage` describes the selected page; `summary.coverage` or
|
||||
`exports.coverage` preserves source-table coverage. An exhausted or empty page
|
||||
does not make a partial source complete. Retained views never reopen the file.
|
||||
The CLI uses the same projector and requires a durable bundle for a summary:
|
||||
|
||||
```sh
|
||||
rea inspect-native-exports ./libexample.so --detail summary --evidence_output ./exports.bundle.json --json
|
||||
# Pass the summary's parent ID and the same view as JSON to inspect-analysis-view,
|
||||
# together with --evidence-bundle ./exports.bundle.json.
|
||||
```
|
||||
|
||||
- PE rows reproduce the export directory: each used export-address-table slot
|
||||
with its biased `ordinal` (`OrdinalBase` plus the slot index), one row per
|
||||
|
||||
@@ -71,7 +71,7 @@ debugging client, not an REA-owned observing session. The selected page-session
|
||||
collector does not collect worker-session requests, console events, or scripts,
|
||||
and workers that disappeared before the snapshot may be missing; worker
|
||||
coverage is therefore attach-limited. Missing ownership metadata is reported as
|
||||
`unknown_target_relationship` with unavailable worker attribution, separately
|
||||
`unattributed_target` with unavailable worker attribution, separately
|
||||
from a proven foreign owner excluded as `out_of_target_scope`. A shared origin
|
||||
alone establishes neither relationship.
|
||||
|
||||
@@ -462,6 +462,11 @@ Frames, resources, scripts, events, and workers outside the selected origin scop
|
||||
Retained workers include validated opener-target and parent-frame IDs when CDP
|
||||
provides them; these relationships do not expand the exact-origin boundary or
|
||||
prove static module ownership.
|
||||
Workers without either relationship are withheld with an `unattributed_target`
|
||||
exclusion and `workers` in `unavailable_sections`: a same-origin URL does not
|
||||
establish which page owns a worker. A supplied relationship to another target or
|
||||
frame remains `out_of_target_scope`. Missing ownership also remains unknown when
|
||||
the worker URL is empty; withheld target details are not returned.
|
||||
|
||||
## Completeness and limits
|
||||
|
||||
|
||||
@@ -119,6 +119,11 @@ target, frame, script, and resource, a worker URL must resolve to a local
|
||||
file before it is retained. Relationship IDs improve
|
||||
attribution but do not prove which static module started a worker or that its
|
||||
work completed.
|
||||
Missing opener and parent relationships leave worker ownership unknown. Such
|
||||
targets are withheld with an `unattributed_target` exclusion and unavailable
|
||||
`workers` coverage, even when their URL names a local file. Explicit foreign
|
||||
relationships remain `out_of_target_scope`; malformed supplied relationship IDs
|
||||
are `invalid_protocol_value`.
|
||||
|
||||
Inspection does not retain DOM values, execute renderer code, navigate, click,
|
||||
invoke Electron IPC, close a target, or terminate the application.
|
||||
|
||||
+22
-8
@@ -214,9 +214,13 @@ assembly image. Each module record starts with its module name, its method
|
||||
pointer count and its method pointer array. The reader finds, for every metadata
|
||||
image, the single record whose name pointer reaches that image's name string and
|
||||
whose count equals the image's method count. It then requires exactly one
|
||||
`codeGenModulesCount`/`codeGenModules` pair that lists exactly those records. No
|
||||
symbols are needed, so stripped builds work. A method's token row id selects its
|
||||
slot in the module's method pointer array. Within one image, each method must
|
||||
`codeGenModulesCount`/`codeGenModules` pair that lists exactly those records.
|
||||
`codeGenModulesCount` and module method counts own four bytes, followed by
|
||||
alignment padding before their eight-byte pointers. Unresolved loader writes
|
||||
overlapping those four bytes prevent the registration from being established;
|
||||
padding outside the field is neither required nor interpreted as part of its
|
||||
count. No symbols are needed, so stripped builds work. A method's token row id
|
||||
selects its slot in the module's method pointer array. Within one image, each method must
|
||||
claim a distinct MethodDef token row; duplicate rows are refused before any
|
||||
linked method table or seed output is published. Row ids may repeat across
|
||||
different images, and distinct rows may still resolve to the same address when
|
||||
@@ -275,17 +279,27 @@ The result adds:
|
||||
metadata name (`TKey`). Nested generic arguments stay with the component
|
||||
that declares them, for example `Game.Outer<int>+Inner<string>` or
|
||||
`Game.Outer<int>+Leaf` when the nested type adds no parameters. A trailing
|
||||
`&` marks a by-reference type. `name` is `null` only for type index `-1`.
|
||||
`&` marks a by-reference type. `name` is `null` for type index `-1`, metadata
|
||||
version 27 as described below, or an unresolved type-table slot. A dyld bind
|
||||
or unresolved ELF relocation retains the type index without interpreting its
|
||||
stored word as a resolved pointer. If an unresolved loader value occurs
|
||||
inside a type graph, linking is refused with `unsupported-pointer-encoding`;
|
||||
a concrete name is not synthesized from that placeholder.
|
||||
|
||||
`binary` also records `types_field` and `types_count`: the address of
|
||||
`Il2CppMetadataRegistration.typesCount` and its value. That record is found as
|
||||
the single place whose `fieldOffsetsCount` and `typeDefinitionsSizesCount` both
|
||||
equal the metadata type-definition count and whose type table is file mapped.
|
||||
Counts use their declared 32-bit width independently of alignment padding.
|
||||
The complete registration must have mapped coverage; trailing declared zero-fill
|
||||
fields do not need physical file bytes. All dereferenced pointers use the loader
|
||||
fixup interpretation: unresolved dyld bindings cannot supply concrete type names
|
||||
from their stored placeholders and native linking refuses an unavailable type record.
|
||||
The complete 128-byte registration must have unambiguous loader coverage. Its
|
||||
used fields through byte 112 must be file-backed; a trailing declared zero-fill
|
||||
range can cover the final unused fields. An unmapped or conflicting tail is
|
||||
refused. These layouts follow the versioned [Unity 2022.3.8f1 producer header](https://github.com/MlgmXyysd/libil2cpp/blob/master/libil2cpp/Unity_2022.3/2022.3.8f1/il2cpp-class-internals.h#L596-L637)
|
||||
(redistributed source) and the [AArch64 aggregate alignment rules](https://github.com/ARM-software/abi-aa/blob/2025Q1/aapcs64/aapcs64.rst#aggregates).
|
||||
Dyld pointer ownership comes from the parsed bind stream, as in Apple's
|
||||
[dyld analyzer](https://github.com/apple-oss-distributions/dyld/blob/dyld-1165.3/common/MachOAnalyzer.cpp#L1508-L1511), rather than the slot's placeholder word.
|
||||
The padding, zero-fill-tail and unresolved-type cases have authored regression
|
||||
fixtures; they do not add real Unity-build verification to the checks above.
|
||||
|
||||
Without `binary_path`, `binary` is `null` and the four binary collections are
|
||||
empty in the complete producer report to indicate unrequested observations. Their summary counts, including state counts, are `null`, and native page requests report unavailable observations. This does not establish an empty native inventory. Missing native fields in historical Evidence remain unknown too.
|
||||
|
||||
+12
-1
@@ -246,7 +246,12 @@ A missing entry proves creation or deletion only when the corresponding
|
||||
observation root's path enumeration was exhausted. Otherwise its effect has
|
||||
`status: "unknown"`, the available before/after state, and a reason; a null
|
||||
state in this variant means unobserved. Paths below symlinks that REA did not
|
||||
follow also remain unknown. Partial enumeration of one root does not erase
|
||||
follow also remain unknown. Each observed symlink records a
|
||||
`symlink_not_followed` enumeration omission with a null system code, including
|
||||
a stable selected root. The link's own metadata can remain unchanged while
|
||||
descendant coverage and filesystem comparison remain unknown. Physical
|
||||
parent-directory enumeration still proves unrelated sibling absence.
|
||||
Partial enumeration of one root does not erase
|
||||
known effects in another, and unavailable content hashes do not by themselves
|
||||
make path absence unknown.
|
||||
|
||||
@@ -275,6 +280,12 @@ denied or otherwise unavailable lookup records an `entry_unavailable` omission
|
||||
for that path instead, and its root stays incomplete. Cancellation and
|
||||
unexpected I/O failures still stop capture.
|
||||
|
||||
An initial selected-root permission denial returns `access_denied` with the
|
||||
selected path, native `EACCES` or `EPERM`, and filesystem-read recovery. Capture
|
||||
does not launch the target. Earlier snapshot facts and the denied root's
|
||||
omission remain in the partial observation; unresolved read cleanup retains its
|
||||
own classification until its exact resources are released.
|
||||
|
||||
Terminal frame `data` is the comparison text after the selected normalization.
|
||||
New captures also preserve `raw_data` when normalization changes a PTY chunk;
|
||||
otherwise `data` is already the original text. Older captures may lack the
|
||||
|
||||
@@ -64,3 +64,17 @@ fails closure even when another proof passes; unknown, unsupported or skipped
|
||||
applicable outcomes keep it unknown. The result cites all applicable outcomes.
|
||||
Proof array order does not establish freshness or supersession. A proof for
|
||||
unrelated artifact digests does not contradict the selected boundary.
|
||||
|
||||
## Function reconstruction claims
|
||||
|
||||
`verify_reconstruction` evaluates declared claims over retained comparison
|
||||
Evidence. An `overall` function claim includes active residual unknowns scoped
|
||||
to any of that comparison's dimensions. For example, an open pseudocode
|
||||
question keeps the overall claim `unknown` and retains its unknown ID and
|
||||
recommended probe; an independent assembly claim can still pass. An unknown
|
||||
scoped only to `overall` does not block every individual dimension.
|
||||
|
||||
Selection uses each unknown's latest revision, its comparison identity and
|
||||
Evidence references. Explicit unknown dependencies remain relevant across
|
||||
dimensions. Unknowns for another comparison and resolved heads do not become
|
||||
blockers merely because their source Evidence overlaps.
|
||||
@@ -29,6 +29,12 @@ note interpretation; an unspecified OSABI alone does not prove OS origin.
|
||||
from the decoder's `type` label. Source binding and semantic decisions use the
|
||||
numeric identity; an architecture-specific display name does not establish
|
||||
support for that segment's payload semantics.
|
||||
Header `type` names follow the pinned parser's machine-specific projection:
|
||||
`PT_AARCH64_ARCHEXT` and `PT_AARCH64_UNWIND` denote `0x70000000` and
|
||||
`0x70000001` only for AArch64. The same codes on x86-64 remain numeric, as
|
||||
does AArch64's `0x70000002` in pyelftools 0.33. These display projections
|
||||
are checked against the architecture and source-bound `type_id`; they do not
|
||||
decode architecture compatibility, unwind or MTE payloads.
|
||||
- Every note's original identity, owner/descriptor bytes and source ranges,
|
||||
including unfamiliar owners and note kinds.
|
||||
CORE symbolic note labels apply only to the exact `CORE\0` owner. Other
|
||||
@@ -113,8 +119,14 @@ GDB/Python/pwndbg/strace. Fixture generation explicitly launches its owned test
|
||||
program; subsequent inspection only reads the resulting core. Real acceptance
|
||||
checks recorded threads, high registers, missing/malformed notes, opaque owners,
|
||||
CLI/MCP parity, optional debugger context and historical PID collision with an
|
||||
owned live sentinel. An observed Node thread group must launch the exact ownership
|
||||
`ps` inventory before it can read that live process's `stat` or `status` for native
|
||||
owned live sentinel. Authored AArch64 auxiliary-header and numeric-unknown
|
||||
carriers also pass through the configured decoder and both public adapters;
|
||||
they are metadata fixtures, rather than captured AArch64 crashes. Their expected
|
||||
names follow [pyelftools 0.33's enums](https://github.com/eliben/pyelftools/blob/v0.33/elftools/elf/enums.py#L431-L435)
|
||||
and [machine-specific selection](https://github.com/eliben/pyelftools/blob/v0.33/elftools/elf/structs.py#L179-L184).
|
||||
The [AArch64 ELF ABI](https://github.com/ARM-software/abi-aa/blob/2025Q1/aaelf64/aaelf64.rst#program-header)
|
||||
defines their raw codes. An observed Node thread group must launch the exact
|
||||
ownership `ps` inventory before it can read that live process's `stat` or `status` for native
|
||||
identity/UID facts. Only traced `CLONE_THREAD` members share this allowance;
|
||||
exec-replaced threads, ordinary children, debugger and decoder processes cannot
|
||||
use it. The purpose of stat/status reads within the same Node thread group
|
||||
|
||||
+16
-12
@@ -40,20 +40,24 @@ unrelated producer warnings do not change request status.
|
||||
Omitting `input_format` requests sigrok's autodetection. REA uses an immutable
|
||||
private copy whose filename retains the selected recording's final suffix,
|
||||
including a dotfile basename such as `.csv`. Filename-based format detection
|
||||
receives that hint without exposing the
|
||||
caller's directory to sigrok. The operand is an explicit relative path, so a
|
||||
recording named `-` still selects the private file rather than sigrok stdin.
|
||||
Omitting `annotations`
|
||||
retains every annotation class. Both omitted selections are recorded as null.
|
||||
receives that hint without exposing the caller's directory to sigrok. The
|
||||
input operand is an explicit relative path within the private workspace,
|
||||
so a recording named `-` is opened as a file rather than
|
||||
[sigrok's stdin operand](https://github.com/sigrokproject/sigrok-cli/blob/sigrok-cli-0.7.2/input.c#L49-L75).
|
||||
Omitting `annotations` retains every annotation class. Both omitted selections
|
||||
are recorded as null.
|
||||
If sigrok reports an explicitly selected input format as unknown, REA returns
|
||||
`invalid_request` for `input_format`, retains the producer diagnostic, and
|
||||
advises correcting the format or omitting it for autodetection. An explicit
|
||||
`Unknown keyword` refusal naming a caller-selected format option also returns
|
||||
`invalid_request` for `input_format`, even when sigrok exits zero and produces
|
||||
fallback output. Output and diagnostics remain in `captured_output`; unrelated
|
||||
warnings remain diagnostics. A completed nonzero `--version` probe is a
|
||||
configuration refusal with its captured output and the selected sigrok/prlimit
|
||||
configuration guidance. See the
|
||||
advises correcting the format or omitting it for autodetection. The
|
||||
[tagged option parser](https://github.com/sigrokproject/sigrok-cli/blob/sigrok-cli-0.7.2/parsers.c#L387-L404)
|
||||
warns when an option key is unknown. When that diagnostic's key exactly matches
|
||||
a selected `input_format` option, REA returns `invalid_request`, even on exit zero.
|
||||
Fallback output and the diagnostic remain in `captured_output`; unrelated
|
||||
option warnings stay visible without rejecting the request. Matching keeps
|
||||
the complete diagnostic caption, including an LF inside a selected key;
|
||||
embedded warning substrings do not establish a refusal. A completed nonzero
|
||||
`--version` probe is a configuration refusal with its captured output and the
|
||||
selected sigrok/prlimit configuration guidance. See the
|
||||
[upstream manual](https://sigrok.org/wiki/Sigrok-cli) for formats, decoder
|
||||
stacking and annotation selectors. No arbitrary command flags are accepted.
|
||||
|
||||
|
||||
@@ -171,7 +171,24 @@ fully qualified message type. All imports must be included. REA never fetches
|
||||
schemas or executes the selected application. Descriptor identity and parser
|
||||
version remain in the body facts. Scalars, repeated numeric packing and nested
|
||||
messages are interpreted alongside the unchanged wire facts; disagreements
|
||||
remain explicit. Map semantics, extensions and required-field validation are
|
||||
remain explicit. Descriptor admission rejects duplicate field numbers within
|
||||
each message, including nested messages and oneof fields, before schema
|
||||
annotation. This does not reject repeated occurrences in captured wire bytes
|
||||
or reuse of a number in a different message. Fully qualified message, enum,
|
||||
service and extension names must also identify one declaration across the
|
||||
descriptor set, including nested and synthetic map-entry messages. Different
|
||||
packages or enclosing messages may reuse a local type name. Schema-admission
|
||||
failures retain the partial capture Evidence and its original artifact identity.
|
||||
REA also requires an unambiguous descriptor file identity. Repeated exact file
|
||||
names are coalesced only when protobuf-es equality confirms matching field
|
||||
presence, declared values and retained opaque unknown/custom-option fields.
|
||||
Opaque fields must match in their retained order and bytes; REA does not
|
||||
interpret them. Conflicting definitions are refused rather than selecting a
|
||||
file by its position. This is REA's supplied-schema admission policy, separate
|
||||
from protobuf wire validity and the [first-occurrence precedence in protoc v32.1](https://github.com/protocolbuffers/protobuf/blob/v32.1/src/google/protobuf/compiler/command_line_interface.cc#L1402).
|
||||
The descriptor-set byte count and SHA-256 continue to identify the complete
|
||||
original input, including repeated file facts.
|
||||
Map semantics, extensions and required-field validation are
|
||||
unsupported. A supplied schema does not establish the sender's actual schema.
|
||||
An invalid nested interpretation records `schema_status: "invalid-message"`
|
||||
and `schema_error` locally; later outer fields remain available. Valid string
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
"""Check equivalent export questions and count explicitly projected tokenizer costs."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import statistics
|
||||
|
||||
import tiktoken
|
||||
|
||||
|
||||
def compact(value):
|
||||
return json.dumps(value, ensure_ascii=False, separators=(",", ":"), allow_nan=False)
|
||||
|
||||
|
||||
def inspect(path, encoding):
|
||||
report = json.loads((path / "report.json").read_text())
|
||||
catalog = json.loads((path / "catalog.json").read_text())
|
||||
transcript = json.loads((path / "transcript.json").read_text())
|
||||
scope = next(json.loads(line) for line in reversed(path.with_suffix(".log").read_text().splitlines())
|
||||
if line.startswith('{"resource_scope":'))
|
||||
observed = scope.get("payload_resource_observation") or {}
|
||||
events = observed.get("resource_events", {})
|
||||
assert report["status"] == "passed" and report.get("runtime_identity_sha256")
|
||||
assert not report["cleanup_failures"] and not report["receipt_failures"]
|
||||
assert scope["abort"] is None and scope.get("payload_resource_observation_verified") is True
|
||||
assert observed.get("payload_exit_code") == 0 and observed.get("payload_cleanup_verified") is True
|
||||
assert all(events.get("memory.events", {}).get(key) == 0 for key in ["high", "max", "oom", "oom_kill"])
|
||||
assert events.get("pids.events", {}).get("max") == 0
|
||||
assert len(catalog["tools"]) == 220
|
||||
count = lambda text: len(encoding.encode(text, disallowed_special=()))
|
||||
directory = [{key: tool[key] for key in ["name", "description", "inputSchema"]} for tool in catalog["tools"]]
|
||||
question = (f"For {report['artifact']['path']}, report the export count and every version/alias of "
|
||||
f"{', '.join(dict.fromkeys(row['name'] for row in report['answer']['rows']))}, including addresses, "
|
||||
"sizes, types, binding and symbol-version facts. Check that __rea_export_oracle_missing__ "
|
||||
"has no recorded match, preserve source coverage and export the complete Evidence. "
|
||||
"Report static declarations without claiming runtime execution.")
|
||||
arguments = sum(count(compact({"name": event["name"], "arguments": event["input"]})) for event in transcript)
|
||||
replies = sum(count(block["text"]) for event in transcript for block in event["reply"]["content"] if block["type"] == "text")
|
||||
inputs = count(compact(directory))
|
||||
raw_catalog = count(compact(catalog))
|
||||
task_directory = [tool for tool in directory if tool["name"] in ["inspect_native_exports", "inspect_analysis_view", "export_evidence_bundle"]]
|
||||
return {"case": path.name, "lane": report["lane"], "artifact": report["artifact"],
|
||||
"answer": report["answer"], "parent_evidence_id": report["parent_evidence_id"],
|
||||
"source_report_sha256": report["source_report_sha256"],
|
||||
"runtime_identity_sha256": report["runtime_identity_sha256"],
|
||||
"catalog_sha256": report["catalog_sha256"], "catalog_tools": len(catalog["tools"]),
|
||||
"buffer_bytes": report["buffer_bytes"], "calls": report["calls"], "phases": report["phases"],
|
||||
"wall_seconds": scope["elapsed_seconds"], "cpu_seconds": observed["cpu_stat"]["usage_usec"] / 1e6,
|
||||
"kernel_family_peak_bytes": observed["memory_peak_bytes"],
|
||||
"events_at_payload_exit": events, "payload_cleanup_verified": True,
|
||||
"final_whole_scope_events": scope.get("resource_events_final"),
|
||||
"token_proxy": {"question": count(question), "input_catalog_once": inputs, "arguments": arguments,
|
||||
"text_replies": replies, "combined": count(question) + inputs + arguments + replies,
|
||||
"only_three_task_tools": count(question) + count(compact(task_directory)) + arguments + replies,
|
||||
"complete_transport_catalog": raw_catalog},
|
||||
"receipt_sha256": {file.name: hashlib.sha256(file.read_bytes()).hexdigest() for file in
|
||||
[path / "report.json", path / "catalog.json", path / "transcript.json", path / "evidence.json", path.with_suffix(".log")]}}
|
||||
|
||||
|
||||
def summarize(rows):
|
||||
result = {}
|
||||
for artifact in sorted({row["artifact"]["sha256"] for row in rows}):
|
||||
selected = [row for row in rows if row["artifact"]["sha256"] == artifact]
|
||||
assert len({row["parent_evidence_id"] for row in selected}) == 1
|
||||
assert len({row["source_report_sha256"] for row in selected}) == 1
|
||||
assert all(row["answer"] == selected[0]["answer"] for row in selected)
|
||||
result[artifact] = {}
|
||||
for lane in ["complete", "selected"]:
|
||||
samples = [row for row in selected if row["lane"] == lane]
|
||||
assert len(samples) == 3
|
||||
metrics = {key: [row[key] for row in samples] for key in ["wall_seconds", "cpu_seconds", "kernel_family_peak_bytes"]}
|
||||
metrics["projected_tokens"] = [row["token_proxy"]["combined"] for row in samples]
|
||||
metrics["mcp_call_count"] = [len(row["calls"]) for row in samples]
|
||||
metrics["mcp_reply_bytes"] = [sum(call["reply_bytes"] for call in row["calls"]) for row in samples]
|
||||
result[artifact][lane] = {metric: {"median": statistics.median(values), "minimum": min(values), "maximum": max(values)} for metric, values in metrics.items()}
|
||||
return result
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
parser.add_argument("--cohort", type=Path, required=True)
|
||||
parser.add_argument("cases", type=Path, nargs="+")
|
||||
args = parser.parse_args()
|
||||
manifest = json.loads((args.cohort / "manifest.json").read_text())
|
||||
index = json.loads((args.cohort / "index.json").read_text())
|
||||
assert len(index) == 12 and all(case["exit"] == 0 and case["status"] == "passed" for case in index)
|
||||
assert {path.resolve() for path in args.cases} == {(args.cohort / case["name"]).resolve() for case in index}
|
||||
assert len(args.cases) == len(index)
|
||||
for file, digest in manifest["driver_files"].items():
|
||||
assert hashlib.sha256(Path(file).read_bytes()).hexdigest() == digest, file
|
||||
runtimes = {}
|
||||
for case in index:
|
||||
path = Path(case["runtime"]) / "runtime-identity.json"
|
||||
data = path.read_bytes()
|
||||
identity = json.loads(data)
|
||||
assert identity["source_clean"] is True
|
||||
runtimes[hashlib.sha256(data).hexdigest()] = {
|
||||
"source_revision": identity["source_revision"],
|
||||
"compiled_files": len(identity["compiled_files"]),
|
||||
"entrypoints": len(identity["entrypoints"]),
|
||||
}
|
||||
encoding = tiktoken.get_encoding("cl100k_base")
|
||||
rows = [inspect(path, encoding) for path in args.cases]
|
||||
assert all(row["runtime_identity_sha256"] in runtimes for row in rows)
|
||||
receipt = {"tiktoken": tiktoken.__version__, "encoding": encoding.name, "actual_model_usage": None,
|
||||
"cohort": manifest, "commands": index, "runtime_identities": runtimes,
|
||||
"projection": "Scripted SDK tasks, not an actual model run. The estimate includes a fixed task description, every discovered tool name/description/input schema once, every tool argument and text reply, including Evidence export. Complete raw SDK directory token costs and a three-tool-only alternative are reported separately; output schemas are not included in the primary model-context projection. Host injection, history replay, inference quality and billable usage are unmeasured.",
|
||||
"resources": "CPU quota 100%; launcher binds clients/servers to CPUs 0-1 (the outer observer starts on CPU15). The guard receipt affinity is initial placement, not child affinity; a live cohort observation records the override. 3 GiB early stop/4 GiB hard/no swap. Fresh client/server/HOME per task, Node compile cache disabled, OS caches unflushed. Later retained pages are warm queries; no separate warm-server cohort. Whole-family costs include client/oracle comparisons, source hashing, runtime binding, serialization and receipts. Tokenizer processing itself is outside these measured tasks. Verified counters are at payload exit while the observer remains alive; unavailable final whole-scope counters remain null.",
|
||||
"summary": summarize(rows), "recordings": rows}
|
||||
args.output.write_text(json.dumps(receipt, ensure_ascii=False, indent=2) + "\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,48 @@
|
||||
"""Independent readelf symbol facts and directly mapped ELF64 version indices."""
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
path = Path(sys.argv[1]).resolve()
|
||||
selected_names = sys.argv[2:]
|
||||
source = path.read_bytes()
|
||||
assert source[:6] == b'\x7fELF\x02\x01', 'This experiment oracle only covers ELF64 little-endian'
|
||||
section_offset = struct.unpack_from('<Q', source, 40)[0]
|
||||
section_width, section_count = struct.unpack_from('<HH', source, 58)
|
||||
assert section_width == 64 and section_count > 0
|
||||
sections = [struct.unpack_from('<IIQQQQIIQQ', source, section_offset + i * section_width) for i in range(section_count)]
|
||||
dynsym_index = next(i for i, row in enumerate(sections) if row[1] == 11)
|
||||
version_section = next(row for row in sections if row[1] == 0x6fffffff and row[6] == dynsym_index)
|
||||
text = subprocess.check_output(['readelf', '--dyn-syms', '--wide', str(path)], text=True)
|
||||
pattern = re.compile(r'^\s*(\d+):\s+([0-9a-f]+)\s+(\d+)\s+(\w+)\s+(\w+)\s+(\w+)\s+(\S+)\s+(\S+)')
|
||||
rows = []
|
||||
for line in text.splitlines():
|
||||
match = pattern.match(line)
|
||||
if not match:
|
||||
continue
|
||||
ordinal, value, size, kind, binding, visibility, section, display = match.groups()
|
||||
ordinal = int(ordinal)
|
||||
if binding not in ['GLOBAL', 'WEAK', 'UNIQUE'] or section == 'UND' or visibility not in ['DEFAULT', 'PROTECTED']:
|
||||
continue
|
||||
version_word = struct.unpack_from('<H', source, version_section[4] + ordinal * 2)[0]
|
||||
if (version_word & 0x7fff) == 0:
|
||||
continue
|
||||
name, _, version_name = display.partition('@@' if '@@' in display else '@')
|
||||
version = None if (version_word & 0x7fff) < 2 else {
|
||||
'index': version_word & 0x7fff, 'name': version_name or None,
|
||||
'name_bytes_base64': None, 'name_utf8_decodable': True if version_name else None,
|
||||
'hidden': bool(version_word & 0x8000),
|
||||
}
|
||||
rows.append({'source_index': len(rows), 'name': name, 'name_bytes_base64': None, 'name_utf8_decodable': True,
|
||||
'value': hex(int(value, 16)), 'size': size, 'binding': binding.lower(),
|
||||
'type': {'FUNC': 'function', 'IFUNC': 'ifunc', 'OBJECT': 'object', 'TLS': 'tls'}.get(kind, 'other'),
|
||||
'version': version})
|
||||
selected = [row for row in rows if row['name'] in selected_names]
|
||||
assert set(row['name'] for row in selected) == set(selected_names)
|
||||
print(json.dumps({'artifact': {'path': str(path), 'sha256': hashlib.sha256(source).hexdigest(), 'bytes': len(source)},
|
||||
'oracle': subprocess.check_output(['readelf', '--version'], text=True).splitlines()[0],
|
||||
'names': selected_names, 'export_count': len(rows), 'rows': selected}, indent=2))
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,167 @@
|
||||
# Native export questions over retained Evidence
|
||||
|
||||
Issue #6900: retrieve the export count and every declared version/alias of two
|
||||
named exports, verify a missing-name query within the examined table, and export
|
||||
the complete Evidence. The previous workflow returns the complete export table
|
||||
to the client. The selected workflow returns a summary, queries matching rows,
|
||||
and keeps the same complete parent accessible through Evidence export.
|
||||
|
||||
## Pinned inputs and correctness
|
||||
|
||||
- Baseline: `f713087132021d235682f908a1c0f3bc6ab8762d` (main, including the ELF
|
||||
unknown-coverage fix #6827). Candidate and experiment driver:
|
||||
`c7a90b742a29328e5b0069c6f8a34381e0c11e57`.
|
||||
- Linux x86-64, Node 24.22.1, npm 11.16.0; package and kernel versions are in
|
||||
[the receipt](native-export-views.json). Each runtime's compiled files and
|
||||
entrypoints were SHA-256 checked before and after every task. Source checkouts
|
||||
were clean when frozen. Later reporting changes do not change that measured
|
||||
implementation revision.
|
||||
- `/usr/lib/x86_64-linux-gnu/libLLVM-14.so.1`: 109,967,296 bytes, 44,459 exports,
|
||||
SHA-256 `436887791de0478d72c8323be99df69d6d0cf82745e5abec79d5e0374f4df560`.
|
||||
Query `LLVMContextCreate` and `LLVMContextDispose`: two rows.
|
||||
- `/usr/lib/x86_64-linux-gnu/libc.so.6`: 1,926,232 bytes, 3,025 exports,
|
||||
SHA-256 `6b4a45352fd0c540a9c7c718f35ce8c8e46a4e482f9d3885a910c32d1a0e1421`.
|
||||
Query `malloc` and `memcpy`: three rows, including both `memcpy` versions and
|
||||
the IFUNC declaration. No runtime address or execution claim is made.
|
||||
|
||||
The independent oracle combines GNU `readelf` 2.40 dynamic symbol facts with
|
||||
direct ELF64 little-endian mapping of version indices and hidden flags in
|
||||
[native-export-oracle.py](native-export-oracle.py). It checks the full export
|
||||
count and each selected row, including its index in the original report. All six
|
||||
tasks for each artifact produced identical answers, complete parent Evidence IDs
|
||||
and source report digests. Target bytes were unchanged. This oracle is qualified
|
||||
for these ELF64 little-endian artifacts and selected ASCII names; it does not
|
||||
establish real PE performance or arbitrary ELF encoding coverage.
|
||||
|
||||
## Measured completed tasks
|
||||
|
||||
Three alternating pairs per artifact, serial execution, fresh stdio client,
|
||||
server and HOME per task; Node compile cache disabled. Retained page queries
|
||||
reuse their parent within the task. OS file caches were not flushed, and there
|
||||
was no separate warm-server cohort. All twelve tasks passed without resource
|
||||
stops, OOM events, process-limit events or owned payload processes remaining.
|
||||
|
||||
| Artifact / workflow | Calls | Reply bytes | Projected tokens | Elapsed seconds, median [range] | Family CPU seconds, median [range] | Kernel family peak MiB, median [range] |
|
||||
| ------------------- | ----: | ----------: | ---------------: | ------------------------------- | ---------------------------------- | -------------------------------------- |
|
||||
| libc / complete | 2 | 1,653,415 | 289,912 | 18.848 [18.657–30.361] | 17.260 [15.342–29.125] | 993.1 [962.6–1,048.7] |
|
||||
| libc / selected | 5 | 25,952 | 58,253 | 18.858 [17.044–32.088] | 17.476 [15.765–31.440] | 928.6 [922.2–939.7] |
|
||||
| LLVM / complete | 2 | 28,986,227 | 4,355,448 | 97.516 [94.112–119.443] | 88.030 [84.867–111.504] | 1,833.8 [1,806.6–1,861.8] |
|
||||
| LLVM / selected | 4 | 19,695 | 57,346 | 37.094 [29.132–39.676] | 32.645 [19.386–38.746] | 1,142.6 [1,123.0–1,233.6] |
|
||||
|
||||
These are scripted SDK tasks, with the full 220-tool directory discovered once.
|
||||
The `cl100k_base` estimate (tiktoken 0.12.0) counts a fixed task description,
|
||||
every tool name/description/input schema once, all call arguments and text
|
||||
replies, including Evidence export. It excludes host injection and history
|
||||
replay. No actual model ran; these are neither billable usage nor investigation
|
||||
quality measurements. The SDK also receives output schemas: the entire raw
|
||||
catalog costs 812,145 tokens before and 815,046 after if a host injects all of it.
|
||||
That cost is reported separately. With only the three task tools injected,
|
||||
estimated task totals are 238,857 → 7,198 for libc and 4,304,393 → 6,291 for LLVM.
|
||||
Tokenizer work ran separately from the twelve task measurements.
|
||||
|
||||
Both libc lanes used a 10 MiB transport budget. Both LLVM timing lanes used
|
||||
64 MiB to permit a complete-response comparison. Separate final probes at
|
||||
10 MiB observed the baseline complete call return `resource_constraint`, while
|
||||
the selected task completed. The failed complete task is preserved and excluded
|
||||
from successful-task timing statistics. These boundary probes establish behavior,
|
||||
not an additional performance pair.
|
||||
|
||||
The selected libc task has extra calls and shows no established latency gain;
|
||||
its medians are essentially equal within the observed ranges.
|
||||
LLVM has substantial observed task savings, but the result does not identify a
|
||||
faster native parser: both workflows still analyze and retain the full table.
|
||||
The `inspect` RPC includes analysis, Evidence, validation and delivery; no
|
||||
internal phase attribution was measured. Per-call timings are retained for each pair; first selected-page calls validate
|
||||
the retained parent, while later queries reuse that checked representation.
|
||||
The weak cache holds validation of an exact immutable retained report, not live
|
||||
filesystem state. Unfiltered pages slice the original order; filtered pages scan
|
||||
the retained rows and preserve every matching alternative.
|
||||
|
||||
Resources: one CPU's aggregate quota (100%), clients/servers bound to CPUs 0–1,
|
||||
observer initially on CPU 15; 3 GiB early stop, 4 GiB hard limit, no swap, 192
|
||||
tasks. The guard's affinity field records initial placement; it does not describe
|
||||
the wrapper's child affinity override. A live cgroup/process observation confirmed
|
||||
the quota and override. Family CPU and peak memory come from kernel cgroup
|
||||
counters at payload exit while an observer remains alive. Cleanup was verified
|
||||
at that point. Final whole-scope counters became unavailable after scope removal
|
||||
and remain null; no final whole-scope counter verification is claimed. Timing
|
||||
includes discovery, runtime hashing, client checks, Evidence export and receipts.
|
||||
Variation in the slower complete samples is retained in the ranges.
|
||||
|
||||
## Reproduction and validation
|
||||
|
||||
Run from the candidate checkout with its locked dependencies and pinned Node.
|
||||
Freeze a clean built runtime with `scripts/rea.mjs`, `package.json`, `dist/` and
|
||||
dependencies; record source revision plus SHA-256 for compiled files and
|
||||
entrypoints as demonstrated by the committed receipt's runtime identities.
|
||||
|
||||
```sh
|
||||
python3 experiments/analysis-efficiency/native-export-oracle.py \
|
||||
/usr/lib/x86_64-linux-gnu/libLLVM-14.so.1 LLVMContextCreate LLVMContextDispose \
|
||||
> /absolute/path/oracle.json
|
||||
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
|
||||
node experiments/analysis-efficiency/native-export-views.mjs \
|
||||
--runtime /absolute/path/runtime --identity /absolute/path/runtime/runtime-identity.json \
|
||||
--oracle /absolute/path/oracle.json --output /absolute/path/unused-receipt-directory \
|
||||
--lane selected --buffer 67108864
|
||||
```
|
||||
|
||||
Use `--lane complete` against the baseline. Repeat three pairs in alternating
|
||||
order. The JSON receipt records every executed command, runtime binding,
|
||||
per-call timings/bytes, results and SHA-256 of raw reports/catalogs/transcripts/
|
||||
Evidence/resource logs. Its `cohort` manifest records driver and guard hashes.
|
||||
Generate token estimates with `native-export-costs.py --cohort COHORT --output
|
||||
OUTPUT CASE...`, using tiktoken 0.12.0. Raw local receipts are under
|
||||
`/tmp/rea-perf-native-exports-20261023/`; they are not committed binaries or
|
||||
portable artifact downloads. `selector-cohort/index.json` and `manifest.json` bind
|
||||
the twelve cases. Preserve these files together when rerunning the cost script.
|
||||
|
||||
Executed functional validation of the measured implementation:
|
||||
|
||||
- 48 selected tests passed: parser, export views, MCP schemas/retention preflight,
|
||||
and actual public CLI/stdio workflows for PE and ELF fixtures. Source deletion
|
||||
after retention still permits equivalent offline CLI and retained MCP views.
|
||||
Sparse field checks also recompute the returned view digest in the native
|
||||
regression and actual MCP acceptance workflow.
|
||||
- `npm run check`, documentation generation/check and test-discovery verification
|
||||
passed; 2,180 tests were discovered exactly once.
|
||||
- Real LLVM public CLI summary → full Evidence bundle → offline selected view
|
||||
was also checked; the complete bundle retains 44,459 exports.
|
||||
- Earlier assertion spelling mistakes, an architecture-boundary test placement
|
||||
violation and a concurrent-worker timeout are retained in the local logs.
|
||||
The test was moved to the native layer. A later field-omission regression
|
||||
reproduced source defaults leaking into projected columns; the projection
|
||||
schema now removes those defaults, while complete source normalization retains
|
||||
them. Final bounded tests and static gates passed. No real Hopper/Ghidra/IDA behavior changed or was qualified here.
|
||||
|
||||
Partial/unsupported source coverage, duplicate names/versions, PE ordinal-only
|
||||
exports, forwarded exports, field projection, malformed retained input and older
|
||||
encoding metadata are covered by deterministic boundary regressions. Those tests
|
||||
supplement the real ELF tasks; they are not real PE provider measurements.
|
||||
|
||||
## Initial CI attribution
|
||||
|
||||
[CI run 40015745440](https://github.com/morluto/rea/actions/runs/40015745440)
|
||||
tested head `5e9a17bea7a6bc5931df60256eef61010a9c67e1`, merge
|
||||
`e6f10a331aed5228504ed45ca6244b9498548cb9`, against main
|
||||
`f713087132021d235682f908a1c0f3bc6ab8762d`. Static checks, build, docs and selected
|
||||
real-format qualification jobs passed. Source tests and built tests failed.
|
||||
Every cancelled shard's log was inspected: 11 source and 46 built failures were
|
||||
actually observed, including the ADB floating-point regex rejected in `v` mode.
|
||||
|
||||
Bounded serial controls compared that exact main base with implementation
|
||||
`c7a90b742a29328e5b0069c6f8a34381e0c11e57`. The source selection reproduced all
|
||||
11 failures on both. A broader built selection executed 52 cases on both, with
|
||||
48 failures and four passes; it includes every observed CI built failure. The
|
||||
two extra failures and four passes in this selection are not presented as CI
|
||||
observations. All selected case identities, statuses and full failure messages
|
||||
were identical after replacing only the two checkout roots, including the
|
||||
physical path of their shared dependency symlink. No assertion text was removed.
|
||||
All 57 actual CI failures were therefore reproduced on the base and candidate.
|
||||
|
||||
Local `ci-attribution.json`, `ci-observed-case-identities.json`,
|
||||
`ci-control-index.json`, the four control reports and their resource logs under
|
||||
the receipt directory retain commands, pinned revisions, complete differences
|
||||
and SHA-256 bindings. Controls had no resource stop or OOM and verified payload
|
||||
cleanup. This attribution applies to that initial CI run and those revisions;
|
||||
later CI results must be inspected separately.
|
||||
@@ -0,0 +1,224 @@
|
||||
/** Equivalent export-table questions over public production stdio MCP workflows. */
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { createWriteStream } from "node:fs";
|
||||
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
import { finished } from "node:stream/promises";
|
||||
import { parseArgs } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { createRetainedQueryCaller } from "./retained-query-calls.mjs";
|
||||
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
|
||||
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
runtime: { type: "string" },
|
||||
oracle: { type: "string" },
|
||||
output: { type: "string" },
|
||||
lane: { type: "string" },
|
||||
buffer: { type: "string", default: "67108864" },
|
||||
identity: { type: "string" },
|
||||
},
|
||||
});
|
||||
for (const key of ["runtime", "oracle", "output"])
|
||||
assert.ok(values[key] && values[key] === resolve(values[key]), key);
|
||||
assert.ok(["complete", "selected"].includes(values.lane));
|
||||
const buffer = Number(values.buffer);
|
||||
assert.ok(Number.isSafeInteger(buffer) && buffer >= 10485760);
|
||||
const expected = JSON.parse(await readFile(values.oracle, "utf8"));
|
||||
const hash = (value) => createHash("sha256").update(value).digest("hex");
|
||||
const artifact = expected.artifact.path;
|
||||
assert.equal(hash(await readFile(artifact)), expected.artifact.sha256);
|
||||
await mkdir(values.output, { mode: 0o700 });
|
||||
const account = await mkdtemp(join(tmpdir(), "rea-export-views-"));
|
||||
const client = new Client({ name: "native-export-views", version: "1" });
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
|
||||
maxBufferSize: buffer,
|
||||
env: {
|
||||
HOME: account,
|
||||
TMPDIR: account,
|
||||
PATH: process.env.PATH,
|
||||
REA_LOG_LEVEL: "silent",
|
||||
REA_MCP_MAX_RESPONSE_BYTES: String(buffer),
|
||||
NODE_OPTIONS: "--max-old-space-size=2048",
|
||||
NODE_DISABLE_COMPILE_CACHE: "1",
|
||||
UV_THREADPOOL_SIZE: "1",
|
||||
},
|
||||
stderr: "pipe",
|
||||
});
|
||||
const serverLog = createWriteStream(join(values.output, "server.log"));
|
||||
const serverLogFinished = finished(serverLog);
|
||||
serverLogFinished.catch(() => {});
|
||||
transport.stderr?.pipe(serverLog);
|
||||
const report = {
|
||||
lane: values.lane,
|
||||
buffer_bytes: buffer,
|
||||
artifact: expected.artifact,
|
||||
calls: [],
|
||||
phases: {},
|
||||
status: "running",
|
||||
};
|
||||
const verifyRuntime = async (phase) => {
|
||||
if (values.identity === undefined) return;
|
||||
const started = performance.now();
|
||||
const identityBytes = await readFile(values.identity);
|
||||
const identity = JSON.parse(identityBytes);
|
||||
for (const entry of [...identity.compiled_files, ...identity.entrypoints])
|
||||
assert.equal(
|
||||
hash(await readFile(join(values.runtime, entry.path))),
|
||||
entry.sha256,
|
||||
entry.path,
|
||||
);
|
||||
report.runtime_identity_sha256 ??= hash(identityBytes);
|
||||
assert.equal(hash(identityBytes), report.runtime_identity_sha256);
|
||||
report.phases[`runtime_validation_${phase}_ms`] = performance.now() - started;
|
||||
};
|
||||
const transcript = [];
|
||||
const call = createRetainedQueryCaller(client, (receipt, event) => {
|
||||
report.calls.push(receipt);
|
||||
transcript.push(event);
|
||||
});
|
||||
const observe = async (name, input, phase) => {
|
||||
const reply = await call(name, input, phase);
|
||||
if (reply.isError === true) assert.fail(JSON.stringify(reply));
|
||||
assert.ok(reply.structuredContent);
|
||||
return reply.structuredContent;
|
||||
};
|
||||
const choose = (source) =>
|
||||
source.exports.flatMap((row, source_index) =>
|
||||
expected.names.includes(row.name) ? [{ ...row, source_index }] : [],
|
||||
);
|
||||
try {
|
||||
await verifyRuntime("before");
|
||||
let start = performance.now();
|
||||
await client.connect(transport);
|
||||
const catalog = await client.listTools();
|
||||
report.phases.discovery_ms = performance.now() - start;
|
||||
report.catalog_bytes = Buffer.byteLength(JSON.stringify(catalog));
|
||||
report.catalog_sha256 = hash(JSON.stringify(catalog));
|
||||
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
|
||||
const selected = values.lane === "selected";
|
||||
const reply = await observe(
|
||||
"inspect_native_exports",
|
||||
{ path: artifact, ...(selected ? { detail: "summary" } : {}) },
|
||||
"inspect",
|
||||
);
|
||||
const parentId = selected
|
||||
? reply.normalized_result.parent_evidence_id
|
||||
: reply.evidence_id;
|
||||
report.parent_evidence_id = parentId;
|
||||
let rows;
|
||||
if (selected) {
|
||||
rows = [];
|
||||
let offset = 0;
|
||||
do {
|
||||
const page = (
|
||||
await observe(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source: { kind: "retained-evidence", evidence_id: parentId },
|
||||
view: {
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
names: expected.names,
|
||||
offset,
|
||||
limit: 2,
|
||||
},
|
||||
},
|
||||
"page",
|
||||
)
|
||||
).normalized_result;
|
||||
assert.equal(page.exports.coverage.status, "complete");
|
||||
assert.equal(page.coverage.total, expected.rows.length);
|
||||
rows.push(...page.items);
|
||||
offset = page.coverage.next_offset;
|
||||
} while (offset !== null);
|
||||
const empty = (
|
||||
await observe(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source: { kind: "retained-evidence", evidence_id: parentId },
|
||||
view: {
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
names: ["__rea_export_oracle_missing__"],
|
||||
limit: 1,
|
||||
},
|
||||
},
|
||||
"negative",
|
||||
)
|
||||
).normalized_result;
|
||||
assert.deepEqual(empty.items, []);
|
||||
assert.equal(empty.exports.coverage.status, "complete");
|
||||
assert.equal(empty.coverage.total, 0);
|
||||
} else {
|
||||
rows = choose(reply.normalized_result);
|
||||
assert.equal(
|
||||
reply.normalized_result.exports.some(
|
||||
(row) => row.name === "__rea_export_oracle_missing__",
|
||||
),
|
||||
false,
|
||||
);
|
||||
}
|
||||
assert.deepEqual(rows, expected.rows);
|
||||
await observe(
|
||||
"export_evidence_bundle",
|
||||
{ path: join(values.output, "evidence.json") },
|
||||
"export",
|
||||
);
|
||||
start = performance.now();
|
||||
const bundle = JSON.parse(
|
||||
await readFile(join(values.output, "evidence.json"), "utf8"),
|
||||
);
|
||||
const parent = bundle.records.find(
|
||||
(record) => record.evidence_id === parentId,
|
||||
);
|
||||
assert.ok(parent);
|
||||
assert.deepEqual(choose(parent.normalized_result), expected.rows);
|
||||
assert.equal(
|
||||
parent.normalized_result.coverage.exports,
|
||||
expected.export_count,
|
||||
);
|
||||
assert.equal(parent.normalized_result.exports.length, expected.export_count);
|
||||
assert.equal(parent.normalized_result.coverage.status, "complete");
|
||||
assert.equal(parent.subject.digest.sha256, expected.artifact.sha256);
|
||||
report.source_report_sha256 = hash(JSON.stringify(parent.normalized_result));
|
||||
report.phases.bundle_verification_ms = performance.now() - start;
|
||||
assert.equal(hash(await readFile(artifact)), expected.artifact.sha256);
|
||||
report.answer = {
|
||||
rows,
|
||||
export_count: expected.export_count,
|
||||
negative: [],
|
||||
coverage: parent.normalized_result.coverage,
|
||||
};
|
||||
report.status = "passed";
|
||||
} catch (cause) {
|
||||
report.status = "failed";
|
||||
report.failure = String(cause?.stack ?? cause);
|
||||
process.exitCode = 1;
|
||||
} finally {
|
||||
try {
|
||||
await verifyRuntime("after");
|
||||
} catch (cause) {
|
||||
report.status = "failed";
|
||||
report.runtime_identity_failure = String(cause?.stack ?? cause);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
if (
|
||||
!(await finalizeRetainedQueryReceipt({
|
||||
report,
|
||||
client,
|
||||
transport,
|
||||
account,
|
||||
serverLog,
|
||||
serverLogFinished,
|
||||
output: values.output,
|
||||
transcript,
|
||||
}))
|
||||
)
|
||||
process.exitCode = 1;
|
||||
}
|
||||
@@ -5,6 +5,7 @@ import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
from elftools.elf.elffile import ELFFile
|
||||
from elftools.elf.structs import ELFStructs
|
||||
from elftools.construct import Container
|
||||
from pwnlib.elf.datatypes import elf_prstatus_amd64
|
||||
|
||||
@@ -31,6 +32,26 @@ def note_header(content, note, **changes):
|
||||
content[note.n_offset:note.n_offset + len(encoded)] = encoded
|
||||
|
||||
|
||||
# Authored metadata carriers exercise actual parser projection through public
|
||||
# inspection, independently of the host that produced the captured amd64 core.
|
||||
for name, machine, kind in [
|
||||
("aarch64-archext", "EM_AARCH64", "PT_AARCH64_ARCHEXT"),
|
||||
("aarch64-unwind", "EM_AARCH64", "PT_AARCH64_UNWIND"),
|
||||
("aarch64-mte-opaque", "EM_AARCH64", 0x70000002),
|
||||
("amd64-processor-opaque", "EM_X86_64", 0x70000001),
|
||||
]:
|
||||
structs = ELFStructs(little_endian=True, elfclass=64)
|
||||
structs.create_basic_structs()
|
||||
structs.create_advanced_structs("ET_CORE", machine, image.header.e_ident.EI_OSABI)
|
||||
header_bytes, entry_bytes = structs.Elf_Ehdr.sizeof(), structs.Elf_Phdr.sizeof()
|
||||
header = Container(**dict(image.header, e_machine=machine, e_entry=0, e_ehsize=header_bytes,
|
||||
e_phoff=header_bytes, e_phentsize=entry_bytes, e_phnum=1,
|
||||
e_shoff=0, e_shentsize=64, e_shnum=0, e_shstrndx=0, e_flags=0))
|
||||
segment = Container(p_type=kind, p_flags=4, p_offset=header_bytes + entry_bytes,
|
||||
p_vaddr=0, p_paddr=0, p_filesz=4, p_memsz=4, p_align=4)
|
||||
save(name, structs.Elf_Ehdr.build(header) + structs.Elf_Phdr.build(segment) + bytes(4))
|
||||
|
||||
|
||||
save("truncated-header", original[:32])
|
||||
for name, osabi, version in [("linux-osabi", 3, 0), ("solaris-osabi", 6, 0), ("freebsd-osabi", 9, 0), ("unsupported-abi-version", 0, 1)]:
|
||||
content = bytearray(original)
|
||||
|
||||
@@ -216,6 +216,22 @@ try {
|
||||
assert.equal(noNotes.threads.length, 0);
|
||||
assert.equal(noNotes.note_interpretation_completeness, "unknown");
|
||||
cases++;
|
||||
for (const [name, architecture, type] of [
|
||||
["aarch64-archext", "aarch64-little-endian", "PT_AARCH64_ARCHEXT"],
|
||||
["aarch64-unwind", "aarch64-little-endian", "PT_AARCH64_UNWIND"],
|
||||
["aarch64-mte-opaque", "aarch64-little-endian", 0x70000002],
|
||||
["amd64-processor-opaque", "x86_64-little-endian", 0x70000001],
|
||||
]) {
|
||||
const auxiliary = await basic.inspect(mode, fixtures[name]);
|
||||
assert.equal(auxiliary.architecture, architecture);
|
||||
assert.equal(auxiliary.segments.length, 1);
|
||||
assert.equal(auxiliary.segments[0].type, type);
|
||||
assert.equal(auxiliary.notes.length, 0);
|
||||
assert.equal(auxiliary.threads.length, 0);
|
||||
assert.equal(auxiliary.note_interpretation_completeness, "unknown");
|
||||
assert.equal(auxiliary.live_process_identity, "unknown");
|
||||
cases++;
|
||||
}
|
||||
const negative = await basic.inspect(mode, fixtures["negative-pid"]);
|
||||
assert.ok(negative.threads.some((thread) => thread.historical_pid === -2));
|
||||
cases++;
|
||||
|
||||
@@ -227,8 +227,23 @@ try {
|
||||
const candidate = join(candidateRoot, "module.wasm");
|
||||
await writeFile(join(candidateRoot, "source.wat"), "(module (memory 2))");
|
||||
await tool("wat2wasm", ["source.wat", "-o", "module.wasm"], candidateRoot);
|
||||
const glueSource =
|
||||
'fetch("./module.wasm"); new URL("https://example.invalid/module.wasm?version=1", import.meta.url);';
|
||||
const glueValues = [
|
||||
"./module.wasm",
|
||||
"https://example.invalid/module.wasm?version=1",
|
||||
...["\n", "\r", "\r\n", "\u2028", "\u2029"].flatMap((separator) => [
|
||||
`https://example.invalid/module.wasm?query=a${separator}b`,
|
||||
`https://example.invalid/module.wasm#fragment=a${separator}b`,
|
||||
]),
|
||||
];
|
||||
const glueLiterals = glueValues.map((value) =>
|
||||
JSON.stringify(value)
|
||||
.replace(/\u2028/gu, "\\u2028")
|
||||
.replace(/\u2029/gu, "\\u2029"),
|
||||
);
|
||||
const glueSource = [
|
||||
`fetch(${glueLiterals[0]}); new URL(${glueLiterals[1]}, import.meta.url);`,
|
||||
...glueLiterals.slice(2).map((literal) => `fetch(${literal});`),
|
||||
].join("\n");
|
||||
const glueSources = [
|
||||
glueSource,
|
||||
'import source module from "./module.wasm"; new URL("https://example.invalid/module.wasm?version=1", import.meta.url);',
|
||||
@@ -285,21 +300,32 @@ try {
|
||||
assert.equal(observed.artifact.bytes, Buffer.byteLength(source));
|
||||
assert.equal(await readFile(gluePaths[index], "utf8"), source);
|
||||
assert.equal(observed.parse_status, "complete");
|
||||
assert.deepEqual(
|
||||
observed.references.map(({ value }) => value),
|
||||
glueValues,
|
||||
);
|
||||
assert.deepEqual(
|
||||
observed.references.map(({ candidate_paths }) => candidate_paths),
|
||||
[[path], [path, candidate]],
|
||||
glueValues.map((_, index) =>
|
||||
index === 0 ? [path] : [path, candidate],
|
||||
),
|
||||
);
|
||||
assert.deepEqual(
|
||||
observed.references.map(({ association }) => association),
|
||||
glueValues.map((_, index) =>
|
||||
index === 0 ? "local-path-candidate" : "basename-candidates",
|
||||
),
|
||||
);
|
||||
assert.deepEqual(
|
||||
observed.references.map(({ line, column }) => ({ line, column })),
|
||||
["./module.wasm", "https://example.invalid/module.wasm?version=1"].map(
|
||||
(value) => {
|
||||
const offset = source.indexOf(`"${value}"`);
|
||||
return {
|
||||
line: source.slice(0, offset).split("\n").length,
|
||||
column: offset - source.lastIndexOf("\n", offset) - 1,
|
||||
};
|
||||
},
|
||||
),
|
||||
glueLiterals.map((literal) => {
|
||||
const offset = source.indexOf(literal);
|
||||
assert.notEqual(offset, -1);
|
||||
return {
|
||||
line: source.slice(0, offset).split("\n").length,
|
||||
column: offset - source.lastIndexOf("\n", offset) - 1,
|
||||
};
|
||||
}),
|
||||
);
|
||||
}
|
||||
assert.notEqual(report.candidates[0].sha256, report.candidates[1].sha256);
|
||||
|
||||
@@ -541,7 +541,7 @@ const registerSignatureInputAdmissionCases = () => {
|
||||
if (!damaged.ok) {
|
||||
expect(damaged.error._tag).toBe("AnalysisInputError");
|
||||
expect(JSON.stringify(damaged.error)).toContain(
|
||||
"malformed Zip64 EOCD record",
|
||||
"invalid Zip64 EOCD signature",
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -44,6 +44,12 @@ export const mapApkBlockError = (
|
||||
`The selected file is not a ZIP/APK (${error.detail}).`,
|
||||
{ remediationAction: "Select a local .apk/.zip/.aab archive." },
|
||||
);
|
||||
case "zip64-unsupported":
|
||||
return new AnalysisUnsupportedTargetError(
|
||||
OPERATION,
|
||||
path,
|
||||
`This archive uses a Zip64 structure this reader does not decode (${error.detail}).`,
|
||||
);
|
||||
case "malformed-zip":
|
||||
return inputError(
|
||||
`Damaged ZIP/APK at byte ${String(error.offset)}: ${error.detail}.`,
|
||||
|
||||
@@ -80,7 +80,7 @@ const unwrapCmdCommand = (command: readonly string[]): readonly string[] => {
|
||||
};
|
||||
|
||||
/** Preserve the package argument from a supported package-runner command. */
|
||||
export const clientRegistrationPackageReference = (
|
||||
const clientRegistrationPackageReference = (
|
||||
command: readonly string[],
|
||||
): string | undefined => {
|
||||
const arguments_ = unwrapCmdCommand(command);
|
||||
@@ -100,20 +100,6 @@ export const clientRegistrationPackageReference = (
|
||||
return undefined;
|
||||
};
|
||||
|
||||
const isOwnedPackageReference = (packageReference: string): boolean => {
|
||||
if (
|
||||
packageReference === PRODUCT_IDENTITY.packageName ||
|
||||
packageReference === PRODUCT_IDENTITY.packageSpecifier ||
|
||||
packageReference === PRODUCT_IDENTITY.registrationPackageSpecifier
|
||||
)
|
||||
return true;
|
||||
const versionPrefix = `${PRODUCT_IDENTITY.packageName}@`;
|
||||
return (
|
||||
packageReference.startsWith(versionPrefix) &&
|
||||
valid(packageReference.slice(versionPrefix.length)) !== null
|
||||
);
|
||||
};
|
||||
|
||||
/** Check that one parsed command points to REA's MCP entry point. */
|
||||
export type ClientRegistrationCommandIdentity = "owned" | "foreign" | "unknown";
|
||||
|
||||
@@ -182,69 +168,110 @@ export const equivalentClientRegistrationCommands = (
|
||||
previous.every((value, index) => value === requested[index])
|
||||
)
|
||||
return true;
|
||||
const entrypoint = (command: readonly string[]): string | undefined => {
|
||||
if (
|
||||
command.length === 2 &&
|
||||
command[1] === "mcp" &&
|
||||
isAbsolute(command[0] ?? "")
|
||||
)
|
||||
return command[0];
|
||||
if (
|
||||
command.length === 3 &&
|
||||
command[2] === "mcp" &&
|
||||
isAbsolute(command[1] ?? "") &&
|
||||
entrypointIdentity(command[0] ?? "", process.execPath) === "owned"
|
||||
)
|
||||
return normalizeEntrypointPath(command[1]);
|
||||
return undefined;
|
||||
};
|
||||
const current = entrypoint(requested);
|
||||
return (
|
||||
current !== undefined &&
|
||||
entrypoint(previous) !== undefined &&
|
||||
classifyClientRegistrationCommand(previous, current) === "owned"
|
||||
);
|
||||
const current = inspectClientRegistrationCommand(
|
||||
requested,
|
||||
undefined,
|
||||
).entrypoint;
|
||||
if (current === undefined) return false;
|
||||
const prior = inspectClientRegistrationCommand(previous, current);
|
||||
return prior.entrypoint !== undefined && prior.identity === "owned";
|
||||
};
|
||||
|
||||
/** Classify the command once; uncertainty never grants positive ownership. */
|
||||
export interface ClientRegistrationCommandFacts {
|
||||
readonly identity: ClientRegistrationCommandIdentity;
|
||||
/** Selected absolute launcher/script path; excludes package and unresolved PATH commands. */
|
||||
readonly entrypoint: string | undefined;
|
||||
readonly package:
|
||||
| {
|
||||
readonly reference: string;
|
||||
readonly exactVersion: string | undefined;
|
||||
}
|
||||
| undefined;
|
||||
}
|
||||
|
||||
/** Parse package identity once, retaining exact source spelling and version facts. */
|
||||
export const inspectClientRegistrationCommand = (
|
||||
command: readonly string[],
|
||||
currentCommandPath: string | undefined,
|
||||
): ClientRegistrationCommandFacts => {
|
||||
const reference = clientRegistrationPackageReference(command);
|
||||
if (reference !== undefined) {
|
||||
const prefix = `${PRODUCT_IDENTITY.packageName}@`;
|
||||
const version = reference.startsWith(prefix)
|
||||
? reference.slice(prefix.length)
|
||||
: undefined;
|
||||
const normalized = version === undefined ? null : valid(version);
|
||||
const owned =
|
||||
reference === PRODUCT_IDENTITY.packageName ||
|
||||
reference === PRODUCT_IDENTITY.packageSpecifier ||
|
||||
reference === PRODUCT_IDENTITY.registrationPackageSpecifier ||
|
||||
normalized !== null;
|
||||
return {
|
||||
identity: owned ? "owned" : "foreign",
|
||||
entrypoint: undefined,
|
||||
package: {
|
||||
reference,
|
||||
exactVersion: normalized === version ? version : undefined,
|
||||
},
|
||||
};
|
||||
}
|
||||
return inspectDirectRegistrationCommand(command, currentCommandPath);
|
||||
};
|
||||
|
||||
/** Ownership-only consumers share the same parsed command facts. */
|
||||
export const classifyClientRegistrationCommand = (
|
||||
command: readonly string[],
|
||||
currentCommandPath: string | undefined,
|
||||
): ClientRegistrationCommandIdentity => {
|
||||
const packageReference = clientRegistrationPackageReference(command);
|
||||
if (packageReference !== undefined)
|
||||
return isOwnedPackageReference(packageReference) ? "owned" : "foreign";
|
||||
): ClientRegistrationCommandIdentity =>
|
||||
inspectClientRegistrationCommand(command, currentCommandPath).identity;
|
||||
|
||||
const inspectDirectRegistrationCommand = (
|
||||
command: readonly string[],
|
||||
currentCommandPath: string | undefined,
|
||||
): ClientRegistrationCommandFacts => {
|
||||
const currentPathAvailable =
|
||||
currentCommandPath !== undefined && currentCommandPath.length > 0;
|
||||
if (command.length === 3 && command[2] === "mcp") {
|
||||
const interpreter = command[0] ?? "";
|
||||
const entrypoint = command[1] ?? "";
|
||||
if (
|
||||
!isAbsolute(interpreter) ||
|
||||
!isAbsolute(entrypoint) ||
|
||||
!currentPathAvailable
|
||||
)
|
||||
return "unknown";
|
||||
if (!isAbsolute(interpreter) || !isAbsolute(entrypoint))
|
||||
return { identity: "unknown", entrypoint: undefined, package: undefined };
|
||||
const interpreterIdentity = entrypointIdentity(
|
||||
interpreter,
|
||||
process.execPath,
|
||||
);
|
||||
if (interpreterIdentity !== "owned") return interpreterIdentity;
|
||||
if (interpreterIdentity !== "owned")
|
||||
return {
|
||||
identity: currentPathAvailable ? interpreterIdentity : "unknown",
|
||||
entrypoint: undefined,
|
||||
package: undefined,
|
||||
};
|
||||
// Node selects its script from normalized argv; executable paths use native
|
||||
// traversal semantics, where symlink/.. can denote a different file.
|
||||
const selectedEntrypoint = normalizeEntrypointPath(entrypoint);
|
||||
return selectedEntrypoint === undefined
|
||||
? "unknown"
|
||||
: entrypointIdentity(selectedEntrypoint, currentCommandPath);
|
||||
return {
|
||||
identity:
|
||||
selectedEntrypoint === undefined || !currentPathAvailable
|
||||
? "unknown"
|
||||
: entrypointIdentity(selectedEntrypoint, currentCommandPath),
|
||||
entrypoint: selectedEntrypoint,
|
||||
package: undefined,
|
||||
};
|
||||
}
|
||||
if (command.length === 2 && command[1] === "mcp") {
|
||||
const executable = command[0] ?? "";
|
||||
if (executable === PRODUCT_IDENTITY.cliBinary) return "owned";
|
||||
if (executable === PRODUCT_IDENTITY.cliBinary)
|
||||
return { identity: "owned", entrypoint: undefined, package: undefined };
|
||||
if (isAbsolute(executable))
|
||||
return currentPathAvailable
|
||||
? directEntrypointIdentity(executable, currentCommandPath)
|
||||
: "unknown";
|
||||
if (/[./\\]/u.test(executable)) return "unknown";
|
||||
return {
|
||||
identity: currentPathAvailable
|
||||
? directEntrypointIdentity(executable, currentCommandPath)
|
||||
: "unknown",
|
||||
entrypoint: executable,
|
||||
package: undefined,
|
||||
};
|
||||
if (/[./\\]/u.test(executable))
|
||||
return { identity: "unknown", entrypoint: undefined, package: undefined };
|
||||
}
|
||||
return "foreign";
|
||||
return { identity: "foreign", entrypoint: undefined, package: undefined };
|
||||
};
|
||||
@@ -11,8 +11,6 @@ import { lstat } from "node:fs/promises";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { isAbsolute } from "node:path";
|
||||
|
||||
import { valid } from "semver";
|
||||
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
|
||||
import { CURRENT_ENTRYPOINT_PATH } from "../process/CurrentEntrypointPath.js";
|
||||
@@ -24,8 +22,8 @@ import {
|
||||
type McpToolTimeoutInputs,
|
||||
} from "../mcpStartupPolicy.js";
|
||||
import {
|
||||
clientRegistrationPackageReference,
|
||||
classifyClientRegistrationCommand,
|
||||
inspectClientRegistrationCommand,
|
||||
type ClientRegistrationCommandFacts,
|
||||
packageRunnerRegistrationCommand,
|
||||
} from "./ClientRegistrationIdentity.js";
|
||||
import {
|
||||
@@ -279,16 +277,21 @@ export const readClientRegistrationInspections = async (
|
||||
registration.command,
|
||||
...registration.args,
|
||||
];
|
||||
const packageReference = clientRegistrationPackageReference(command);
|
||||
const commandFacts = inspectClientRegistrationCommand(
|
||||
command,
|
||||
currentCommandPath,
|
||||
);
|
||||
const packageReference = commandFacts.package?.reference;
|
||||
const activation = registrationActivation(registration, client, parsed);
|
||||
const packageVersion = exactRegisteredPackageVersion(packageReference);
|
||||
const packageVersion = commandFacts.package?.exactVersion;
|
||||
const packageVersionDrift =
|
||||
packageVersion !== undefined &&
|
||||
packageVersion !== PRODUCT_IDENTITY.packageVersion;
|
||||
const alignment = registrationAligned({
|
||||
registration,
|
||||
client: effectiveClient,
|
||||
currentCommandPath,
|
||||
command,
|
||||
commandFacts,
|
||||
platform: options.platform ?? process.platform,
|
||||
kimiInputSchemaProfileSelected:
|
||||
registrationEnvironment?.ok === true &&
|
||||
@@ -373,21 +376,19 @@ const registrationActivation = (
|
||||
const registrationAligned = ({
|
||||
registration,
|
||||
client,
|
||||
currentCommandPath,
|
||||
command,
|
||||
commandFacts,
|
||||
platform,
|
||||
kimiInputSchemaProfileSelected,
|
||||
}: {
|
||||
readonly registration: ClientRegistration;
|
||||
readonly client: SetupClient;
|
||||
readonly currentCommandPath: string | undefined;
|
||||
readonly command: readonly string[];
|
||||
readonly commandFacts: ClientRegistrationCommandFacts;
|
||||
readonly platform: NodeJS.Platform;
|
||||
readonly kimiInputSchemaProfileSelected: boolean;
|
||||
}): "aligned" | "stale" | "unknown" => {
|
||||
const command = [registration.command, ...registration.args];
|
||||
const identity = classifyClientRegistrationCommand(
|
||||
command,
|
||||
currentCommandPath,
|
||||
);
|
||||
const { identity } = commandFacts;
|
||||
if (identity === "unknown") return "unknown";
|
||||
if (identity === "foreign") return "stale";
|
||||
if (
|
||||
@@ -427,7 +428,7 @@ const registrationAligned = ({
|
||||
)
|
||||
return "aligned";
|
||||
// Other owned package references remain stale until pinned to this version.
|
||||
if (clientRegistrationPackageReference(command) !== undefined) return "stale";
|
||||
if (commandFacts.package !== undefined) return "stale";
|
||||
// The identity owner already established an exact direct launcher match.
|
||||
// A bare command still depends on the client's PATH, so cannot prove alignment.
|
||||
return isAbsolute(command[0] ?? "") ? "aligned" : "stale";
|
||||
@@ -505,16 +506,6 @@ const timeoutAdvisory = (
|
||||
return `The configured ${setting} (${currentValue}) is shorter than REA's known provider allowance (${String(providerAllowance)} ms), so provider startup or an upstream request may exceed this client deadline. REA recommends ${setting} = ${recommendedValue} to cover that allowance plus ${headroom} of operation headroom; this additional headroom is a recommendation, not a guarantee for every analysis.`;
|
||||
};
|
||||
|
||||
const exactRegisteredPackageVersion = (
|
||||
packageReference: string | undefined,
|
||||
): string | undefined => {
|
||||
if (packageReference === undefined) return undefined;
|
||||
const prefix = `${PRODUCT_IDENTITY.packageName}@`;
|
||||
if (!packageReference.startsWith(prefix)) return undefined;
|
||||
const version = packageReference.slice(prefix.length);
|
||||
return valid(version) === version ? version : undefined;
|
||||
};
|
||||
|
||||
const packageVersionRemediation = (
|
||||
packageReference: string,
|
||||
client: SetupClient,
|
||||
|
||||
@@ -46,8 +46,7 @@ export class NativeExportsService {
|
||||
if (
|
||||
!report.success ||
|
||||
report.data.artifact.path !== input.data.path ||
|
||||
report.data.artifact.bytes > input.data.max_file_bytes ||
|
||||
report.data.coverage.exports !== report.data.exports.length
|
||||
report.data.artifact.bytes > input.data.max_file_bytes
|
||||
)
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { decodedSegment } from "../../domain/analysisView/environmentStateRules.js";
|
||||
import type { FieldRule } from "../../domain/analysisView/environmentStateWriteRules.js";
|
||||
import { fromJsonSchema } from "@modelcontextprotocol/server";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
@@ -139,22 +140,37 @@ const entityOf = (
|
||||
run: ProbeRun,
|
||||
tool: EnvironmentManifestTool,
|
||||
segments: readonly string[],
|
||||
answer: ReplayResult,
|
||||
): {
|
||||
readonly scope: string;
|
||||
readonly at: string;
|
||||
readonly identity: boolean;
|
||||
readonly rules: readonly FieldRule[];
|
||||
} => {
|
||||
const view = run.store?.state.views.get(tool.name);
|
||||
const plan = run.store?.state.writes.get(tool.name);
|
||||
const response = plan?.response;
|
||||
const scope = collectionScope(tool, segments);
|
||||
if (view?.shape === "item" || view?.shape === "list")
|
||||
return { scope, at: view.pointer, identity: false };
|
||||
return { scope, at: view.pointer, identity: false, rules: [] };
|
||||
const identity = response?.shape === "identity";
|
||||
const at = identity
|
||||
? (response.pointer ?? "")
|
||||
: (response?.pointer ?? plan?.observation?.pointer ?? "");
|
||||
return { scope, at, identity };
|
||||
const provenance = (answer as Partial<StateAnswer>).state;
|
||||
const renderedWrite =
|
||||
provenance?.source === "state" &&
|
||||
(provenance.view === "create" ||
|
||||
provenance.view === "update" ||
|
||||
provenance.view === "document-write" ||
|
||||
provenance.view === "delete");
|
||||
// A missing target replays a recorded absence with state provenance too;
|
||||
// only an applied write can render the plan's entity response.
|
||||
const rules =
|
||||
plan?.served === true && response?.shape === "entity" && renderedWrite
|
||||
? plan.rules
|
||||
: [];
|
||||
return { scope, at, identity, rules };
|
||||
};
|
||||
|
||||
const fromState = (answer: ReplayResult): boolean => {
|
||||
@@ -353,7 +369,7 @@ const compareStep = (
|
||||
status: answered.value.status,
|
||||
body: envBody(run, answered.value),
|
||||
};
|
||||
const entity = entityOf(run, call.tool, liveSegments);
|
||||
const entity = entityOf(run, call.tool, liveSegments, answered.value);
|
||||
// Another call's recording holds other entities, whose ids would map wrongly and, the first pairing winning, for good.
|
||||
if (answersCall(run, call, args, answered.value))
|
||||
learnIds(identityValue(live.body), identityValue(env.body), run.map, {
|
||||
@@ -362,21 +378,7 @@ const compareStep = (
|
||||
entityAt: entity.at,
|
||||
identity: entity.identity,
|
||||
});
|
||||
const plan = run.store?.state.writes.get(call.tool.name);
|
||||
const generatedFields = new Set(
|
||||
fromState(answered.value) && plan?.served === true
|
||||
? plan.rules.flatMap((rule) =>
|
||||
rule.rule === "unique" ||
|
||||
(rule.rule === "identity" && rule.generator !== "sequence")
|
||||
? [rule.field]
|
||||
: [],
|
||||
)
|
||||
: [],
|
||||
);
|
||||
const compared = compareAnswers(live, env, run.map, {
|
||||
...entity,
|
||||
generatedFields,
|
||||
});
|
||||
const compared = compareAnswers(live, env, run.map, entity);
|
||||
for (const key of NORMALISED_KEYS)
|
||||
run.normalised[key] += compared.normalised[key];
|
||||
const differences = [...compared.differences];
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import type { FieldRule } from "../../domain/analysisView/environmentStateWriteRules.js";
|
||||
import { compareAnswers } from "./environmentCompareDiff.js";
|
||||
import { IdMap, learnIds } from "./environmentCompareIds.js";
|
||||
|
||||
@@ -9,11 +10,13 @@ const compare = (
|
||||
env: unknown,
|
||||
map = new IdMap(),
|
||||
status: [number, number] = [200, 200],
|
||||
rules: readonly FieldRule[] = [],
|
||||
) =>
|
||||
compareAnswers(
|
||||
{ status: status[0], body: { kind: "json", value: live as JsonValue } },
|
||||
{ status: status[1], body: { kind: "json", value: env as JsonValue } },
|
||||
map,
|
||||
{ rules },
|
||||
);
|
||||
|
||||
describe("early-year clock formats", () => {
|
||||
@@ -31,8 +34,27 @@ describe("early-year clock formats", () => {
|
||||
);
|
||||
});
|
||||
|
||||
const registerValueComparisonCases = () => {
|
||||
it("normalizes mapped ids and clocks while comparing unproven generated-looking values literally", () => {
|
||||
const registerGeneratedValueComparisonCases = () => {
|
||||
it.each([
|
||||
{
|
||||
name: "normalizes mapped ids and clocks while comparing unproven generated-looking values literally",
|
||||
rules: [],
|
||||
generated: 0,
|
||||
differences: [
|
||||
["value", "/token"],
|
||||
["value", "/uuid"],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "agrees on mapped ids, clocks and confirmed generated values, and counts each",
|
||||
rules: [
|
||||
{ field: "token", rule: "identity", generator: "token" },
|
||||
{ field: "uuid", rule: "unique" },
|
||||
],
|
||||
generated: 2,
|
||||
differences: [],
|
||||
},
|
||||
] as const)("$name", ({ rules, generated, differences }) => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1);
|
||||
const result = compare(
|
||||
@@ -49,17 +71,16 @@ const registerValueComparisonCases = () => {
|
||||
uuid: "0a1b2c3d-4e5f-4a6b-9c7d-8e9f0a1b2c3d",
|
||||
},
|
||||
map,
|
||||
[200, 200],
|
||||
rules,
|
||||
);
|
||||
expect(
|
||||
result.differences.map(({ category, pointer }) => [category, pointer]),
|
||||
).toEqual([
|
||||
["value", "/token"],
|
||||
["value", "/uuid"],
|
||||
]);
|
||||
).toEqual(differences);
|
||||
expect(result.normalised).toEqual({
|
||||
ids: 1,
|
||||
clocks: 1,
|
||||
generated: 0,
|
||||
generated,
|
||||
large_bodies: 0,
|
||||
unobserved_bodies: 0,
|
||||
});
|
||||
@@ -69,20 +90,24 @@ const registerValueComparisonCases = () => {
|
||||
const result = compareAnswers(
|
||||
{
|
||||
status: 201,
|
||||
body: { kind: "json", value: { token: "live-token", sku: "product2" } },
|
||||
body: { kind: "json", value: { token: "token00001", sku: "product2" } },
|
||||
},
|
||||
{
|
||||
status: 201,
|
||||
body: { kind: "json", value: { token: "env-token", sku: "product1" } },
|
||||
body: { kind: "json", value: { token: "token00002", sku: "product1" } },
|
||||
},
|
||||
new IdMap(),
|
||||
{ generatedFields: new Set(["token"]) },
|
||||
{ rules: [{ field: "token", rule: "identity", generator: "token" }] },
|
||||
);
|
||||
expect(result.differences).toMatchObject([
|
||||
{ category: "value", pointer: "/sku" },
|
||||
]);
|
||||
expect(result.normalised.generated).toBe(1);
|
||||
});
|
||||
};
|
||||
|
||||
describe("compareAnswers", () => {
|
||||
registerGeneratedValueComparisonCases();
|
||||
|
||||
it("categorises status, missing and extra fields, values and clock formats", () => {
|
||||
const result = compare(
|
||||
@@ -129,9 +154,9 @@ const registerValueComparisonCases = () => {
|
||||
),
|
||||
).toEqual(["value"]);
|
||||
});
|
||||
};
|
||||
});
|
||||
|
||||
const registerMappedIdentityComparisonCases = () => {
|
||||
describe("mapped fields and literal collections", () => {
|
||||
it("compares an object keyed by mapped ids under the environment's keys", () => {
|
||||
const map = new IdMap();
|
||||
map.learn("liveTable", "envTable");
|
||||
@@ -193,7 +218,7 @@ const registerMappedIdentityComparisonCases = () => {
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("reports scalar reordering and keeps unnamed epoch-shaped array measurements literal", () => {
|
||||
it("reports a reordered set of scalars as order alone and keeps nested numeric measurements literal", () => {
|
||||
expect(
|
||||
compare(
|
||||
{ access: ["a", "b", "c"] },
|
||||
@@ -203,13 +228,224 @@ const registerMappedIdentityComparisonCases = () => {
|
||||
expect(
|
||||
compare({ stats: [[1792450800, 3]] }, { stats: [[1792285200, 3]] })
|
||||
.differences,
|
||||
).toMatchObject([{ category: "value", pointer: "/stats/0/0" }]);
|
||||
).toEqual([
|
||||
{
|
||||
category: "value",
|
||||
pointer: "/stats/0/0",
|
||||
live: "1792450800",
|
||||
env: "1792285200",
|
||||
},
|
||||
]);
|
||||
});
|
||||
};
|
||||
});
|
||||
|
||||
describe("compareAnswers", () => {
|
||||
registerValueComparisonCases();
|
||||
registerMappedIdentityComparisonCases();
|
||||
describe("normalization context", () => {
|
||||
it.each([
|
||||
{ live: 1800000000, env: 1700000000, pointer: "" },
|
||||
{ live: [1800000000], env: [1700000000], pointer: "/0" },
|
||||
{
|
||||
live: { samples: [1800000000] },
|
||||
env: { samples: [1700000000] },
|
||||
pointer: "/samples/0",
|
||||
},
|
||||
{
|
||||
live: { updated_at: [{ samples: 1800000000 }] },
|
||||
env: { updated_at: [{ samples: 1700000000 }] },
|
||||
pointer: "/updated_at/0/samples",
|
||||
},
|
||||
])(
|
||||
"keeps epoch-shaped numbers literal at $pointer without a time field",
|
||||
({ live, env, pointer }) => {
|
||||
const result = compare(live, env);
|
||||
expect(result.differences).toEqual([
|
||||
{ category: "value", pointer, live: "1800000000", env: "1700000000" },
|
||||
]);
|
||||
expect(result.normalised.clocks).toBe(0);
|
||||
},
|
||||
);
|
||||
|
||||
it("preserves the time field through nested numeric clock arrays", () => {
|
||||
const result = compare(
|
||||
{ updated_at: [[1700000000]] },
|
||||
{ updated_at: [[1800000000]] },
|
||||
);
|
||||
expect(result.differences).toEqual([]);
|
||||
expect(result.normalised.clocks).toBe(1);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["product1", "product2"],
|
||||
["product1", "productX"],
|
||||
["3f2a9c8d7e", "9b8e7d6c5a"],
|
||||
[
|
||||
"6f1c2b9e-2a3d-4f5e-8a9b-0c1d2e3f4a5b",
|
||||
"0a1b2c3d-4e5f-4a6b-9c7d-8e9f0a1b2c3d",
|
||||
],
|
||||
])(
|
||||
"keeps literal business strings %s versus %s without generation facts",
|
||||
(live, env) => {
|
||||
const result = compare(
|
||||
{ sku: live, items: [live] },
|
||||
{ sku: env, items: [env] },
|
||||
);
|
||||
expect(
|
||||
result.differences.map(({ category, pointer }) => [category, pointer]),
|
||||
).toEqual([
|
||||
["value", "/sku"],
|
||||
["value", "/items/0"],
|
||||
]);
|
||||
expect(result.normalised.generated).toBe(0);
|
||||
},
|
||||
);
|
||||
|
||||
it("scopes confirmed renewal rules to their response entity and exact array pointers", () => {
|
||||
const first = "6f1c2b9e-2a3d-4f5e-8a9b-0c1d2e3f4a5b";
|
||||
const second = "0a1b2c3d-4e5f-4a6b-9c7d-8e9f0a1b2c3d";
|
||||
const answer = (token: string) => ({
|
||||
status: 200,
|
||||
body: {
|
||||
kind: "json" as const,
|
||||
value: {
|
||||
data: { values: [{ token, sku: token }] },
|
||||
values: [{ token }],
|
||||
},
|
||||
},
|
||||
});
|
||||
const result = compareAnswers(answer(first), answer(second), new IdMap(), {
|
||||
at: "/data",
|
||||
rules: [{ field: "values", rule: "renewed", pointers: ["/*/token"] }],
|
||||
});
|
||||
expect(result.normalised.generated).toBe(1);
|
||||
expect(
|
||||
result.differences.map(({ category, pointer }) => [category, pointer]),
|
||||
).toEqual([
|
||||
["value", "/data/values/0/sku"],
|
||||
["value", "/values/0/token"],
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not extend generated-string normalization to numeric sequences", () => {
|
||||
const result = compare(
|
||||
{ code: 17 },
|
||||
{ code: 18 },
|
||||
new IdMap(),
|
||||
[200, 200],
|
||||
[{ field: "code", rule: "sequence", scope: "type" }],
|
||||
);
|
||||
expect(result.differences).toEqual([
|
||||
{ category: "value", pointer: "/code", live: "17", env: "18" },
|
||||
]);
|
||||
expect(result.normalised.generated).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("generated response identities", () => {
|
||||
const first = "6f1c2b9e-2a3d-4f5e-8a9b-0c1d2e3f4a5b";
|
||||
const second = "0a1b2c3d-4e5f-4a6b-9c7d-8e9f0a1b2c3d";
|
||||
const fresh = "123e4567-e89b-42d3-a456-426614174000";
|
||||
const other = "223e4567-e89b-42d3-a456-426614174000";
|
||||
const rules: readonly FieldRule[] = [
|
||||
{ field: "left", rule: "renewed", pointers: ["/*"] },
|
||||
{ field: "right", rule: "renewed", pointers: ["/*"] },
|
||||
];
|
||||
const answer = (left: string, right: string) => ({
|
||||
status: 200,
|
||||
body: { kind: "json" as const, value: { left: [left], right: [right] } },
|
||||
});
|
||||
|
||||
it.each([
|
||||
{
|
||||
name: "consistent fresh renaming",
|
||||
liveLeft: fresh,
|
||||
liveRight: fresh,
|
||||
envLeft: first,
|
||||
envRight: first,
|
||||
differs: false,
|
||||
generated: 2,
|
||||
},
|
||||
{
|
||||
name: "case-equivalent UUID aliases",
|
||||
liveLeft: fresh.toUpperCase(),
|
||||
liveRight: fresh,
|
||||
envLeft: first,
|
||||
envRight: first.toUpperCase(),
|
||||
differs: false,
|
||||
generated: 2,
|
||||
},
|
||||
{
|
||||
name: "split alias",
|
||||
liveLeft: fresh,
|
||||
liveRight: other,
|
||||
envLeft: first,
|
||||
envRight: first,
|
||||
differs: true,
|
||||
generated: 1,
|
||||
},
|
||||
{
|
||||
name: "merged identities",
|
||||
liveLeft: fresh,
|
||||
liveRight: fresh,
|
||||
envLeft: first,
|
||||
envRight: second,
|
||||
differs: true,
|
||||
generated: 1,
|
||||
},
|
||||
{
|
||||
name: "split after an unchanged array",
|
||||
liveLeft: first,
|
||||
liveRight: fresh,
|
||||
envLeft: first,
|
||||
envRight: first,
|
||||
differs: true,
|
||||
generated: 0,
|
||||
},
|
||||
{
|
||||
name: "merge after an unchanged array",
|
||||
liveLeft: first,
|
||||
liveRight: first,
|
||||
envLeft: first,
|
||||
envRight: second,
|
||||
differs: true,
|
||||
generated: 0,
|
||||
},
|
||||
])(
|
||||
"preserves $name within one confirmed renewal response",
|
||||
({ liveLeft, liveRight, envLeft, envRight, differs, generated }) => {
|
||||
const result = compareAnswers(
|
||||
answer(liveLeft, liveRight),
|
||||
answer(envLeft, envRight),
|
||||
new IdMap(),
|
||||
{ rules },
|
||||
);
|
||||
expect(result.normalised.generated).toBe(generated);
|
||||
expect(result.differences).toEqual(
|
||||
differs
|
||||
? [
|
||||
{
|
||||
category: "value",
|
||||
pointer: "/right/0",
|
||||
live: JSON.stringify(liveRight),
|
||||
env: JSON.stringify(envRight),
|
||||
},
|
||||
]
|
||||
: [],
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it("retains order-only comparison for reordered generated scalar sets", () => {
|
||||
const result = compare(
|
||||
{ left: [first, second] },
|
||||
{ left: [second, first] },
|
||||
new IdMap(),
|
||||
[200, 200],
|
||||
rules,
|
||||
);
|
||||
expect(result.differences).toEqual([
|
||||
{ category: "order", pointer: "/left" },
|
||||
]);
|
||||
expect(result.normalised.generated).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("clock field names", () => {
|
||||
@@ -347,6 +583,32 @@ describe("identity responses two collections reuse", () => {
|
||||
});
|
||||
|
||||
describe("wrapped answers", () => {
|
||||
it("keeps collection identity context separate from a wrapped array's clock field context", () => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1, "/api/books");
|
||||
map.learn(101, 2, "/api/authors");
|
||||
const result = compareAnswers(
|
||||
{
|
||||
status: 200,
|
||||
body: {
|
||||
kind: "json",
|
||||
value: { data: [{ id: 101, updated_at: [1700000000] }] },
|
||||
},
|
||||
},
|
||||
{
|
||||
status: 200,
|
||||
body: {
|
||||
kind: "json",
|
||||
value: { data: [{ id: 2, updated_at: [1800000000] }] },
|
||||
},
|
||||
},
|
||||
map,
|
||||
{ scope: "/api/authors", at: "/data" },
|
||||
);
|
||||
expect(result.differences).toEqual([]);
|
||||
expect(result.normalised).toMatchObject({ ids: 1, clocks: 1 });
|
||||
});
|
||||
|
||||
it("compares the identity of an entity inside an envelope within the call's collection", () => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1, "/api/books");
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
import type { JsonObject } from "../../domain/analysisView/environmentStateRules.js";
|
||||
import type { FieldRule } from "../../domain/analysisView/environmentStateWriteRules.js";
|
||||
import {
|
||||
isUuid,
|
||||
readClock,
|
||||
type ClockFormat,
|
||||
} from "../../domain/analysisView/environmentStateWriteValues.js";
|
||||
@@ -88,7 +90,54 @@ const clockFormat = (
|
||||
return clock === undefined ? undefined : formatKey(clock.format);
|
||||
};
|
||||
|
||||
const characterClasses = (text: string): string =>
|
||||
[
|
||||
...new Set(
|
||||
text
|
||||
.replaceAll(/\d/gu, "0")
|
||||
.replaceAll(/[a-z]/gu, "a")
|
||||
.replaceAll(/[A-Z]/gu, "A"),
|
||||
),
|
||||
]
|
||||
.sort()
|
||||
.join("");
|
||||
|
||||
/** A confirmed response field's rule, and the pointer within that field (`*` marks actual array traversal). */
|
||||
interface Generation {
|
||||
readonly rule: FieldRule;
|
||||
readonly pointer: string;
|
||||
}
|
||||
|
||||
/** Whether a confirmed generation rule applies here and both strings retain its generated format. */
|
||||
const generatedPair = (
|
||||
live: string,
|
||||
env: string,
|
||||
generation: Generation | undefined,
|
||||
): boolean => {
|
||||
if (generation === undefined) return false;
|
||||
const { rule, pointer } = generation;
|
||||
if (rule.rule === "renewed")
|
||||
return rule.pointers.includes(pointer) && isUuid(live) && isUuid(env);
|
||||
if (pointer !== "") return false;
|
||||
if (
|
||||
rule.rule === "unique" ||
|
||||
(rule.rule === "identity" && rule.generator === "uuid")
|
||||
)
|
||||
return isUuid(live) && isUuid(env);
|
||||
if (rule.rule !== "sequence" && rule.rule !== "identity") return false;
|
||||
return (
|
||||
live.length === env.length &&
|
||||
live.length >= 8 &&
|
||||
/\d/u.test(live) &&
|
||||
/\d/u.test(env) &&
|
||||
characterClasses(live) === characterClasses(env)
|
||||
);
|
||||
};
|
||||
|
||||
class Differ {
|
||||
readonly #rules: ReadonlyMap<string, FieldRule>;
|
||||
readonly #generatedToEnv = new Map<string, string>();
|
||||
readonly #generatedToLive = new Map<string, string>();
|
||||
readonly differences: Difference[] = [];
|
||||
readonly normalised: Normalised = {
|
||||
ids: 0,
|
||||
@@ -105,14 +154,39 @@ class Differ {
|
||||
readonly scope: string | undefined;
|
||||
readonly at: string;
|
||||
readonly identity: boolean;
|
||||
readonly generatedFields: ReadonlySet<string>;
|
||||
/** Parsed rules of the confirmed write whose entity response is being compared. */
|
||||
readonly rules: readonly FieldRule[];
|
||||
},
|
||||
) {}
|
||||
) {
|
||||
this.#rules = new Map(entity.rules.map((rule) => [rule.field, rule]));
|
||||
}
|
||||
|
||||
add(difference: Difference): void {
|
||||
this.differences.push(difference);
|
||||
}
|
||||
|
||||
/** One answer's generated identities must rename consistently in both directions. */
|
||||
#pairGenerated(live: string, env: string, rule: FieldRule): boolean {
|
||||
const uuid =
|
||||
rule.rule === "unique" ||
|
||||
rule.rule === "renewed" ||
|
||||
(rule.rule === "identity" && rule.generator === "uuid");
|
||||
const identity = (value: string) =>
|
||||
uuid ? `uuid:${value.toLowerCase()}` : `string:${value}`;
|
||||
const liveKey = identity(live);
|
||||
const envKey = identity(env);
|
||||
const knownEnv = this.#generatedToEnv.get(liveKey);
|
||||
const knownLive = this.#generatedToLive.get(envKey);
|
||||
if (
|
||||
(knownEnv !== undefined && knownEnv !== envKey) ||
|
||||
(knownLive !== undefined && knownLive !== liveKey)
|
||||
)
|
||||
return false;
|
||||
this.#generatedToEnv.set(liveKey, envKey);
|
||||
this.#generatedToLive.set(envKey, liveKey);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Whether a value compares within the call's collection, as an entity's own identity or a create's bare id does; other ids compare by the probe's mapping. */
|
||||
#inCollection(
|
||||
pointer: string,
|
||||
@@ -140,10 +214,29 @@ class Differ {
|
||||
live: JsonValue,
|
||||
env: JsonValue,
|
||||
pointer: string,
|
||||
at: { readonly key?: string; readonly own?: boolean } = {},
|
||||
at: {
|
||||
readonly key?: string;
|
||||
readonly own?: boolean;
|
||||
readonly clockKey?: string | undefined;
|
||||
} = {},
|
||||
generation?: Generation,
|
||||
): void {
|
||||
const key = at.clockKey ?? at.key;
|
||||
const liveClock = clockFormat(live, key);
|
||||
const envClock = clockFormat(env, key);
|
||||
const generatedMatch =
|
||||
generation !== undefined &&
|
||||
typeof live === "string" &&
|
||||
typeof env === "string" &&
|
||||
liveClock === undefined &&
|
||||
envClock === undefined &&
|
||||
!this.map.hasLive(live) &&
|
||||
!this.map.hasEnv(env) &&
|
||||
generatedPair(live, env, generation)
|
||||
? this.#pairGenerated(live, env, generation.rule)
|
||||
: undefined;
|
||||
if (live === env) {
|
||||
if (this.#reassigned(live, pointer, at))
|
||||
if (generatedMatch === false || this.#reassigned(live, pointer, at))
|
||||
this.add({
|
||||
category: "value",
|
||||
pointer,
|
||||
@@ -152,17 +245,17 @@ class Differ {
|
||||
});
|
||||
return;
|
||||
}
|
||||
const { key } = at;
|
||||
const field = at.key;
|
||||
if (
|
||||
(typeof live === "string" || typeof live === "number") &&
|
||||
(typeof env === "string" || typeof env === "number")
|
||||
) {
|
||||
const mapsAsId =
|
||||
(key !== undefined && ID_FIELD.test(key)) ||
|
||||
(field !== undefined && ID_FIELD.test(field)) ||
|
||||
this.#inCollection(pointer, at) ||
|
||||
(typeof live === "string" &&
|
||||
key !== undefined &&
|
||||
this.map.isStateReference(live, key));
|
||||
field !== undefined &&
|
||||
this.map.isStateReference(live, field));
|
||||
const mapped = mapsAsId
|
||||
? this.map.env(
|
||||
live,
|
||||
@@ -173,8 +266,6 @@ class Differ {
|
||||
this.normalised.ids += 1;
|
||||
return;
|
||||
}
|
||||
const liveClock = clockFormat(live, key);
|
||||
const envClock = clockFormat(env, key);
|
||||
if (liveClock !== undefined && envClock !== undefined) {
|
||||
if (liveClock === envClock) this.normalised.clocks += 1;
|
||||
else
|
||||
@@ -186,15 +277,7 @@ class Differ {
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (
|
||||
typeof live === "string" &&
|
||||
typeof env === "string" &&
|
||||
at.own === true &&
|
||||
key !== undefined &&
|
||||
this.entity.generatedFields.has(key) &&
|
||||
!this.map.hasLive(live) &&
|
||||
!this.map.hasEnv(env)
|
||||
) {
|
||||
if (generatedMatch === true) {
|
||||
this.normalised.generated += 1;
|
||||
return;
|
||||
}
|
||||
@@ -207,9 +290,28 @@ class Differ {
|
||||
});
|
||||
}
|
||||
|
||||
array(live: JsonValue[], env: JsonValue[], pointer: string): void {
|
||||
array(
|
||||
live: JsonValue[],
|
||||
env: JsonValue[],
|
||||
pointer: string,
|
||||
clockKey: string | undefined,
|
||||
generation: Generation | undefined,
|
||||
): void {
|
||||
const own = pointer === this.entity.at;
|
||||
if (this.sameItems(live, env, pointer)) return;
|
||||
if (this.sameItems(live, env, pointer)) {
|
||||
// Equal/reordered scalar sets still establish unchanged generated pairs;
|
||||
// otherwise another field could split an alias hidden by this fast path.
|
||||
if (generation !== undefined)
|
||||
for (const [index, item] of live.entries())
|
||||
this.scalar(
|
||||
item,
|
||||
item,
|
||||
`${pointer}/${String(index)}`,
|
||||
{ own, clockKey },
|
||||
{ ...generation, pointer: `${generation.pointer}/*` },
|
||||
);
|
||||
return;
|
||||
}
|
||||
const pairs = pairElements(live, env);
|
||||
if (live.length !== env.length || pairs.length !== live.length)
|
||||
this.add({
|
||||
@@ -224,9 +326,18 @@ class Differ {
|
||||
)
|
||||
this.add({ category: "order", pointer });
|
||||
for (const [left, right] of pairs)
|
||||
this.value(live[left]!, env[right]!, `${pointer}/${String(left)}`, {
|
||||
own,
|
||||
});
|
||||
this.value(
|
||||
live[left]!,
|
||||
env[right]!,
|
||||
`${pointer}/${String(left)}`,
|
||||
{
|
||||
own,
|
||||
clockKey,
|
||||
},
|
||||
generation === undefined
|
||||
? undefined
|
||||
: { ...generation, pointer: `${generation.pointer}/*` },
|
||||
);
|
||||
}
|
||||
|
||||
/** Arrays of scalars holding the same items in another order differ only in order, as sets an application returns unordered do. */
|
||||
@@ -262,6 +373,7 @@ class Differ {
|
||||
env: JsonObject,
|
||||
pointer: string,
|
||||
own = false,
|
||||
generation?: Generation,
|
||||
): void {
|
||||
const self = own || pointer === this.entity.at;
|
||||
// An object keyed by ids compares each member under the environment's key; any other compares field by field.
|
||||
@@ -286,11 +398,25 @@ class Differ {
|
||||
});
|
||||
else {
|
||||
matched.add(other);
|
||||
this.value(value, env[other]!, child, {
|
||||
key,
|
||||
own: self,
|
||||
keyed: self && keyed,
|
||||
});
|
||||
const rule = self ? this.#rules.get(key) : undefined;
|
||||
this.value(
|
||||
value,
|
||||
env[other]!,
|
||||
child,
|
||||
{
|
||||
key,
|
||||
own: self,
|
||||
keyed: self && keyed,
|
||||
},
|
||||
rule !== undefined
|
||||
? { rule, pointer: "" }
|
||||
: generation === undefined
|
||||
? undefined
|
||||
: {
|
||||
...generation,
|
||||
pointer: `${generation.pointer}/${escapePointer(key)}`,
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
for (const [key, value] of Object.entries(env))
|
||||
@@ -311,16 +437,19 @@ class Differ {
|
||||
readonly key?: string;
|
||||
readonly own?: boolean;
|
||||
readonly keyed?: boolean;
|
||||
readonly clockKey?: string | undefined;
|
||||
} = {},
|
||||
generation?: Generation,
|
||||
): void {
|
||||
if (Array.isArray(live) && Array.isArray(env))
|
||||
this.array(live, env, pointer);
|
||||
this.array(live, env, pointer, at.clockKey ?? at.key, generation);
|
||||
else if (isObject(live) && isObject(env))
|
||||
this.object(
|
||||
live,
|
||||
env,
|
||||
pointer,
|
||||
at.own === true && (at.key === undefined || at.keyed === true),
|
||||
generation,
|
||||
);
|
||||
else if (
|
||||
Array.isArray(live) ||
|
||||
@@ -334,7 +463,7 @@ class Differ {
|
||||
live: shown(live),
|
||||
env: shown(env),
|
||||
});
|
||||
else this.scalar(live, env, pointer, at);
|
||||
else this.scalar(live, env, pointer, at, generation);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -355,15 +484,14 @@ export const compareAnswers = (
|
||||
readonly scope?: string;
|
||||
readonly at?: string;
|
||||
readonly identity?: boolean;
|
||||
/** Direct entity fields whose executed write plan proves independent generation. */
|
||||
readonly generatedFields?: ReadonlySet<string>;
|
||||
readonly rules?: readonly FieldRule[];
|
||||
} = {},
|
||||
): { readonly differences: Difference[]; readonly normalised: Normalised } => {
|
||||
const differ = new Differ(map, {
|
||||
scope: entity.scope,
|
||||
at: entity.at ?? "",
|
||||
identity: entity.identity === true,
|
||||
generatedFields: entity.generatedFields ?? new Set(),
|
||||
rules: entity.rules ?? [],
|
||||
});
|
||||
if (live.status !== env.status)
|
||||
differ.add({
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
import { fixedAllowanceOptions } from "../../domain/fixedAllowanceRemediation.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { lstat } from "node:fs/promises";
|
||||
import { streamChunkToBuffer } from "../StreamBytes.js";
|
||||
import { DirectoryArtifactReader } from "../DirectoryArtifactReader.js";
|
||||
import { ArtifactResourceScope } from "../ArtifactResourceScope.js";
|
||||
import { translateArtifactFailure } from "../ArtifactOperationFailure.js";
|
||||
@@ -489,7 +490,7 @@ const digestBounded = async (
|
||||
"Asset catalog inspection cancelled",
|
||||
{ cause: signal.reason },
|
||||
);
|
||||
const bytes = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
|
||||
const bytes = streamChunkToBuffer(chunk);
|
||||
length += bytes.length;
|
||||
if (length > MAX_CATALOG_BYTES)
|
||||
throw new ArtifactReaderFailure(
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
isSourceLessAppKitAnnotationConnector,
|
||||
isUIKitRuntimeConnectionClass,
|
||||
} from "../../domain/apple/interfaceBuilderKeyedArchive.js";
|
||||
import { streamChunkToBuffer } from "../StreamBytes.js";
|
||||
import { DirectoryArtifactReader } from "../DirectoryArtifactReader.js";
|
||||
import { ArtifactResourceScope } from "../ArtifactResourceScope.js";
|
||||
import { ArtifactTraversalObservation } from "../ArtifactTraversalObservation.js";
|
||||
@@ -1015,9 +1016,7 @@ const readEntry = async (
|
||||
const chunks: Buffer[] = [];
|
||||
let length = 0;
|
||||
for await (const chunk of stream) {
|
||||
const bytes = Buffer.isBuffer(chunk)
|
||||
? chunk
|
||||
: Buffer.from(chunk as Uint8Array);
|
||||
const bytes = streamChunkToBuffer(chunk);
|
||||
length += bytes.length;
|
||||
if (length > maxInputBytes) {
|
||||
stream.destroy();
|
||||
|
||||
@@ -108,6 +108,7 @@ it.each([
|
||||
{ command: LC.VERSION_MIN_IPHONEOS, cpu: CPU.i386, platform: 7 },
|
||||
{ command: LC.VERSION_MIN_IPHONEOS, cpu: CPU.x86_64, platform: 7 },
|
||||
{ command: LC.VERSION_MIN_IPHONEOS, cpu: CPU.arm64, platform: 2 },
|
||||
{ command: LC.VERSION_MIN_TVOS, cpu: CPU.i386, platform: 3 },
|
||||
{ command: LC.VERSION_MIN_TVOS, cpu: CPU.x86_64, platform: 8 },
|
||||
{ command: LC.VERSION_MIN_TVOS, cpu: CPU.arm64, platform: 3 },
|
||||
{ command: LC.VERSION_MIN_WATCHOS, cpu: CPU.i386, platform: 9 },
|
||||
|
||||
@@ -5,6 +5,10 @@ import type {
|
||||
MachoSlice,
|
||||
} from "../../domain/apple/dylibResolution.js";
|
||||
import { machoCpuSubtypeBase } from "../../domain/apple/machoCpuIdentity.js";
|
||||
import {
|
||||
legacyMachoPlatform,
|
||||
type MachoVersionMinPlatform,
|
||||
} from "../../domain/apple/machoPlatform.js";
|
||||
import {
|
||||
classifyFatContainerIdentity,
|
||||
readFatContainerHeader,
|
||||
@@ -53,53 +57,18 @@ const LC_LOAD_UPWARD_DYLIB = 0x80000023;
|
||||
const LC_RPATH = 0x8000001c;
|
||||
const LC_DYLD_ENVIRONMENT = 0x27;
|
||||
const LC_CODE_SIGNATURE = 0x1d;
|
||||
const LC_VERSION_MIN_MACOSX = 0x24;
|
||||
const LC_VERSION_MIN_IPHONEOS = 0x25;
|
||||
const LC_VERSION_MIN_TVOS = 0x2f;
|
||||
const LC_VERSION_MIN_WATCHOS = 0x30;
|
||||
const LC_BUILD_VERSION = 0x32;
|
||||
|
||||
// `mach-o/loader.h` `PLATFORM_*` numbers observed via Apple linker output.
|
||||
const PLATFORM_MACOS = 1;
|
||||
const PLATFORM_IOS = 2;
|
||||
const PLATFORM_TVOS = 3;
|
||||
const PLATFORM_WATCHOS = 4;
|
||||
const PLATFORM_IOSSIMULATOR = 7;
|
||||
const PLATFORM_TVOSSIMULATOR = 8;
|
||||
const PLATFORM_WATCHOSSIMULATOR = 9;
|
||||
|
||||
const CPU_TYPE_I386 = 7;
|
||||
const CPU_TYPE_X86_64 = 0x01000007;
|
||||
const CPU_TYPE_ARM64 = 0x0100000c;
|
||||
|
||||
/**
|
||||
* Legacy `LC_VERSION_MIN_*` commands predate explicit simulator platform
|
||||
* IDs. Per Apple's dyld rules the image CPU type distinguishes simulator
|
||||
* from device: Intel images are simulator, ARM images are device
|
||||
* (tvOS simulator applies to x86_64 only). `LC_BUILD_VERSION` platform
|
||||
* numbers are preserved verbatim; unknown future numbers stay unknown
|
||||
* downstream, never mapped to nearest.
|
||||
*/
|
||||
const legacyVersionMinPlatform = (
|
||||
command: number,
|
||||
cpuType: number,
|
||||
): number | undefined => {
|
||||
const isIntel = cpuType === CPU_TYPE_I386 || cpuType === CPU_TYPE_X86_64;
|
||||
switch (command) {
|
||||
case LC_VERSION_MIN_MACOSX:
|
||||
return PLATFORM_MACOS;
|
||||
case LC_VERSION_MIN_IPHONEOS:
|
||||
return isIntel ? PLATFORM_IOSSIMULATOR : PLATFORM_IOS;
|
||||
case LC_VERSION_MIN_TVOS:
|
||||
return cpuType === CPU_TYPE_X86_64
|
||||
? PLATFORM_TVOSSIMULATOR
|
||||
: PLATFORM_TVOS;
|
||||
case LC_VERSION_MIN_WATCHOS:
|
||||
return isIntel ? PLATFORM_WATCHOSSIMULATOR : PLATFORM_WATCHOS;
|
||||
default:
|
||||
return undefined;
|
||||
}
|
||||
};
|
||||
/** loader.h command codes and their device platform identities. */
|
||||
const VERSION_MIN_PLATFORMS: ReadonlyMap<number, MachoVersionMinPlatform> =
|
||||
new Map([
|
||||
[0x24, 1], // LC_VERSION_MIN_MACOSX
|
||||
[0x25, 2], // LC_VERSION_MIN_IPHONEOS
|
||||
[0x2f, 3], // LC_VERSION_MIN_TVOS
|
||||
[0x30, 4], // LC_VERSION_MIN_WATCHOS
|
||||
]);
|
||||
|
||||
/** `mach-o/loader.h` `dylib_use_command`: `nameoff == 28` and this marker. */
|
||||
const DYLIB_USE_MARKER = 0x1a741800;
|
||||
@@ -679,14 +648,14 @@ const decodeCommand = (
|
||||
slice.platforms.push(platform);
|
||||
return;
|
||||
}
|
||||
const legacyPlatform = legacyVersionMinPlatform(command, slice.cpu_type);
|
||||
const legacyPlatform = VERSION_MIN_PLATFORMS.get(command);
|
||||
if (legacyPlatform !== undefined) {
|
||||
if (body.byteLength < 16)
|
||||
throw new MachoFormatIssue(
|
||||
"malformed",
|
||||
`load command ${index} is too short for a version-min command`,
|
||||
);
|
||||
slice.platforms.push(legacyPlatform);
|
||||
slice.platforms.push(legacyMachoPlatform(legacyPlatform, slice.cpu_type));
|
||||
return;
|
||||
}
|
||||
if (command === LC_LAZY_LOAD_DYLIB_INFO)
|
||||
|
||||
@@ -62,6 +62,7 @@ export class CdpCaptureCompleteness {
|
||||
if (POLICY_REASONS.has(reason)) this.#policyFiltered.add(section);
|
||||
if (
|
||||
reason === "provider_unavailable" ||
|
||||
reason === "unattributed_target" ||
|
||||
reason === "invalid_protocol_value" ||
|
||||
reason === "unknown_target_relationship"
|
||||
)
|
||||
|
||||
@@ -11,6 +11,7 @@ import { optionalCdpCommand } from "./CdpOptionalCommand.js";
|
||||
import { authorizedElectronFile } from "./ElectronFileScope.js";
|
||||
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
|
||||
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
|
||||
import { workerTargetOwnership } from "./CdpWorkerTargetOwnership.js";
|
||||
|
||||
/** Inventory local worker targets without attaching to them. */
|
||||
export const captureElectronWorkers = async (input: {
|
||||
@@ -53,13 +54,22 @@ export const captureElectronWorkers = async (input: {
|
||||
}
|
||||
const type = cdpStringValue(target.type) ?? "";
|
||||
if (!type.includes("worker")) continue;
|
||||
const openerTargetId = boundedTargetField(target.openerId);
|
||||
const parentFrameId = boundedTargetField(target.parentFrameId);
|
||||
if (
|
||||
openerTargetId !== input.target.id &&
|
||||
(parentFrameId === null || !input.frameIds.has(parentFrameId))
|
||||
) {
|
||||
input.completeness.exclude("workers", "out_of_target_scope");
|
||||
const ownership = workerTargetOwnership(
|
||||
target,
|
||||
input.target.id,
|
||||
input.frameIds,
|
||||
);
|
||||
if (!ownership.ok) {
|
||||
input.completeness.exclude("workers", "invalid_protocol_value");
|
||||
continue;
|
||||
}
|
||||
if (ownership.value.scope !== "selected") {
|
||||
input.completeness.exclude(
|
||||
"workers",
|
||||
ownership.value.scope === "unknown"
|
||||
? "unattributed_target"
|
||||
: "out_of_target_scope",
|
||||
);
|
||||
continue;
|
||||
}
|
||||
const path = await authorizedElectronFile(
|
||||
@@ -86,8 +96,8 @@ export const captureElectronWorkers = async (input: {
|
||||
type,
|
||||
file_path: path,
|
||||
attached: target.attached,
|
||||
opener_target_id: openerTargetId,
|
||||
parent_frame_id: parentFrameId,
|
||||
opener_target_id: ownership.value.openerTargetId,
|
||||
parent_frame_id: ownership.value.parentFrameId,
|
||||
});
|
||||
}
|
||||
return workers.sort((left, right) =>
|
||||
|
||||
@@ -125,3 +125,49 @@ describe("web worker target inventory", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it.each([`${origin}/worker.js`, ""])(
|
||||
"keeps an unattributed worker unavailable rather than policy filtered: %s",
|
||||
async (url) => {
|
||||
const result = await capture({
|
||||
targetInfos: [
|
||||
{ targetId: "worker", type: "worker", url, attached: false },
|
||||
],
|
||||
});
|
||||
expect(result.workers).toEqual([]);
|
||||
expect(result.completeness).toMatchObject({
|
||||
status: "attach_limited",
|
||||
conditions: ["attach_limited"],
|
||||
unavailable_sections: ["workers"],
|
||||
policy_filtered_sections: [],
|
||||
excluded: [
|
||||
{ section: "workers", reason: "unattributed_target", count: 1 },
|
||||
],
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it.each([{ openerId: "" }, { parentFrameId: null }])(
|
||||
"retains invalid supplied relationship coverage: %j",
|
||||
async (relationship) => {
|
||||
const result = await capture({
|
||||
targetInfos: [
|
||||
{
|
||||
targetId: "worker",
|
||||
type: "worker",
|
||||
url: `${origin}/worker.js`,
|
||||
attached: false,
|
||||
...relationship,
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(result.workers).toEqual([]);
|
||||
expect(result.completeness).toMatchObject({
|
||||
unavailable_sections: ["workers"],
|
||||
policy_filtered_sections: [],
|
||||
excluded: [
|
||||
{ section: "workers", reason: "invalid_protocol_value", count: 1 },
|
||||
],
|
||||
});
|
||||
},
|
||||
);
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
} from "./CdpCaptureValues.js";
|
||||
import type { CaptureContext } from "./CdpPageCapture.js";
|
||||
import { optionalCdpCommand } from "./CdpOptionalCommand.js";
|
||||
import { workerTargetOwnership } from "./CdpWorkerTargetOwnership.js";
|
||||
|
||||
interface CaptureWorkersInput {
|
||||
readonly context: CaptureContext;
|
||||
@@ -62,39 +63,25 @@ export const captureWorkers = async (
|
||||
malformedEntries += 1;
|
||||
continue;
|
||||
}
|
||||
const openerId =
|
||||
target.openerId === undefined
|
||||
? undefined
|
||||
: cdpStringValue(target.openerId);
|
||||
const parentFrameId =
|
||||
target.parentFrameId === undefined
|
||||
? undefined
|
||||
: cdpStringValue(target.parentFrameId);
|
||||
if (
|
||||
(target.openerId !== undefined && openerId === undefined) ||
|
||||
(target.parentFrameId !== undefined && parentFrameId === undefined)
|
||||
) {
|
||||
const ownership = workerTargetOwnership(
|
||||
target,
|
||||
context.target.id,
|
||||
frameIds,
|
||||
);
|
||||
if (!ownership.ok) {
|
||||
malformedEntries += 1;
|
||||
continue;
|
||||
}
|
||||
const url = allowedSanitizedUrl(target.url, allowedOrigins);
|
||||
const relatedToPage =
|
||||
openerId === context.target.id ||
|
||||
(parentFrameId !== undefined && frameIds.has(parentFrameId));
|
||||
const relationship = relatedToPage
|
||||
? "related"
|
||||
: openerId !== undefined || parentFrameId !== undefined
|
||||
? "foreign"
|
||||
: "unknown";
|
||||
if (relationship !== "related") {
|
||||
if (ownership.value.scope !== "selected") {
|
||||
completeness.exclude(
|
||||
"workers",
|
||||
relationship === "foreign"
|
||||
? "out_of_target_scope"
|
||||
: "unknown_target_relationship",
|
||||
ownership.value.scope === "unknown"
|
||||
? "unattributed_target"
|
||||
: "out_of_target_scope",
|
||||
);
|
||||
continue;
|
||||
}
|
||||
const url = allowedSanitizedUrl(target.url, allowedOrigins);
|
||||
if (url === undefined) {
|
||||
const rawUrl = cdpStringValue(target.url);
|
||||
completeness.exclude(
|
||||
@@ -113,8 +100,8 @@ export const captureWorkers = async (
|
||||
url: url.url,
|
||||
origin: url.origin,
|
||||
attached: target.attached,
|
||||
opener_target_id: openerId ?? null,
|
||||
parent_frame_id: parentFrameId ?? null,
|
||||
opener_target_id: ownership.value.openerTargetId,
|
||||
parent_frame_id: ownership.value.parentFrameId,
|
||||
});
|
||||
}
|
||||
if (malformedEntries > 0)
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { cdpStringValue } from "./CdpCaptureValues.js";
|
||||
|
||||
/** Optional CDP relationships establish ownership only when supplied. */
|
||||
export const workerTargetOwnership = (
|
||||
target: Readonly<Record<string, unknown>>,
|
||||
selectedTargetId: string,
|
||||
frameIds: ReadonlySet<string>,
|
||||
): Result<
|
||||
{
|
||||
readonly scope: "selected" | "foreign" | "unknown";
|
||||
readonly openerTargetId: string | null;
|
||||
readonly parentFrameId: string | null;
|
||||
},
|
||||
{ readonly _tag: "InvalidWorkerRelationship" }
|
||||
> => {
|
||||
const openerTargetId = cdpStringValue(target.openerId);
|
||||
const parentFrameId = cdpStringValue(target.parentFrameId);
|
||||
if (
|
||||
(target.openerId !== undefined &&
|
||||
(openerTargetId === undefined || openerTargetId === "")) ||
|
||||
(target.parentFrameId !== undefined &&
|
||||
(parentFrameId === undefined || parentFrameId === ""))
|
||||
)
|
||||
return err({ _tag: "InvalidWorkerRelationship" });
|
||||
const selected =
|
||||
openerTargetId === selectedTargetId ||
|
||||
(parentFrameId !== undefined && frameIds.has(parentFrameId));
|
||||
return ok({
|
||||
scope: selected
|
||||
? "selected"
|
||||
: openerTargetId === undefined && parentFrameId === undefined
|
||||
? "unknown"
|
||||
: "foreign",
|
||||
openerTargetId: openerTargetId ?? null,
|
||||
parentFrameId: parentFrameId ?? null,
|
||||
});
|
||||
};
|
||||
@@ -1,6 +1,8 @@
|
||||
import { err, ok, type Result } from "../../../domain/result.js";
|
||||
import {
|
||||
createFileRegistry,
|
||||
createRegistry,
|
||||
equals,
|
||||
fromBinary,
|
||||
ScalarType,
|
||||
type DescField,
|
||||
@@ -9,6 +11,8 @@ import {
|
||||
} from "@bufbuild/protobuf";
|
||||
import {
|
||||
FileDescriptorSetSchema,
|
||||
FileDescriptorProtoSchema,
|
||||
type FileDescriptorProto,
|
||||
type FileDescriptorSet,
|
||||
} from "@bufbuild/protobuf/wkt";
|
||||
import { BinaryReader } from "@bufbuild/protobuf/wire";
|
||||
@@ -52,55 +56,91 @@ const scalarWire = (scalar: ScalarType): ProtobufField["wire_type"] => {
|
||||
const floatingValue = (value: number): string =>
|
||||
Object.is(value, -0) ? "-0" : String(value);
|
||||
|
||||
/** Qualified names can expand a small descriptor into a much larger registry. */
|
||||
const admitDescriptorRegistry = async (
|
||||
// Descriptor custom options and future fields remain opaque identity facts.
|
||||
const descriptorEquality = { registry: createRegistry(), unknown: true };
|
||||
|
||||
/** Admit descriptor identities and bound qualified names before registry construction. */
|
||||
const admitRegistryDescriptor = async (
|
||||
descriptor: FileDescriptorSet,
|
||||
budget: ProtobufBudget,
|
||||
): Promise<ProtobufSchemaAdmissionFailure | undefined> => {
|
||||
): Promise<Result<FileDescriptorSet, ProtobufSchemaAdmissionFailure>> => {
|
||||
const files = new Map<string, FileDescriptorProto>();
|
||||
const registryNames = new Set<string>();
|
||||
const duplicateType: ProtobufSchemaAdmissionFailure = {
|
||||
kind: "malformed-descriptor",
|
||||
field: "descriptor_set_path",
|
||||
message: "Duplicate fully qualified type name in FileDescriptorSet",
|
||||
};
|
||||
const chargeQualifiedName = (name: string, prefix: string) =>
|
||||
budget.charge((prefix.length + 1 + name.length) * 8 + 128);
|
||||
const admitTypeName = (name: string, prefix: string): string | null => {
|
||||
chargeQualifiedName(name, prefix);
|
||||
budget.charge(32);
|
||||
const qualified = prefix === "" ? name : `${prefix}.${name}`;
|
||||
if (registryNames.has(qualified)) return null;
|
||||
registryNames.add(qualified);
|
||||
return qualified;
|
||||
};
|
||||
for (const file of descriptor.file) {
|
||||
await budget.checkpoint();
|
||||
budget.charge(file.name.length * 8 + 128);
|
||||
const previous = files.get(file.name);
|
||||
if (previous !== undefined) {
|
||||
if (
|
||||
!equals(FileDescriptorProtoSchema, previous, file, descriptorEquality)
|
||||
)
|
||||
return err({
|
||||
kind: "malformed-descriptor",
|
||||
field: "descriptor_set_path",
|
||||
message: "Conflicting file identity in FileDescriptorSet",
|
||||
});
|
||||
continue;
|
||||
}
|
||||
files.set(file.name, file);
|
||||
const stack = file.messageType.map((message) => ({
|
||||
message,
|
||||
prefix: file.package.length,
|
||||
prefix: file.package,
|
||||
}));
|
||||
const chargeQualifiedName = (name: string, prefix: number) =>
|
||||
budget.charge((prefix + 1 + name.length) * 8 + 128);
|
||||
for (const enumeration of file.enumType)
|
||||
chargeQualifiedName(enumeration.name, file.package.length);
|
||||
if (admitTypeName(enumeration.name, file.package) === null)
|
||||
return err(duplicateType);
|
||||
for (const extension of file.extension)
|
||||
chargeQualifiedName(extension.name, file.package.length);
|
||||
if (admitTypeName(extension.name, file.package) === null)
|
||||
return err(duplicateType);
|
||||
for (const service of file.service) {
|
||||
const length = file.package.length + 1 + service.name.length;
|
||||
budget.charge(length * 8 + 128);
|
||||
const qualified = admitTypeName(service.name, file.package);
|
||||
if (qualified === null) return err(duplicateType);
|
||||
for (const method of service.method)
|
||||
chargeQualifiedName(method.name, length);
|
||||
chargeQualifiedName(method.name, qualified);
|
||||
}
|
||||
while (stack.length > 0) {
|
||||
await budget.checkpoint();
|
||||
const { message, prefix } = stack.pop()!;
|
||||
const numbers = new Set<number>();
|
||||
const qualified = admitTypeName(message.name, prefix);
|
||||
if (qualified === null) return err(duplicateType);
|
||||
budget.charge(message.field.length * 32);
|
||||
const fieldNumbers = new Set<number>();
|
||||
for (const field of message.field) {
|
||||
if (numbers.has(field.number))
|
||||
return {
|
||||
chargeQualifiedName(field.name, qualified);
|
||||
if (fieldNumbers.has(field.number))
|
||||
return err({
|
||||
kind: "malformed-descriptor",
|
||||
field: "descriptor_set_path",
|
||||
message: `Duplicate field number ${field.number} in descriptor message ${message.name}`,
|
||||
};
|
||||
numbers.add(field.number);
|
||||
});
|
||||
fieldNumbers.add(field.number);
|
||||
}
|
||||
const length = prefix + 1 + message.name.length;
|
||||
budget.charge(length * 8 + 128);
|
||||
for (const field of message.field)
|
||||
chargeQualifiedName(field.name, length);
|
||||
for (const extension of message.extension)
|
||||
chargeQualifiedName(extension.name, length);
|
||||
if (admitTypeName(extension.name, qualified) === null)
|
||||
return err(duplicateType);
|
||||
for (const enumeration of message.enumType)
|
||||
chargeQualifiedName(enumeration.name, length);
|
||||
if (admitTypeName(enumeration.name, qualified) === null)
|
||||
return err(duplicateType);
|
||||
for (const nested of message.nestedType)
|
||||
stack.push({ message: nested, prefix: length });
|
||||
stack.push({ message: nested, prefix: qualified });
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
return ok({ ...descriptor, file: [...files.values()] });
|
||||
};
|
||||
|
||||
const scalarValue = (reader: BinaryReader, scalar: ScalarType): string => {
|
||||
@@ -166,10 +206,17 @@ export class ProtobufSchema implements WireSchema {
|
||||
redact,
|
||||
new ProtobufSchema(FileDescriptorSetSchema, budget, redact),
|
||||
);
|
||||
const invalidField = checked.fields.find(
|
||||
(field) => field.schema_status === "invalid-message",
|
||||
);
|
||||
const wireError =
|
||||
checked.error ??
|
||||
checked.fields.find((field) => field.schema_status === "invalid-message")
|
||||
?.schema_error;
|
||||
(invalidField === undefined
|
||||
? undefined
|
||||
: (invalidField.schema_error ?? {
|
||||
offset: invalidField.offset,
|
||||
reason: "Invalid descriptor message",
|
||||
}));
|
||||
if (wireError !== undefined)
|
||||
return err({
|
||||
kind: "malformed-descriptor",
|
||||
@@ -179,7 +226,7 @@ export class ProtobufSchema implements WireSchema {
|
||||
let descriptor: FileDescriptorSet;
|
||||
try {
|
||||
descriptor = fromBinary(FileDescriptorSetSchema, bytes, {
|
||||
readUnknownFields: false,
|
||||
readUnknownFields: true,
|
||||
recursionLimit: WEB_NETWORK_CAPTURE_LIMITS.depth,
|
||||
});
|
||||
} catch (cause) {
|
||||
@@ -189,11 +236,11 @@ export class ProtobufSchema implements WireSchema {
|
||||
message: `Malformed FileDescriptorSet: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
});
|
||||
}
|
||||
const failure = await admitDescriptorRegistry(descriptor, budget);
|
||||
if (failure !== undefined) return err(failure);
|
||||
const admitted = await admitRegistryDescriptor(descriptor, budget);
|
||||
if (!admitted.ok) return admitted;
|
||||
let message: DescMessage | undefined;
|
||||
try {
|
||||
message = createFileRegistry(descriptor).getMessage(
|
||||
message = createFileRegistry(admitted.value).getMessage(
|
||||
typeName.replace(/^\./, ""),
|
||||
);
|
||||
} catch (cause) {
|
||||
|
||||
@@ -23,6 +23,10 @@ import { inspectElfNotesInputSchema } from "../domain/native/elfNotes.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
import {
|
||||
analysisEvidenceDeliveryOptions,
|
||||
deliverCliAnalysisEvidence,
|
||||
} from "./analysisEvidenceDelivery.js";
|
||||
|
||||
/** Inspect operator-selected files through the same shared workflow as MCP. */
|
||||
export const registerBinaryDiagnosticsCommands = (
|
||||
@@ -173,6 +177,7 @@ export const registerBinaryDiagnosticsCommands = (
|
||||
args: z.object({ path: z.string().describe("Local PE or ELF image") }),
|
||||
options: z.object({
|
||||
maxFileBytes: inspectNativeExportsInputSchema.shape.max_file_bytes,
|
||||
...analysisEvidenceDeliveryOptions,
|
||||
}),
|
||||
run: ({ args, options }) =>
|
||||
withCommandCancellation(async (signal) => {
|
||||
@@ -181,8 +186,8 @@ export const registerBinaryDiagnosticsCommands = (
|
||||
logger,
|
||||
CLI_COMMANDS.inspectNativeExports,
|
||||
async () =>
|
||||
finishAnalysisCommand(
|
||||
() =>
|
||||
deliverCliAnalysisEvidence({
|
||||
run: () =>
|
||||
exports.inspect(
|
||||
{
|
||||
path: absoluteCallerPath(args.path),
|
||||
@@ -190,11 +195,13 @@ export const registerBinaryDiagnosticsCommands = (
|
||||
},
|
||||
{ signal },
|
||||
),
|
||||
exports,
|
||||
"rea-native-exports",
|
||||
"inspect_native_exports",
|
||||
service: exports,
|
||||
providerId: "rea-native-exports",
|
||||
operation: "inspect_native_exports",
|
||||
signal,
|
||||
),
|
||||
detail: options.detail,
|
||||
evidenceOutput: options.evidence_output,
|
||||
}),
|
||||
);
|
||||
}),
|
||||
});
|
||||
|
||||
@@ -101,7 +101,7 @@ export const registerGpuCommands = (cli: CliInstance, logger: Logger): void => {
|
||||
run: () =>
|
||||
service.inspect(
|
||||
{
|
||||
path: absoluteCallerPath(args.path),
|
||||
path: selected.data.path,
|
||||
analysis: selected.data.analysis,
|
||||
...(selected.data.expected_sha256 === undefined
|
||||
? {}
|
||||
|
||||
@@ -84,8 +84,8 @@ export const registerModelCommands = (
|
||||
run: () =>
|
||||
service.inspect(
|
||||
{
|
||||
path: absoluteCallerPath(args.path),
|
||||
hash: options.hash,
|
||||
path: selected.data.path,
|
||||
hash: selected.data.hash,
|
||||
detail: "complete",
|
||||
},
|
||||
{ signal },
|
||||
|
||||
@@ -17,7 +17,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
|
||||
...toolContractMetadata("inspect_analysis_view"),
|
||||
kind: "application",
|
||||
description:
|
||||
'Use javascript-routes collections to query route patterns, source-linked occurrences, source status, unknowns or summary counts from retained JavaScript application or web-bundle Evidence. Inspect selected views of supported completed Evidence without repeating analysis. Choose a view compatible with the parent operation described below. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized reads that are not entities and have one initial request line (item-shaped paths stay recorded), and a render plan per read tool: item and list views reproduce their latest recordings from the state (a list also learns the filters, search, sort and paging its recordings verify, and answers new values of them from the state), and every other read tool is a recorded view with a reason. It also reports a write plan per state-changing tool (create, update, delete or document, built from its first successful call) and write_clock; a plan is served only when replaying the capture shows a later read confirming its effect and none contradicting it. Types and identity are inferences; values are recorded data, so it refuses parents inspected with sensitive values, and a scenario parent must have selected response bodies; its values keep the capture redaction markers, and a write whose URL it redacted ends the state of every type. For a retained record use source: {kind: "retained-evidence", evidence_id: "<parent_evidencLine truncated
|
||||
'Use javascript-routes collections to query route patterns, source-linked occurrences, source status, unknowns or summary counts from retained JavaScript application or web-bundle Evidence. Inspect selected views of supported completed Evidence without repeating analysis. Choose a view compatible with the parent operation described below. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Native export tables use kind native-exports, collection exports/issues, exact names or original indices, selected fields and matching-row pagination; parent Evidence plus source_index identifies each row. Source coverage stays separate from page coverage. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized reads that are not entities and have one initial request line (item-shaped paths stay recorded), and a render plan per read tool: item and list views reproduce their latest recordings from the state (a list also learns the filters, search, sort and paging its recordings verify, and answers new values of them from the state), and every other read tool is a recorded view with a reason. It also reports a write plan per state-changing tool (create, update, delete or document, built from its first successful call) and write_clock; a plan is served only when replaying the capture shows a later read confirming its effect and none contradicting it. Types and identity are inferences; values are recorded data, so it refuses parents inspected with sensitive values, andLine truncated
|
||||
inputSchema: inspectAnalysisViewInputSchema,
|
||||
outputSchema: evidenceResultOf(analysisViewResultSchema),
|
||||
examples: [
|
||||
|
||||
@@ -9,7 +9,10 @@ import {
|
||||
inspectBinaryLayoutInputSchema,
|
||||
binaryLayoutSchema,
|
||||
} from "../../domain/native/binaryLayout.js";
|
||||
import { binaryLayoutSummaryViewResultSchema } from "../../domain/analysisView/analysisView.js";
|
||||
import {
|
||||
binaryLayoutSummaryViewResultSchema,
|
||||
nativeExportsSummaryViewResultSchema,
|
||||
} from "../../domain/analysisView/analysisView.js";
|
||||
import {
|
||||
analysisDetailOutputSchemaOf,
|
||||
analysisDetailSchema,
|
||||
@@ -38,6 +41,9 @@ import {
|
||||
export const inspectBinaryLayoutRequestSchema =
|
||||
inspectBinaryLayoutInputSchema.extend({ detail: analysisDetailSchema });
|
||||
|
||||
export const inspectNativeExportsRequestSchema =
|
||||
inspectNativeExportsInputSchema.extend({ detail: analysisDetailSchema });
|
||||
|
||||
/**
|
||||
* Public `inspect_byte_entropy` input. The shared constructor runs before
|
||||
* `detail` is added so the advertised input stays closed.
|
||||
@@ -123,9 +129,12 @@ export const BINARY_DIAGNOSTICS_TOOL_CONTRACTS = [
|
||||
...toolContractMetadata("inspect_native_exports"),
|
||||
kind: "native-provider",
|
||||
description:
|
||||
"Inspect one explicit local PE or ELF image's export table without executing it or opening an engine: PE export-directory names, ordinals, addresses and forwarders, or ELF defined default/protected global, weak and unique .dynsym symbols with type, binding and symbol version. Ordinal-only, forwarded and hidden-version exports stay distinct. Portable on any supported host. Coverage is partial when table entries are unreadable and unsupported for an ELF without section headers; an empty complete inventory means the image declares no exports. Mach-O exports use inspect_macho.",
|
||||
inputSchema: inspectNativeExportsInputSchema,
|
||||
outputSchema: evidenceResultOf(nativeExportsSchema),
|
||||
"Inspect one explicit local PE or ELF image's export table without executing it or opening an engine: PE export-directory names, ordinals, addresses and forwarders, or ELF defined default/protected global, weak and unique .dynsym symbols with type, binding and symbol version. Ordinal-only, forwarded and hidden-version exports stay distinct. Portable on any supported host. Coverage is partial when table entries are unreadable and unsupported when ELF dynamic symbols cannot be enumerated from the available section metadata; an empty complete inventory means the image declares no exports. Mach-O exports use inspect_macho. Omitted/complete detail preserves inline delivery. Select detail summary to retain complete Evidence and return identity, counts and source coverage. Query inspect_analysis_view kind native-exports, collection exports or issues, with exact names/original indices, fields and matching-row pagination. Names return every version or alias; source_index is scoped to the parent Evidence. ELF name projections keep raw-byte and encoding qualifications.",
|
||||
inputSchema: inspectNativeExportsRequestSchema,
|
||||
outputSchema: analysisDetailOutputSchemaOf(
|
||||
nativeExportsSchema,
|
||||
nativeExportsSummaryViewResultSchema,
|
||||
),
|
||||
examples: [
|
||||
{
|
||||
title: "List a native library's exports",
|
||||
|
||||
@@ -8,6 +8,12 @@ import {
|
||||
apkSignatureSourceCoverageSchema,
|
||||
} from "./apkSignatureView.js";
|
||||
import { byteEntropySchema } from "../native/byteEntropy.js";
|
||||
import {
|
||||
nativeExportsViewRequestSchema,
|
||||
nativeExportsViewItemsSchema,
|
||||
nativeExportsSummarySchema,
|
||||
projectNativeExportsView,
|
||||
} from "./nativeExportsView.js";
|
||||
import {
|
||||
byteEntropySummarySchema,
|
||||
projectByteEntropyView,
|
||||
@@ -325,6 +331,7 @@ const nativeViewSchema = z.strictObject({
|
||||
|
||||
/** Caller-selected projection of already completed analysis Evidence. */
|
||||
export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
|
||||
nativeExportsViewRequestSchema,
|
||||
javaScriptRoutesViewRequestSchema,
|
||||
nativeFunctionMatchingViewRequestSchema,
|
||||
sourceToBundleViewRequestSchema,
|
||||
@@ -443,6 +450,7 @@ const viewResultBase = {
|
||||
parent_evidence_id: prefixedDigestSchema("ev"),
|
||||
parent_operation: z.enum([
|
||||
"inspect_byte_entropy",
|
||||
"inspect_native_exports",
|
||||
"inspect_binary_layout",
|
||||
"inspect_go_binary",
|
||||
"gdb_console",
|
||||
@@ -490,6 +498,12 @@ export const byteEntropySummaryViewResultSchema = z.strictObject({
|
||||
parent_operation: z.literal("inspect_byte_entropy"),
|
||||
});
|
||||
|
||||
export const nativeExportsSummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultBase,
|
||||
parent_operation: z.literal("inspect_native_exports"),
|
||||
summary: nativeExportsSummarySchema,
|
||||
});
|
||||
|
||||
/** Opt-in command delivery keeps output once and retains the complete parent. */
|
||||
export const gdbCommandSummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultBase,
|
||||
@@ -807,6 +821,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
||||
view: environmentStateViewRequestSchema,
|
||||
}),
|
||||
z.discriminatedUnion("parent_operation", [
|
||||
nativeExportsSummaryViewResultSchema,
|
||||
byteEntropySummaryViewResultSchema,
|
||||
gdbCommandSummaryViewResultSchema,
|
||||
objdumpSummaryViewResultSchema,
|
||||
@@ -835,6 +850,14 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
||||
nativeFunctionMatchingSummaryViewResultSchema,
|
||||
sourceToBundleSummaryViewResultSchema,
|
||||
]),
|
||||
z.strictObject({
|
||||
...viewResultBase,
|
||||
kind: z.literal("native-exports"),
|
||||
parent_operation: z.literal("inspect_native_exports"),
|
||||
view: nativeExportsViewRequestSchema,
|
||||
exports: nativeExportsSummarySchema,
|
||||
items: nativeExportsViewItemsSchema,
|
||||
}),
|
||||
z.strictObject({
|
||||
...viewResultBase,
|
||||
...nativeInventoryViewContext,
|
||||
@@ -1190,6 +1213,10 @@ export const projectAnalysisView = (
|
||||
const projected = projectPcapCaptureView(parent, view);
|
||||
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
|
||||
}
|
||||
if (parent.operation === "inspect_native_exports") {
|
||||
const projected = projectNativeExportsView(parent, view);
|
||||
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
|
||||
}
|
||||
if (parent.operation === "inspect_byte_entropy") {
|
||||
const report = parseAnalysisViewResult(byteEntropySchema, parent);
|
||||
if (!report.ok) return report;
|
||||
|
||||
@@ -0,0 +1,251 @@
|
||||
import { z } from "zod";
|
||||
import {
|
||||
nativeExportsSchema,
|
||||
peNativeExportsSchema as pe,
|
||||
elfNativeExportsSchema as elf,
|
||||
type NativeExports,
|
||||
} from "../native/nativeExports.js";
|
||||
import {
|
||||
canBorrowParsedJsonSnapshot,
|
||||
freezeOwnedJsonTreeSnapshot,
|
||||
isImmutableJsonSnapshot,
|
||||
} from "../immutableJson.js";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import type {
|
||||
AnalysisViewParent,
|
||||
AnalysisViewRequest,
|
||||
UnsignedAnalysisView,
|
||||
} from "./analysisView.js";
|
||||
import { analysisViewInputError } from "./analysisViewErrors.js";
|
||||
import { parseAnalysisViewResult } from "./analysisViewParent.js";
|
||||
import {
|
||||
boundedViewMessages,
|
||||
completeWithinViewCoverage,
|
||||
pageViewCoverage,
|
||||
} from "./analysisViewCoverage.js";
|
||||
import { selectViewRows } from "./selectViewRows.js";
|
||||
|
||||
const peRow = pe.shape.exports.element;
|
||||
const elfRow = elf.shape.exports.element;
|
||||
const issue = pe.shape.issues.element;
|
||||
const sourceIndex = { source_index: z.int().nonnegative() };
|
||||
const page = {
|
||||
kind: z.literal("native-exports"),
|
||||
offset: z.int().nonnegative().default(0),
|
||||
limit: z.int().positive().default(64),
|
||||
};
|
||||
|
||||
/** Exact names preserve all versions/aliases; indices are scoped to the complete parent. */
|
||||
export const nativeExportsViewRequestSchema = z.discriminatedUnion(
|
||||
"collection",
|
||||
[
|
||||
z.strictObject({
|
||||
...page,
|
||||
collection: z.literal("exports"),
|
||||
names: z.array(z.string()).optional(),
|
||||
indices: z.array(z.int().nonnegative()).optional(),
|
||||
fields: z.array(z.union([peRow.keyof(), elfRow.keyof()])).optional(),
|
||||
}),
|
||||
z.strictObject({
|
||||
...page,
|
||||
collection: z.literal("issues"),
|
||||
indices: z.array(z.int().nonnegative()).optional(),
|
||||
}),
|
||||
],
|
||||
);
|
||||
|
||||
export const nativeExportsViewItemsSchema = z.union([
|
||||
z.array(peRow.partial().extend(sourceIndex)),
|
||||
z.array(
|
||||
elfRow.partial().extend({
|
||||
...sourceIndex,
|
||||
// Historical defaults normalize complete source rows, never omitted columns.
|
||||
name_bytes_base64: elfRow.shape.name_bytes_base64
|
||||
.removeDefault()
|
||||
.optional(),
|
||||
name_utf8_decodable: elfRow.shape.name_utf8_decodable
|
||||
.removeDefault()
|
||||
.optional(),
|
||||
}),
|
||||
),
|
||||
z.array(issue.extend(sourceIndex)),
|
||||
]);
|
||||
|
||||
const counts = {
|
||||
export_count: z.int().nonnegative(),
|
||||
issue_count: z.int().nonnegative(),
|
||||
limitation_count: z.int().nonnegative(),
|
||||
};
|
||||
/** Source coverage remains distinct from page coverage, including an empty partial page. */
|
||||
export const nativeExportsSummarySchema = z.discriminatedUnion("format", [
|
||||
pe.omit({ exports: true, issues: true, limitations: true }).extend(counts),
|
||||
elf.omit({ exports: true, issues: true, limitations: true }).extend(counts),
|
||||
]);
|
||||
|
||||
// This validates exact immutable Evidence, never a path or current filesystem state.
|
||||
// Weak keys release their parsed representation with the owning retained record.
|
||||
const checked = new WeakMap<object, NativeExports>();
|
||||
const readReport = (
|
||||
parent: AnalysisViewParent,
|
||||
): Result<NativeExports, AnalysisError> => {
|
||||
const source = parent.normalizedResult;
|
||||
const reusable =
|
||||
parent.sealedResult === true &&
|
||||
typeof source === "object" &&
|
||||
source !== null &&
|
||||
isImmutableJsonSnapshot(source);
|
||||
const known = reusable ? checked.get(source) : undefined;
|
||||
if (known !== undefined) return ok(known);
|
||||
const parsed = parseAnalysisViewResult(nativeExportsSchema, parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
if (reusable) {
|
||||
const report = canBorrowParsedJsonSnapshot(source, parsed.value)
|
||||
? source
|
||||
: freezeOwnedJsonTreeSnapshot(parsed.value);
|
||||
checked.set(source, report);
|
||||
return ok(report);
|
||||
}
|
||||
return parsed;
|
||||
};
|
||||
|
||||
/** Read selected export facts without reopening the artifact or reinterpreting its tables. */
|
||||
export const projectNativeExportsView = (
|
||||
parent: AnalysisViewParent,
|
||||
view: AnalysisViewRequest,
|
||||
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||
const invalid = (message: string) =>
|
||||
err(
|
||||
analysisViewInputError([
|
||||
{ path: ["view"], reason: "invalid_value", message },
|
||||
]),
|
||||
);
|
||||
if (view.kind !== "summary" && view.kind !== "native-exports")
|
||||
return invalid(
|
||||
"Native export Evidence supports summary and native-exports views.",
|
||||
);
|
||||
const parsed = readReport(parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
const report = parsed.value;
|
||||
const { exports, issues, limitations, ...identity } = report;
|
||||
const summary = {
|
||||
...identity,
|
||||
export_count: exports.length,
|
||||
issue_count: issues.length,
|
||||
limitation_count: limitations.length,
|
||||
};
|
||||
const shared = {
|
||||
parent_evidence_id: parent.evidenceId,
|
||||
parent_operation: "inspect_native_exports" as const,
|
||||
parent_digest: parent.evidenceId.slice(3),
|
||||
artifact: { path: report.artifact.path, sha256: report.artifact.sha256 },
|
||||
limitations: boundedViewMessages(
|
||||
[...parent.limitations, ...limitations],
|
||||
parent.evidenceId,
|
||||
"limitations",
|
||||
),
|
||||
unknowns:
|
||||
report.coverage.status === "complete"
|
||||
? []
|
||||
: [
|
||||
report.coverage.reason ??
|
||||
"Export-table coverage is incomplete; no-match pages do not prove absence.",
|
||||
],
|
||||
};
|
||||
if (view.kind === "summary")
|
||||
return ok({
|
||||
...shared,
|
||||
kind: "summary",
|
||||
view,
|
||||
summary,
|
||||
coverage: completeWithinViewCoverage(exports.length, exports.length),
|
||||
});
|
||||
const indices =
|
||||
view.indices === undefined ? undefined : new Set(view.indices);
|
||||
if (view.collection === "issues") {
|
||||
const selected =
|
||||
indices === undefined
|
||||
? {
|
||||
rows: issues
|
||||
.slice(view.offset, view.offset + view.limit)
|
||||
.map((row, index) => ({
|
||||
...row,
|
||||
source_index: view.offset + index,
|
||||
})),
|
||||
total: issues.length,
|
||||
}
|
||||
: selectViewRows(
|
||||
issues,
|
||||
view,
|
||||
(_, index) => indices.has(index),
|
||||
(row, source_index) => ({ ...row, source_index }),
|
||||
);
|
||||
return ok({
|
||||
...shared,
|
||||
kind: "native-exports",
|
||||
view,
|
||||
exports: summary,
|
||||
items: selected.rows,
|
||||
coverage: pageViewCoverage(
|
||||
view.offset,
|
||||
selected.rows.length,
|
||||
selected.total,
|
||||
),
|
||||
});
|
||||
}
|
||||
const allowed =
|
||||
report.format === "pe" ? peRow.keyof().options : elfRow.keyof().options;
|
||||
const fields =
|
||||
view.fields === undefined ? allowed : [...new Set(view.fields)];
|
||||
const allowedFields: readonly string[] = allowed;
|
||||
if (fields.some((field) => !allowedFields.includes(field)))
|
||||
return invalid(
|
||||
`Selected export fields do not belong to ${report.format}; available fields: ${allowed.join(", ")}.`,
|
||||
);
|
||||
const names = view.names === undefined ? undefined : new Set(view.names);
|
||||
// A projected decoded name still carries its exact bytes and encoding qualification.
|
||||
const columns = new Set<string>(fields);
|
||||
if (report.format === "elf" && columns.has("name")) {
|
||||
columns.add("name_bytes_base64");
|
||||
columns.add("name_utf8_decodable");
|
||||
}
|
||||
const rows: readonly NativeExports["exports"][number][] = exports;
|
||||
const projectRow = (
|
||||
row: NativeExports["exports"][number],
|
||||
source_index: number,
|
||||
) => ({
|
||||
...Object.fromEntries(
|
||||
Object.entries(row).filter(([key]) => columns.has(key)),
|
||||
),
|
||||
source_index,
|
||||
});
|
||||
// Unfiltered pages use the owner's original order directly; no whole-table scan.
|
||||
const selected =
|
||||
names === undefined && indices === undefined
|
||||
? {
|
||||
rows: rows
|
||||
.slice(view.offset, view.offset + view.limit)
|
||||
.map((row, index) => projectRow(row, view.offset + index)),
|
||||
total: rows.length,
|
||||
}
|
||||
: selectViewRows(
|
||||
rows,
|
||||
view,
|
||||
(row, index) =>
|
||||
(indices === undefined || indices.has(index)) &&
|
||||
(names === undefined || (row.name !== null && names.has(row.name))),
|
||||
projectRow,
|
||||
);
|
||||
return ok({
|
||||
...shared,
|
||||
kind: "native-exports",
|
||||
view,
|
||||
exports: summary,
|
||||
items: selected.rows,
|
||||
coverage: pageViewCoverage(
|
||||
view.offset,
|
||||
selected.rows.length,
|
||||
selected.total,
|
||||
),
|
||||
});
|
||||
};
|
||||
@@ -38,6 +38,7 @@ export const APK_BLOCK_ID = {
|
||||
|
||||
export type ApkSigningBlockError =
|
||||
| { readonly kind: "not-a-zip"; readonly detail: string }
|
||||
| { readonly kind: "zip64-unsupported"; readonly detail: string }
|
||||
| {
|
||||
/** A ZIP whose end-of-central-directory is damaged or inconsistent. */
|
||||
readonly kind: "malformed-zip";
|
||||
@@ -142,23 +143,48 @@ const centralDirectoryOffset = (
|
||||
return ok({ offset, size, entries, end: eocdOffset });
|
||||
return err({
|
||||
kind: "malformed-zip",
|
||||
offset: Math.max(0, locator),
|
||||
offset: eocdOffset,
|
||||
detail: "missing Zip64 EOCD locator",
|
||||
});
|
||||
}
|
||||
const recordOffset = u64(dv, locator + 8);
|
||||
if (
|
||||
recordOffset === null ||
|
||||
recordOffset + 56 > locator ||
|
||||
dv.getUint32(recordOffset, true) !== ZIP64_EOCD_SIGNATURE
|
||||
)
|
||||
// APPNOTE 6.3.10 §4.3.15 makes the record offset relative to its disk.
|
||||
// A selected file cannot resolve another disk's offset as a local span.
|
||||
const recordDisk = dv.getUint32(locator + 4, true);
|
||||
const totalDisks = dv.getUint32(locator + 16, true);
|
||||
if (totalDisks === 0)
|
||||
return err({
|
||||
kind: "malformed-zip",
|
||||
offset:
|
||||
recordOffset !== null && recordOffset <= bytes.length
|
||||
? recordOffset
|
||||
: locator,
|
||||
detail: "malformed Zip64 EOCD record",
|
||||
offset: locator + 16,
|
||||
detail: "Zip64 locator declares zero disks",
|
||||
});
|
||||
if (recordDisk >= totalDisks)
|
||||
return err({
|
||||
kind: "malformed-zip",
|
||||
offset: locator + 4,
|
||||
detail: "Zip64 EOCD disk is outside the declared disk count",
|
||||
});
|
||||
if (totalDisks > 1)
|
||||
return err({
|
||||
kind: "zip64-unsupported",
|
||||
detail: "multi-disk Zip64 archives are not supported",
|
||||
});
|
||||
const recordOffset = u64(dv, locator + 8);
|
||||
if (recordOffset === null)
|
||||
return err({
|
||||
kind: "zip64-unsupported",
|
||||
detail: "Zip64 EOCD offset exceeds safe range",
|
||||
});
|
||||
if (recordOffset + 56 > locator)
|
||||
return err({
|
||||
kind: "malformed-zip",
|
||||
offset: locator + 8,
|
||||
detail: "Zip64 EOCD record extends beyond its locator",
|
||||
});
|
||||
if (dv.getUint32(recordOffset, true) !== ZIP64_EOCD_SIGNATURE)
|
||||
return err({
|
||||
kind: "malformed-zip",
|
||||
offset: recordOffset,
|
||||
detail: "invalid Zip64 EOCD signature",
|
||||
});
|
||||
const z64Offset = u64(dv, recordOffset + 48);
|
||||
const z64Size = u64(dv, recordOffset + 40);
|
||||
@@ -166,9 +192,8 @@ const centralDirectoryOffset = (
|
||||
const z64Entries = u64(dv, recordOffset + 32);
|
||||
if (z64Offset === null || z64Size === null || z64Entries === null)
|
||||
return err({
|
||||
kind: "malformed-zip",
|
||||
offset: recordOffset,
|
||||
detail: "Zip64 sizes cannot describe the supplied archive extent",
|
||||
kind: "zip64-unsupported",
|
||||
detail: "Zip64 sizes exceed safe range",
|
||||
});
|
||||
return ok({
|
||||
offset: z64Offset,
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
/** Device platform encoded by a legacy LC_VERSION_MIN_* command. */
|
||||
export type MachoVersionMinPlatform = 1 | 2 | 3 | 4;
|
||||
|
||||
/**
|
||||
* dyld derives legacy simulator platforms from the CPU type. tvOS uses
|
||||
* x86_64 only; iOS and watchOS also recognize i386. Explicit LC_BUILD_VERSION
|
||||
* platforms do not use this rule.
|
||||
*/
|
||||
export const legacyMachoPlatform = (
|
||||
platform: MachoVersionMinPlatform,
|
||||
cpuType: number | null,
|
||||
): number => {
|
||||
if (platform === 3) return cpuType === 0x01000007 ? 8 : 3;
|
||||
if (cpuType === 7 || cpuType === 0x01000007) {
|
||||
if (platform === 2) return 7;
|
||||
if (platform === 4) return 9;
|
||||
}
|
||||
return platform;
|
||||
};
|
||||
@@ -36,6 +36,8 @@ const browserExclusionReasonSchema = z.enum([
|
||||
"unattributed_origin",
|
||||
"not_approved",
|
||||
"out_of_target_scope",
|
||||
"unattributed_target",
|
||||
// Accepted for previously retained captures; new producers use unattributed_target.
|
||||
"unknown_target_relationship",
|
||||
"provider_unavailable",
|
||||
"invalid_protocol_value",
|
||||
|
||||
@@ -207,7 +207,7 @@ export const il2cppMethodAddressSchema = z.strictObject({
|
||||
/** Index into the binary Il2CppType table and its C#-like display name. */
|
||||
export const il2cppTypeReferenceSchema = z.strictObject({
|
||||
type: optionalIndex,
|
||||
/** Null for type index -1, and for every row of a version 27 report. */
|
||||
/** Null for type index -1, unresolved native type-table slots, and version 27. */
|
||||
name: z.string().nullable(),
|
||||
});
|
||||
|
||||
|
||||
@@ -100,39 +100,50 @@ const coverageSchema = z.strictObject({
|
||||
});
|
||||
|
||||
/** Static export inventory of one PE or ELF image, read without execution. */
|
||||
export const nativeExportsSchema = z.discriminatedUnion("format", [
|
||||
z.strictObject({
|
||||
artifact: artifactSchema,
|
||||
format: z.literal("pe"),
|
||||
architecture: z.string().min(1),
|
||||
bits: z.union([z.literal(32), z.literal(64)]),
|
||||
/** Null when the image declares no export directory. */
|
||||
export_directory: z
|
||||
.strictObject({
|
||||
rva: uint,
|
||||
size: uint,
|
||||
dll_name: z.string().nullable(),
|
||||
ordinal_base: uint,
|
||||
timestamp: uint,
|
||||
})
|
||||
.nullable(),
|
||||
exports: z.array(peExportSchema).max(MAX_NATIVE_EXPORT_ROWS),
|
||||
coverage: coverageSchema,
|
||||
issues: z.array(nativeExportIssueSchema),
|
||||
limitations: z.array(z.string().min(1)),
|
||||
}),
|
||||
z.strictObject({
|
||||
artifact: artifactSchema,
|
||||
format: z.literal("elf"),
|
||||
architecture: z.string().min(1),
|
||||
bits: z.union([z.literal(32), z.literal(64)]),
|
||||
byte_order: z.enum(["little", "big"]),
|
||||
exports: z.array(elfExportSchema).max(MAX_NATIVE_EXPORT_ROWS),
|
||||
coverage: coverageSchema,
|
||||
issues: z.array(nativeExportIssueSchema),
|
||||
limitations: z.array(z.string().min(1)),
|
||||
}),
|
||||
]);
|
||||
export const peNativeExportsSchema = z.strictObject({
|
||||
artifact: artifactSchema,
|
||||
format: z.literal("pe"),
|
||||
architecture: z.string().min(1),
|
||||
bits: z.union([z.literal(32), z.literal(64)]),
|
||||
/** Null when the image declares no export directory. */
|
||||
export_directory: z
|
||||
.strictObject({
|
||||
rva: uint,
|
||||
size: uint,
|
||||
dll_name: z.string().nullable(),
|
||||
ordinal_base: uint,
|
||||
timestamp: uint,
|
||||
})
|
||||
.nullable(),
|
||||
exports: z.array(peExportSchema).max(MAX_NATIVE_EXPORT_ROWS),
|
||||
coverage: coverageSchema,
|
||||
issues: z.array(nativeExportIssueSchema),
|
||||
limitations: z.array(z.string().min(1)),
|
||||
});
|
||||
|
||||
export const elfNativeExportsSchema = z.strictObject({
|
||||
artifact: artifactSchema,
|
||||
format: z.literal("elf"),
|
||||
architecture: z.string().min(1),
|
||||
bits: z.union([z.literal(32), z.literal(64)]),
|
||||
byte_order: z.enum(["little", "big"]),
|
||||
exports: z.array(elfExportSchema).max(MAX_NATIVE_EXPORT_ROWS),
|
||||
coverage: coverageSchema,
|
||||
issues: z.array(nativeExportIssueSchema),
|
||||
limitations: z.array(z.string().min(1)),
|
||||
});
|
||||
|
||||
export const nativeExportsSchema = z
|
||||
.discriminatedUnion("format", [peNativeExportsSchema, elfNativeExportsSchema])
|
||||
.superRefine((value, context) => {
|
||||
if (value.coverage.exports !== value.exports.length)
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["coverage", "exports"],
|
||||
message:
|
||||
"Export coverage count must equal the retained export row count.",
|
||||
});
|
||||
});
|
||||
export type NativeExports = z.infer<typeof nativeExportsSchema>;
|
||||
export type PeExport = z.infer<typeof peExportSchema>;
|
||||
export type ElfExport = z.infer<typeof elfExportSchema>;
|
||||
|
||||
@@ -92,7 +92,7 @@ export type FileState = FileStateIdentity &
|
||||
export interface ProcessFilesystemSnapshot {
|
||||
readonly files: readonly FileState[];
|
||||
readonly truncated: boolean;
|
||||
/** Root aliases whose path enumeration was exhausted, regardless of hash coverage. */
|
||||
/** Physical root enumeration exhausted; unfollowed symlink scopes remain unknown. */
|
||||
readonly completeRoots: readonly string[];
|
||||
readonly coverage: FilesystemCoverage;
|
||||
}
|
||||
|
||||
@@ -141,6 +141,7 @@ it.each([
|
||||
{ path: "root_0:.", reason: "entry_unavailable", system_code: null },
|
||||
{ path: "root_0:.", reason: "identity_changed", system_code: "ENOENT" },
|
||||
{ path: "root_0:.", reason: "directory_unavailable", system_code: "ENOENT" },
|
||||
{ path: "root_0:.", reason: "symlink_not_followed", system_code: null },
|
||||
] as const)("rejects an inconsistent omission %j", (omission) => {
|
||||
expect(() =>
|
||||
parseProcessCapture(captureWithEnumerationOmission(omission)),
|
||||
@@ -177,6 +178,69 @@ it("requires entry-unavailability details and matching aggregate reasons", () =>
|
||||
).toThrow("enumeration_omissions");
|
||||
});
|
||||
|
||||
it("binds unfollowed coverage to observed symlink metadata without inventing an OS failure", () => {
|
||||
const partial = captureWithEnumerationOmission({
|
||||
path: "root_0:.",
|
||||
reason: "symlink_not_followed",
|
||||
system_code: null,
|
||||
});
|
||||
const link = {
|
||||
path: "root_0:.",
|
||||
type: "symlink" as const,
|
||||
mode: 0o120777,
|
||||
size: 7,
|
||||
sha256: null,
|
||||
symlink_target: "reports",
|
||||
};
|
||||
const effect = {
|
||||
path: link.path,
|
||||
status: "created" as const,
|
||||
before: null,
|
||||
after: link,
|
||||
};
|
||||
const capture = {
|
||||
...partial,
|
||||
files_after: [link],
|
||||
filesystem_effects: [effect],
|
||||
filesystem_checkpoints: partial.filesystem_checkpoints.map((checkpoint) =>
|
||||
checkpoint.name === "after_settlement"
|
||||
? { ...checkpoint, files: [link], effects: [effect] }
|
||||
: checkpoint,
|
||||
),
|
||||
};
|
||||
expect(parseProcessCapture(capture)).toEqual(capture);
|
||||
for (const changes of [
|
||||
{ enumeration_omissions: [] },
|
||||
{
|
||||
enumeration_omissions: [
|
||||
{
|
||||
path: link.path,
|
||||
reason: "symlink_not_followed",
|
||||
system_code: "ENOENT",
|
||||
},
|
||||
],
|
||||
},
|
||||
])
|
||||
expect(() =>
|
||||
parseProcessCapture({
|
||||
...capture,
|
||||
truncation_details: {
|
||||
...capture.truncation_details,
|
||||
filesystem_after: {
|
||||
...capture.truncation_details.filesystem_after,
|
||||
...changes,
|
||||
},
|
||||
},
|
||||
}),
|
||||
).toThrow("enumeration_omissions");
|
||||
expect(() =>
|
||||
parseProcessCapture({
|
||||
...capture,
|
||||
files_after: [{ ...link, type: "directory", symlink_target: null }],
|
||||
}),
|
||||
).toThrow("enumeration_omissions");
|
||||
});
|
||||
|
||||
it("never considers truncated captures equivalent", () => {
|
||||
const complete = emptyCapture();
|
||||
const capture = {
|
||||
|
||||
@@ -16,6 +16,7 @@ const enumerationOmissionReasonSchema = z.enum([
|
||||
"identity_changed",
|
||||
"directory_unavailable",
|
||||
"entry_unavailable",
|
||||
"symlink_not_followed",
|
||||
]);
|
||||
|
||||
/** Filesystem enumeration and whole-file digest coverage for one checkpoint. */
|
||||
|
||||
@@ -378,18 +378,25 @@ const validateCoverage = (
|
||||
coverage.enumeration_omissions.some(
|
||||
({ reason }) => reason === "entry_unavailable",
|
||||
)) &&
|
||||
coverage.enumeration_omissions.every(
|
||||
(omission) =>
|
||||
coverage.enumeration_reasons.includes(omission.reason) &&
|
||||
(omission.reason === "directory_unavailable"
|
||||
? entryTypes.get(omission.path) === "directory" &&
|
||||
omission.system_code !== null
|
||||
: (entryTypes.get(omission.path) === undefined ||
|
||||
entryTypes.get(omission.path) === "directory") &&
|
||||
(omission.reason === "entry_unavailable"
|
||||
? omission.system_code !== null
|
||||
: omission.system_code === null)),
|
||||
), `truncation_details.${name}.enumeration_omissions`, "every enumeration omission must identify an unavailable entry or retained directory, with a matching reason and system error only for unavailable reads");
|
||||
(!coverage.enumeration_reasons.includes("symlink_not_followed") ||
|
||||
coverage.enumeration_omissions.some(
|
||||
({ reason }) => reason === "symlink_not_followed",
|
||||
)) &&
|
||||
coverage.enumeration_omissions.every((omission) => {
|
||||
if (!coverage.enumeration_reasons.includes(omission.reason))
|
||||
return false;
|
||||
const entryType = entryTypes.get(omission.path);
|
||||
if (omission.reason === "symlink_not_followed")
|
||||
return entryType === "symlink" && omission.system_code === null;
|
||||
if (omission.reason === "directory_unavailable")
|
||||
return entryType === "directory" && omission.system_code !== null;
|
||||
return (
|
||||
(entryType === undefined || entryType === "directory") &&
|
||||
(omission.reason === "entry_unavailable"
|
||||
? omission.system_code !== null
|
||||
: omission.system_code === null)
|
||||
);
|
||||
}), `truncation_details.${name}.enumeration_omissions`, "every enumeration omission must identify an unavailable entry, retained directory or unfollowed symlink, with a matching reason and system error only for unavailable reads");
|
||||
require(coverage.hashed_bytes ===
|
||||
files
|
||||
.filter((file) => file.type === "file" && file.sha256 !== null)
|
||||
|
||||
@@ -71,3 +71,32 @@ it("keeps explicit dependencies while excluding unrelated scoped unknowns", () =
|
||||
other.unknown_id,
|
||||
]);
|
||||
});
|
||||
|
||||
it("aggregates constituent unknowns without importing another comparison's scope", () => {
|
||||
const scoped = (comparisonId: string) =>
|
||||
createResidualUnknown(
|
||||
{
|
||||
...base,
|
||||
question: "Unresolved pseudocode",
|
||||
comparison_conclusion_scope: {
|
||||
comparison_evidence_id: comparisonId,
|
||||
dimensions: ["pseudocode"],
|
||||
},
|
||||
relationships: [],
|
||||
},
|
||||
ev("c"),
|
||||
null,
|
||||
);
|
||||
const own = scoped(ev("a"));
|
||||
const foreign = scoped(ev("b"));
|
||||
const heads = new Map([
|
||||
[own.unknown_id, own],
|
||||
[foreign.unknown_id, foreign],
|
||||
]);
|
||||
expect(
|
||||
reconstructionClaimUnknowns(heads, ev("a"), [ev("b")], ["overall"]),
|
||||
).toEqual([own]);
|
||||
expect(
|
||||
reconstructionClaimUnknowns(heads, ev("a"), [ev("b")], ["assembly"]),
|
||||
).toEqual([]);
|
||||
});
|
||||
@@ -23,7 +23,7 @@ export const reconstructionUnknownHeads = (
|
||||
return heads;
|
||||
};
|
||||
|
||||
/** Find active unknowns that affect a comparison or one of its sources. */
|
||||
/** Overall claims include every scoped dimension; individual claims keep their scope. */
|
||||
export const reconstructionClaimUnknowns = (
|
||||
heads: ReadonlyMap<string, ResidualUnknown>,
|
||||
comparisonId: string,
|
||||
@@ -36,6 +36,7 @@ export const reconstructionClaimUnknowns = (
|
||||
scope === undefined ||
|
||||
(scope.comparison_evidence_id === comparisonId &&
|
||||
(dimensions === undefined ||
|
||||
dimensions.includes("overall") ||
|
||||
scope.dimensions.some((dimension) => dimensions.includes(dimension))))
|
||||
);
|
||||
};
|
||||
|
||||
@@ -318,6 +318,95 @@ describe("managed artifact inventory", () => {
|
||||
);
|
||||
});
|
||||
|
||||
describe("managed attribute SerString encoding", () => {
|
||||
it.each([
|
||||
[0, [0x00]],
|
||||
[0x7f, [0x7f]],
|
||||
[0x80, [0x80, 0x80]],
|
||||
[0x3fff, [0xbf, 0xff]],
|
||||
[0x4000, [0xc0, 0x00, 0x40, 0x00]],
|
||||
] as const)(
|
||||
"reads a target-framework SerString of %i UTF-8 bytes",
|
||||
(length, prefix) => {
|
||||
const targetFramework =
|
||||
"é".repeat(Math.floor(length / 2)) + (length % 2 === 0 ? "" : "A");
|
||||
const value = Buffer.concat([
|
||||
Buffer.from([1, 0, ...prefix]),
|
||||
Buffer.from(targetFramework),
|
||||
Buffer.from([0, 0]),
|
||||
]);
|
||||
const bytes = buildManagedPeFixture({ targetFramework });
|
||||
const result = inspectManagedArtifactBytes(
|
||||
bytes,
|
||||
managedPeFixtureTarget(bytes),
|
||||
);
|
||||
|
||||
expect(result.target_frameworks).toEqual([targetFramework]);
|
||||
expect(result.attributes).toMatchObject([
|
||||
{
|
||||
decoded_fixed_string: targetFramework,
|
||||
value_length: value.length,
|
||||
value_sha256: createHash("sha256").update(value).digest("hex"),
|
||||
},
|
||||
]);
|
||||
expect(result.coverage).toEqual({ state: "complete", issues: [] });
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
{ name: "missing length", value: [1, 0], decoded: null },
|
||||
{ name: "null sentinel", value: [1, 0, 0xff], decoded: null },
|
||||
{ name: "truncated two-byte prefix", value: [1, 0, 0x80], decoded: null },
|
||||
{
|
||||
name: "truncated four-byte prefix",
|
||||
value: [1, 0, 0xc0, 0, 0],
|
||||
decoded: null,
|
||||
},
|
||||
{ name: "reserved prefix", value: [1, 0, 0xe0], decoded: null },
|
||||
{ name: "content outside value", value: [1, 0, 5, 0x41], decoded: null },
|
||||
{
|
||||
name: "maximum length without content",
|
||||
value: [1, 0, 0xdf, 0xff, 0xff, 0xff],
|
||||
decoded: null,
|
||||
},
|
||||
{ name: "invalid UTF-8", value: [1, 0, 1, 0xff, 0, 0], decoded: null },
|
||||
{ name: "invalid prolog", value: [0, 0, 1, 0x41, 0, 0], decoded: null },
|
||||
{
|
||||
name: "overlong two-byte length",
|
||||
value: [1, 0, 0x80, 1, 0x41, 0, 0],
|
||||
decoded: "A",
|
||||
},
|
||||
{
|
||||
name: "overlong four-byte length",
|
||||
value: [1, 0, 0xc0, 0, 0, 1, 0x41, 0, 0],
|
||||
decoded: "A",
|
||||
},
|
||||
])("preserves attribute evidence for $name", ({ value, decoded }) => {
|
||||
const customAttributeValue = Buffer.from(value);
|
||||
const bytes = buildManagedPeFixture({ customAttributeValue });
|
||||
const result = inspectManagedArtifactBytes(
|
||||
bytes,
|
||||
managedPeFixtureTarget(bytes),
|
||||
);
|
||||
|
||||
expect(result.attributes).toMatchObject([
|
||||
{
|
||||
parent_token: "0x20000001",
|
||||
constructor_token: "0x0a000001",
|
||||
type_name: "System.Runtime.Versioning.TargetFrameworkAttribute",
|
||||
decoded_fixed_string: decoded,
|
||||
value_length: customAttributeValue.length,
|
||||
value_sha256: createHash("sha256")
|
||||
.update(customAttributeValue)
|
||||
.digest("hex"),
|
||||
},
|
||||
]);
|
||||
expect(result.target_frameworks).toEqual(decoded === null ? [] : [decoded]);
|
||||
expect(result.metadata.status).toBe("complete");
|
||||
expect(result.coverage).toEqual({ state: "complete", issues: [] });
|
||||
});
|
||||
});
|
||||
|
||||
describe("managed artifact coded indexes", () => {
|
||||
it("reports an invalid ManifestResource implementation while retaining the resource", () => {
|
||||
const bytes = buildManagedPeFixture({ resourceImplementationRaw: 9 });
|
||||
|
||||
@@ -940,6 +940,106 @@ describe("managed MethodSpec call targets", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("managed MethodDef heap admission", () => {
|
||||
it("keeps names from unpaged methods without reading their signatures or bodies", () => {
|
||||
const bytes = buildMultiMethodManagedPeFixture({
|
||||
methods: [
|
||||
{ name: "Outside", body: tokenBody([]) },
|
||||
{ name: "Caller", body: tokenBody([[call, 0x06000001]]) },
|
||||
],
|
||||
});
|
||||
const pe = readManagedPeLayout(bytes);
|
||||
if (pe.cli === null) throw new Error("fixture has no CLI header");
|
||||
const { rva, size } = pe.cli.metadata;
|
||||
const layout = readManagedMetadataLayout(
|
||||
bytes,
|
||||
pe.rvaToOffset(rva, size, "metadata"),
|
||||
size,
|
||||
);
|
||||
const row = metadataRowOffset(layout, 6, 1);
|
||||
bytes.writeUInt32LE(0xffffffff, row);
|
||||
const signatureOffset = row + 8 + layout.stringIndexSize;
|
||||
if (layout.blobIndexSize === 2)
|
||||
bytes.writeUInt16LE(0xffff, signatureOffset);
|
||||
else bytes.writeUInt32LE(0xffffffff, signatureOffset);
|
||||
|
||||
const result = inspectManagedMembersBytes(
|
||||
bytes,
|
||||
managedPeFixtureTarget(bytes),
|
||||
{
|
||||
methodOffset: 1,
|
||||
methodLimit: 1,
|
||||
},
|
||||
);
|
||||
|
||||
expect(result.methods).toMatchObject([
|
||||
{ token: "0x06000002", name: "Caller" },
|
||||
]);
|
||||
expect(result.call_edges).toMatchObject([
|
||||
{ target_token: "0x06000001", target_name: "Outside" },
|
||||
]);
|
||||
expect(result.coverage).toEqual({ state: "complete", issues: [] });
|
||||
});
|
||||
|
||||
it.each([
|
||||
{
|
||||
name: "valid instantiation",
|
||||
instantiation: [0x0a, 1, 0x0e],
|
||||
code: "invalid-heap-index",
|
||||
scope: "metadata.#Blob",
|
||||
},
|
||||
{
|
||||
name: "malformed instantiation",
|
||||
instantiation: [0x0a, 0],
|
||||
code: "invalid-blob",
|
||||
scope: "metadata.MethodSpec:0x2b000001",
|
||||
},
|
||||
])(
|
||||
"reads an unpaged MethodDef signature only after a $name",
|
||||
({ instantiation, code, scope }) => {
|
||||
const bytes = buildManagedPeFixture({
|
||||
methodRowCount: 2,
|
||||
methodSignature: genericDef,
|
||||
methodSpecMethods: [2 << 1],
|
||||
methodSpecInstantiation: Buffer.from(instantiation),
|
||||
ilBody: tokenBody([[call, 0x2b000001]]),
|
||||
});
|
||||
const pe = readManagedPeLayout(bytes);
|
||||
if (pe.cli === null) throw new Error("fixture has no CLI header");
|
||||
const { rva, size } = pe.cli.metadata;
|
||||
const layout = readManagedMetadataLayout(
|
||||
bytes,
|
||||
pe.rvaToOffset(rva, size, "metadata"),
|
||||
size,
|
||||
);
|
||||
const signatureOffset =
|
||||
metadataRowOffset(layout, 6, 2) + 8 + layout.stringIndexSize;
|
||||
if (layout.blobIndexSize === 2)
|
||||
bytes.writeUInt16LE(0xffff, signatureOffset);
|
||||
else bytes.writeUInt32LE(0xffffffff, signatureOffset);
|
||||
|
||||
const result = inspectManagedMembersBytes(
|
||||
bytes,
|
||||
managedPeFixtureTarget(bytes),
|
||||
{
|
||||
methodLimit: 1,
|
||||
},
|
||||
);
|
||||
|
||||
expect(result.methods).toMatchObject([
|
||||
{ token: "0x06000001", name: "Main" },
|
||||
]);
|
||||
expect(result.call_edges).toMatchObject([
|
||||
{ target_token: "0x2b000001", target_name: "Main" },
|
||||
]);
|
||||
expect(result.coverage).toMatchObject({
|
||||
state: "partial",
|
||||
issues: [{ code, scope }],
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("managed nested types", () => {
|
||||
const inspectNested = (
|
||||
options: Parameters<typeof buildManagedNestedTypeFixture>[0] = {},
|
||||
@@ -2067,6 +2167,72 @@ it.each([
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
[[], "truncated compressed integer"],
|
||||
[[0x80], "truncated compressed integer"],
|
||||
[[0xc0, 0, 0], "truncated compressed integer"],
|
||||
[[0xe0], "reserved compressed integer"],
|
||||
] as const)(
|
||||
"retains TypeSpec prefix failure %j at its owning row",
|
||||
(prefix, issue) => {
|
||||
const bytes = buildManagedPeFixture({
|
||||
typeSpecSignatures: [Buffer.from([0x12, ...prefix])],
|
||||
memberRefParentRaw: (1 << 3) | 4,
|
||||
});
|
||||
const pe = readManagedPeLayout(bytes);
|
||||
if (pe.cli === null) throw new Error("fixture has no CLI header");
|
||||
const { rva, size } = pe.cli.metadata;
|
||||
const layout = readManagedMetadataLayout(
|
||||
bytes,
|
||||
pe.rvaToOffset(rva, size, "metadata"),
|
||||
size,
|
||||
);
|
||||
const result = inspectManagedMembersBytes(
|
||||
bytes,
|
||||
managedPeFixtureTarget(bytes),
|
||||
);
|
||||
|
||||
expect(result.member_refs[0]?.parent_name).toBeNull();
|
||||
expect(result.coverage.issues).toContainEqual({
|
||||
code: "invalid-blob",
|
||||
scope: "metadata.TypeSpec:0x1b000001",
|
||||
offset: metadataRowOffset(layout, 27, 1),
|
||||
detail: issue,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
[[0x01], -64],
|
||||
[[0x80, 0x01], -8192],
|
||||
[[0xc0, 0x00, 0x00, 0x01], -268435456],
|
||||
] as const)(
|
||||
"preserves compressed signed width for array bound %j",
|
||||
(bound, lower) => {
|
||||
const bytes = buildManagedPeFixture({
|
||||
fieldSignature: Buffer.from([
|
||||
0x06,
|
||||
0x14,
|
||||
0x08,
|
||||
0x01,
|
||||
0x00,
|
||||
0x01,
|
||||
...bound,
|
||||
]),
|
||||
});
|
||||
const result = inspectManagedMembersBytes(
|
||||
bytes,
|
||||
managedPeFixtureTarget(bytes),
|
||||
);
|
||||
|
||||
expect(result.fields[0]?.signature).toMatchObject({
|
||||
parse_status: "decoded",
|
||||
field_type: `i4[${lower}...]`,
|
||||
});
|
||||
expect(result.coverage).toEqual({ state: "complete", issues: [] });
|
||||
},
|
||||
);
|
||||
|
||||
it("decodes generic instantiation member signatures as complete coverage", () => {
|
||||
// System.Runtime.Caching 0x06000001: instance void (Dictionary<string, object>)
|
||||
// shape, with TypeRef/TypeSpec coded tokens kept as encoded.
|
||||
|
||||
@@ -24,6 +24,7 @@ import type {
|
||||
} from "./ManagedMetadataLayout.js";
|
||||
import {
|
||||
metadataRowCursor,
|
||||
readManagedCompressedUnsigned,
|
||||
decodeMetadataCodedIndex,
|
||||
metadataCodedIndexCoordinate,
|
||||
type MetadataCodedIndexCoordinate,
|
||||
@@ -286,36 +287,17 @@ export const readCompressed = (
|
||||
bytes: Buffer,
|
||||
offset: number,
|
||||
): { readonly value: number; readonly next: number } => {
|
||||
const first = bytes[offset];
|
||||
if (first === undefined)
|
||||
const decoded = readManagedCompressedUnsigned(bytes, offset);
|
||||
if (!decoded.ok)
|
||||
throw managedFailure(
|
||||
"invalid-blob",
|
||||
"signature",
|
||||
"truncated compressed integer",
|
||||
decoded.error.kind === "truncated"
|
||||
? "truncated compressed integer"
|
||||
: "reserved compressed integer",
|
||||
offset,
|
||||
);
|
||||
if ((first & 0x80) === 0) return { value: first, next: offset + 1 };
|
||||
const width =
|
||||
(first & 0xc0) === 0x80 ? 2 : (first & 0xe0) === 0xc0 ? 4 : null;
|
||||
if (width !== null) {
|
||||
if (offset > bytes.length - width)
|
||||
throw managedFailure(
|
||||
"invalid-blob",
|
||||
"signature",
|
||||
"truncated compressed integer",
|
||||
offset,
|
||||
);
|
||||
let value = first & (width === 2 ? 0x3f : 0x1f);
|
||||
for (let index = 1; index < width; index += 1)
|
||||
value = value * 256 + bytes[offset + index]!;
|
||||
return { value, next: offset + width };
|
||||
}
|
||||
throw managedFailure(
|
||||
"invalid-blob",
|
||||
"signature",
|
||||
"reserved compressed integer",
|
||||
offset,
|
||||
);
|
||||
return { value: decoded.value.value, next: offset + decoded.value.width };
|
||||
};
|
||||
|
||||
const ELEMENT_TYPES = new Map<number, string>([
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { readMethodDefinitionRow } from "./ManagedMethodDefinitionRows.js";
|
||||
import { jsonStringByteLength } from "../domain/jsonStringByteLength.js";
|
||||
import { isManagedMemberReferenceInstruction } from "../domain/managed/managedMemberInstructionFacts.js";
|
||||
import {
|
||||
@@ -361,17 +362,21 @@ export const parseMethods = ({
|
||||
declared?.fullName === null);
|
||||
const produced = methods.length;
|
||||
readManagedValue(() => {
|
||||
const cursor = metadataRowCursor(bytes, layout, 6, row);
|
||||
const rva = cursor.readUInt32();
|
||||
const implFlags = cursor.readUInt16();
|
||||
const flags = cursor.readUInt16();
|
||||
const {
|
||||
token,
|
||||
rowOffset,
|
||||
rva,
|
||||
implFlags,
|
||||
flags,
|
||||
nameIndex,
|
||||
signatureIndex,
|
||||
} = readMethodDefinitionRow(bytes, layout, row);
|
||||
const name = readMetadataString(
|
||||
bytes,
|
||||
layout,
|
||||
cursor.readIndex(layout.stringIndexSize),
|
||||
nameIndex,
|
||||
layout.strings.size,
|
||||
);
|
||||
const token = metadataToken(6, row);
|
||||
const member = {
|
||||
token,
|
||||
name,
|
||||
@@ -384,10 +389,9 @@ export const parseMethods = ({
|
||||
const sig = readMetadataBlob(
|
||||
bytes,
|
||||
layout,
|
||||
cursor.readIndex(layout.blobIndexSize),
|
||||
signatureIndex,
|
||||
layout.blob?.size ?? 0,
|
||||
);
|
||||
cursor.readIndex(layout.tableIndexSize(8));
|
||||
const decoded = budgetedSignature(
|
||||
sig,
|
||||
layout,
|
||||
@@ -416,7 +420,7 @@ export const parseMethods = ({
|
||||
if (!read.scanned) unscannedMethods += 1;
|
||||
methods.push({
|
||||
token,
|
||||
row_offset: cursor.start,
|
||||
row_offset: rowOffset,
|
||||
declaring_type_token: declared?.token ?? null,
|
||||
declaring_type: declared?.fullName ?? null,
|
||||
name,
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import type { ManagedParseIssue } from "../domain/managed/managedArtifact.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
import { managedDecodeBudget } from "./ManagedDecodeBudget.js";
|
||||
|
||||
@@ -55,6 +56,39 @@ export class MetadataRowCursor {
|
||||
}
|
||||
}
|
||||
|
||||
interface ManagedCompressedUnsigned {
|
||||
readonly value: number;
|
||||
readonly width: 1 | 2 | 4;
|
||||
readonly minimal: boolean;
|
||||
}
|
||||
|
||||
type ManagedCompressedUnsignedFailure =
|
||||
| { readonly kind: "truncated"; readonly width: 1 | 2 | 4 }
|
||||
| { readonly kind: "reserved" };
|
||||
|
||||
/** ECMA-335 unsigned encoding facts, without heap or signature admission policy. */
|
||||
export const readManagedCompressedUnsigned = (
|
||||
bytes: Buffer,
|
||||
offset: number,
|
||||
): Result<ManagedCompressedUnsigned, ManagedCompressedUnsignedFailure> => {
|
||||
const first = bytes[offset];
|
||||
if (first === undefined) return err({ kind: "truncated", width: 1 });
|
||||
if ((first & 0x80) === 0)
|
||||
return ok({ value: first, width: 1, minimal: true });
|
||||
const width =
|
||||
(first & 0xc0) === 0x80 ? 2 : (first & 0xe0) === 0xc0 ? 4 : null;
|
||||
if (width === null) return err({ kind: "reserved" });
|
||||
if (offset > bytes.length - width) return err({ kind: "truncated", width });
|
||||
const value =
|
||||
width === 2
|
||||
? ((first & 0x3f) << 8) | (bytes[offset + 1] ?? 0)
|
||||
: (first & 0x1f) * 0x01_00_00_00 +
|
||||
(bytes[offset + 1] ?? 0) * 0x01_00_00 +
|
||||
(bytes[offset + 2] ?? 0) * 0x0100 +
|
||||
(bytes[offset + 3] ?? 0);
|
||||
return ok({ value, width, minimal: value >= (width === 2 ? 0x80 : 0x4000) });
|
||||
};
|
||||
|
||||
// Issue code and label for the heap item a compressed length prefixes.
|
||||
const PREFIXED_ITEMS = {
|
||||
blob: { code: "invalid-blob", label: "Metadata blob" },
|
||||
@@ -73,38 +107,21 @@ const compressedLength = (
|
||||
readonly minimal: boolean;
|
||||
} => {
|
||||
const { code, label } = PREFIXED_ITEMS[item];
|
||||
const requirePrefix = (size: number): void => {
|
||||
if (offset < 0 || offset > bytes.length - size)
|
||||
throw managedFailure(
|
||||
code,
|
||||
scope,
|
||||
`${label} length prefix leaves the artifact byte range`,
|
||||
offset >= 0 ? offset : null,
|
||||
);
|
||||
const decoded = readManagedCompressedUnsigned(bytes, offset);
|
||||
if (!decoded.ok)
|
||||
throw managedFailure(
|
||||
code,
|
||||
scope,
|
||||
decoded.error.kind === "truncated"
|
||||
? `${label} length prefix leaves the artifact byte range`
|
||||
: `${label} uses a reserved compressed-length prefix`,
|
||||
offset >= 0 ? offset : null,
|
||||
);
|
||||
return {
|
||||
length: decoded.value.value,
|
||||
prefix: decoded.value.width,
|
||||
minimal: decoded.value.minimal,
|
||||
};
|
||||
requirePrefix(1);
|
||||
const first = bytes[offset] ?? 0;
|
||||
if ((first & 0x80) === 0) return { length: first, prefix: 1, minimal: true };
|
||||
if ((first & 0xc0) === 0x80) {
|
||||
requirePrefix(2);
|
||||
const length = ((first & 0x3f) << 8) | (bytes[offset + 1] ?? 0);
|
||||
return { length, prefix: 2, minimal: length >= 0x80 };
|
||||
}
|
||||
if ((first & 0xe0) === 0xc0) {
|
||||
requirePrefix(4);
|
||||
const length =
|
||||
(first & 0x1f) * 0x01_00_00_00 +
|
||||
(bytes[offset + 1] ?? 0) * 0x01_00_00 +
|
||||
(bytes[offset + 2] ?? 0) * 0x0100 +
|
||||
(bytes[offset + 3] ?? 0);
|
||||
return { length, prefix: 4, minimal: length >= 0x4000 };
|
||||
}
|
||||
throw managedFailure(
|
||||
code,
|
||||
scope,
|
||||
`${label} uses a reserved compressed-length prefix`,
|
||||
offset,
|
||||
);
|
||||
};
|
||||
|
||||
/** Read one UTF-8 value from #Strings within its PE/CLI stream extent. */
|
||||
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
} from "./ManagedMetadataLayout.js";
|
||||
import {
|
||||
metadataRowCursor,
|
||||
readManagedCompressedUnsigned,
|
||||
decodeMetadataCodedIndex,
|
||||
type ManagedCodedIndex,
|
||||
metadataToken,
|
||||
@@ -432,52 +433,24 @@ const attributeTypeName = (
|
||||
return null;
|
||||
};
|
||||
|
||||
const decodeCompressedLength = (
|
||||
blob: Buffer,
|
||||
offset: number,
|
||||
): { readonly length: number; readonly prefix: number } | undefined => {
|
||||
const first = blob[offset];
|
||||
if (first === undefined) return undefined;
|
||||
if ((first & 0x80) === 0) return { length: first, prefix: 1 };
|
||||
if ((first & 0xc0) === 0x80 && blob[offset + 1] !== undefined)
|
||||
return {
|
||||
length: ((first & 0x3f) << 8) | (blob[offset + 1] ?? 0),
|
||||
prefix: 2,
|
||||
};
|
||||
if (
|
||||
(first & 0xe0) === 0xc0 &&
|
||||
blob[offset + 1] !== undefined &&
|
||||
blob[offset + 2] !== undefined &&
|
||||
blob[offset + 3] !== undefined
|
||||
)
|
||||
return {
|
||||
length:
|
||||
(first & 0x1f) * 0x01_00_00_00 +
|
||||
(blob[offset + 1] ?? 0) * 0x01_00_00 +
|
||||
(blob[offset + 2] ?? 0) * 0x0100 +
|
||||
(blob[offset + 3] ?? 0),
|
||||
prefix: 4,
|
||||
};
|
||||
return undefined;
|
||||
};
|
||||
|
||||
const decodeFixedString = (
|
||||
blob: Buffer,
|
||||
layout: ManagedMetadataLayout,
|
||||
): string | null => {
|
||||
if (blob.length < 3 || blob.readUInt16LE(0) !== 1 || blob[2] === 0xff)
|
||||
return null;
|
||||
const length = decodeCompressedLength(blob, 2);
|
||||
if (length === undefined) return null;
|
||||
const start = 2 + length.prefix;
|
||||
if (start > blob.length - length.length) return null;
|
||||
const decoded = readManagedCompressedUnsigned(blob, 2);
|
||||
if (!decoded.ok) return null;
|
||||
const { value: length, width } = decoded.value;
|
||||
const start = 2 + width;
|
||||
if (start > blob.length - length) return null;
|
||||
managedDecodeBudget(layout).reserveString(
|
||||
length.length,
|
||||
length,
|
||||
layout.blob?.offset ?? layout.rootOffset,
|
||||
);
|
||||
try {
|
||||
return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(
|
||||
blob.subarray(start, start + length.length),
|
||||
blob.subarray(start, start + length),
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
// Invalid UTF-8 blobs decode as null.
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
import type { ManagedMetadataLayout } from "./ManagedMetadataLayout.js";
|
||||
import { metadataRowCursor, metadataToken } from "./ManagedMetadataHeaps.js";
|
||||
|
||||
/** Scalar MethodDef facts; consumers own heap admission and method selection. */
|
||||
export const readMethodDefinitionRow = (
|
||||
bytes: Buffer,
|
||||
layout: ManagedMetadataLayout,
|
||||
row: number,
|
||||
): {
|
||||
readonly token: string;
|
||||
readonly rowOffset: number;
|
||||
readonly rva: number;
|
||||
readonly implFlags: number;
|
||||
readonly flags: number;
|
||||
readonly nameIndex: number;
|
||||
readonly signatureIndex: number;
|
||||
} => {
|
||||
// Layout admission bounds the whole fixed-width row before a cursor is created.
|
||||
const cursor = metadataRowCursor(bytes, layout, 6, row);
|
||||
return {
|
||||
token: metadataToken(6, row),
|
||||
rowOffset: cursor.start,
|
||||
rva: cursor.readUInt32(),
|
||||
implFlags: cursor.readUInt16(),
|
||||
flags: cursor.readUInt16(),
|
||||
nameIndex: cursor.readIndex(layout.stringIndexSize),
|
||||
signatureIndex: cursor.readIndex(layout.blobIndexSize),
|
||||
};
|
||||
};
|
||||
@@ -1,3 +1,4 @@
|
||||
import { readMethodDefinitionRow } from "./ManagedMethodDefinitionRows.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import type { ManagedParseIssue } from "../domain/managed/managedArtifact.js";
|
||||
import {
|
||||
@@ -157,22 +158,20 @@ const methodSignatureBlob = (
|
||||
token: string,
|
||||
): Buffer => {
|
||||
const table = Number.parseInt(token.slice(2, 4), 16);
|
||||
const cursor = metadataRowCursor(
|
||||
bytes,
|
||||
layout,
|
||||
table,
|
||||
Number.parseInt(token.slice(4), 16),
|
||||
);
|
||||
const row = Number.parseInt(token.slice(4), 16);
|
||||
let signatureIndex: number;
|
||||
if (table === 6) {
|
||||
cursor.readUInt32();
|
||||
cursor.readUInt16();
|
||||
cursor.readUInt16();
|
||||
} else cursor.readIndex(layout.codedIndexSize("MemberRefParent"));
|
||||
cursor.readIndex(layout.stringIndexSize);
|
||||
signatureIndex = readMethodDefinitionRow(bytes, layout, row).signatureIndex;
|
||||
} else {
|
||||
const cursor = metadataRowCursor(bytes, layout, table, row);
|
||||
cursor.readIndex(layout.codedIndexSize("MemberRefParent"));
|
||||
cursor.readIndex(layout.stringIndexSize);
|
||||
signatureIndex = cursor.readIndex(layout.blobIndexSize);
|
||||
}
|
||||
return readMetadataBlob(
|
||||
bytes,
|
||||
layout,
|
||||
cursor.readIndex(layout.blobIndexSize),
|
||||
signatureIndex,
|
||||
layout.blob?.size ?? 0,
|
||||
);
|
||||
};
|
||||
@@ -1,3 +1,4 @@
|
||||
import { readMethodDefinitionRow } from "./ManagedMethodDefinitionRows.js";
|
||||
import {
|
||||
admitManagedProjection,
|
||||
managedDecodeBudget,
|
||||
@@ -113,22 +114,12 @@ export const parseMethods = (
|
||||
const table = layout.table(6);
|
||||
for (let row = 1; row <= (table?.rowCount ?? 0); row += 1) {
|
||||
readManagedValue(() => {
|
||||
const cursor = metadataRowCursor(bytes, layout, 6, row);
|
||||
const rva = cursor.readUInt32();
|
||||
const implFlags = cursor.readUInt16();
|
||||
const flags = cursor.readUInt16();
|
||||
const name = readMetadataString(
|
||||
bytes,
|
||||
layout,
|
||||
cursor.readIndex(layout.stringIndexSize),
|
||||
heapExtent,
|
||||
);
|
||||
cursor.readIndex(layout.blobIndexSize);
|
||||
cursor.readIndex(layout.tableIndexSize(8));
|
||||
const token = metadataToken(6, row);
|
||||
const { token, rowOffset, rva, implFlags, flags, nameIndex } =
|
||||
readMethodDefinitionRow(bytes, layout, row);
|
||||
const name = readMetadataString(bytes, layout, nameIndex, heapExtent);
|
||||
methods.set(token, {
|
||||
token,
|
||||
rowOffset: cursor.start,
|
||||
rowOffset,
|
||||
kind: "method",
|
||||
name,
|
||||
flags,
|
||||
|
||||
@@ -8,6 +8,7 @@ interface ManagedPeFixtureOptions {
|
||||
readonly cliFlags?: number;
|
||||
readonly corruptMetadataSignature?: boolean;
|
||||
readonly customAttributeTypeRaw?: number;
|
||||
readonly customAttributeValue?: Buffer;
|
||||
readonly extendsRaw?: number;
|
||||
readonly exportedTypes?: readonly {
|
||||
readonly namespace: string;
|
||||
@@ -100,6 +101,18 @@ class StringHeap {
|
||||
}
|
||||
}
|
||||
|
||||
const compressedUnsigned = (value: number): Buffer =>
|
||||
value <= 0x7f
|
||||
? Buffer.from([value])
|
||||
: value <= 0x3fff
|
||||
? Buffer.from([0x80 | (value >> 8), value & 0xff])
|
||||
: Buffer.from([
|
||||
0xc0 | (value >> 24),
|
||||
(value >> 16) & 0xff,
|
||||
(value >> 8) & 0xff,
|
||||
value & 0xff,
|
||||
]);
|
||||
|
||||
class BlobHeap {
|
||||
readonly #chunks: Buffer[] = [Buffer.from([0])];
|
||||
#size = 1;
|
||||
@@ -111,17 +124,7 @@ class BlobHeap {
|
||||
add(value: Buffer): number {
|
||||
if (value.length === 0) return 0;
|
||||
const index = this.#size;
|
||||
const prefix =
|
||||
value.length <= 0x7f
|
||||
? Buffer.from([value.length])
|
||||
: value.length <= 0x3fff
|
||||
? Buffer.from([0x80 | (value.length >> 8), value.length & 0xff])
|
||||
: Buffer.from([
|
||||
0xc0 | (value.length >> 24),
|
||||
(value.length >> 16) & 0xff,
|
||||
(value.length >> 8) & 0xff,
|
||||
value.length & 0xff,
|
||||
]);
|
||||
const prefix = compressedUnsigned(value.length);
|
||||
this.#chunks.push(prefix, value);
|
||||
this.#size += value.length + prefix.length;
|
||||
return index;
|
||||
@@ -428,9 +431,12 @@ const exportedTypeRow = (
|
||||
|
||||
const fixedStringAttributeBlob = (value: string): Buffer => {
|
||||
const bytes = Buffer.from(textEncoder.encode(value));
|
||||
if (bytes.length > 0x7f)
|
||||
throw new RangeError("Managed PE fixture target framework is too long");
|
||||
return Buffer.concat([Buffer.from([1, 0, bytes.length]), bytes, u16(0)]);
|
||||
return Buffer.concat([
|
||||
Buffer.from([1, 0]),
|
||||
compressedUnsigned(bytes.length),
|
||||
bytes,
|
||||
u16(0),
|
||||
]);
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -1061,11 +1067,12 @@ const buildManagedFixtureMetadata = (
|
||||
);
|
||||
const referenceBlobs = referenceBlobIndexes(blobs, options);
|
||||
const attributeBlob = blobs.add(
|
||||
fixedStringAttributeBlob(
|
||||
options.targetFramework === null
|
||||
? ".NETCoreApp,Version=v8.0"
|
||||
: (options.targetFramework ?? ".NETCoreApp,Version=v8.0"),
|
||||
),
|
||||
options.customAttributeValue ??
|
||||
fixedStringAttributeBlob(
|
||||
options.targetFramework === null
|
||||
? ".NETCoreApp,Version=v8.0"
|
||||
: (options.targetFramework ?? ".NETCoreApp,Version=v8.0"),
|
||||
),
|
||||
);
|
||||
const stringSize = indexSize(strings.size);
|
||||
const blobSize = indexSize(blobs.size);
|
||||
|
||||
@@ -1,11 +1,6 @@
|
||||
import type { Il2cppMethodAddress } from "../domain/il2cpp/il2cppMetadata.js";
|
||||
import { Il2cppBinaryFormatFailure } from "./Il2cppBinaryFormat.js";
|
||||
import {
|
||||
openImageSteps,
|
||||
readPointer,
|
||||
readSlot,
|
||||
isExecutable,
|
||||
} from "./Il2cppBinaryImage.js";
|
||||
import { openImageSteps, readSlot, isExecutable } from "./Il2cppBinaryImage.js";
|
||||
import {
|
||||
findModules,
|
||||
findCodeRegistration,
|
||||
@@ -180,8 +175,13 @@ export function* linkIl2cppBinarySteps(
|
||||
);
|
||||
// 27.0/27.1 and 27.2 share header version 27 but place byref differently.
|
||||
if (version === 27) return { type, name: null };
|
||||
const address =
|
||||
readPointer(image, registration.types + BigInt(type * 8)) ?? 0n;
|
||||
const address = readSlot(image, registration.types + BigInt(type * 8));
|
||||
if (address === "unknown") return { type, name: null };
|
||||
if (address === null)
|
||||
throw new Il2cppBinaryFormatFailure(
|
||||
"invalid-type",
|
||||
`type ${type} has no mapped type-table slot`,
|
||||
);
|
||||
if (address === 0n)
|
||||
throw new Il2cppBinaryFormatFailure(
|
||||
"invalid-type",
|
||||
|
||||
@@ -76,7 +76,7 @@ export interface NativeImage {
|
||||
readonly imageBase: bigint;
|
||||
readonly unresolvedPointerSlots: ReadonlyMap<bigint, number>;
|
||||
readonly zeroFillPointers: ReadonlyMap<bigint, bigint>;
|
||||
/** Mach-O loader fixups, owned by every pointer interpretation. */
|
||||
/** Mach-O loader fixups shared by every native pointer consumer. */
|
||||
readonly machoPointerFixups?: PointerFixups;
|
||||
}
|
||||
|
||||
@@ -191,10 +191,8 @@ export function* openImageSteps(
|
||||
};
|
||||
}
|
||||
|
||||
const readRawPointer = (image: NativeImage, address: bigint): bigint | null => {
|
||||
if (hasUnresolvedPointer(image, address)) return null;
|
||||
const virtual = image.zeroFillPointers.get(address);
|
||||
if (virtual !== undefined) return virtual;
|
||||
/** Stored bytes only; loader interpretation belongs to readSlot. */
|
||||
const readStoredWord = (image: NativeImage, address: bigint): bigint | null => {
|
||||
try {
|
||||
const offset = mappedFileOffset(
|
||||
image.container.mappings,
|
||||
@@ -213,8 +211,12 @@ const readRawPointer = (image: NativeImage, address: bigint): bigint | null => {
|
||||
export const readUInt32 = (
|
||||
image: NativeImage,
|
||||
address: bigint,
|
||||
): number | null => {
|
||||
if (hasUnresolvedPointer(image, address, 4)) return null;
|
||||
): number | "unknown" | null => {
|
||||
if (
|
||||
hasUnresolvedElfBytes(image, address, 4) ||
|
||||
image.machoPointerFixups?.hasUnresolvedLegacyBytes(address, 4) === true
|
||||
)
|
||||
return "unknown";
|
||||
const virtual = image.zeroFillPointers.get(address);
|
||||
if (virtual !== undefined) return Number(virtual & 0xffffffffn);
|
||||
try {
|
||||
@@ -326,7 +328,7 @@ export const isExecutable = (image: NativeImage, address: bigint): boolean => {
|
||||
}
|
||||
};
|
||||
|
||||
const hasUnresolvedPointer = (
|
||||
const hasUnresolvedElfBytes = (
|
||||
image: NativeImage,
|
||||
address: bigint,
|
||||
size = 8,
|
||||
@@ -343,13 +345,13 @@ export const readSlot = (
|
||||
image: NativeImage,
|
||||
address: bigint,
|
||||
): bigint | "unknown" | null => {
|
||||
if (hasUnresolvedPointer(image, address)) return "unknown";
|
||||
if (hasUnresolvedElfBytes(image, address)) return "unknown";
|
||||
try {
|
||||
const virtual = image.zeroFillPointers.get(address);
|
||||
let raw: bigint | null;
|
||||
if (virtual !== undefined) raw = virtual;
|
||||
else if (isZeroFilled(image, address, 8)) raw = 0n;
|
||||
else raw = readRawPointer(image, address);
|
||||
else raw = readStoredWord(image, address);
|
||||
if (raw === null) return null;
|
||||
if (image.machoPointerFixups !== undefined) {
|
||||
const resolution = image.machoPointerFixups.decode(address, raw);
|
||||
@@ -370,11 +372,11 @@ export const readSlot = (
|
||||
}
|
||||
};
|
||||
|
||||
/** Resolve a pointer through the same loader interpretation as method slots. */
|
||||
/** Require an observed resolved pointer; unknown and unmapped slots stay null. */
|
||||
export const readPointer = (
|
||||
image: NativeImage,
|
||||
address: bigint,
|
||||
): bigint | null => {
|
||||
const slot = readSlot(image, address);
|
||||
return typeof slot === "bigint" ? slot : null;
|
||||
const value = readSlot(image, address);
|
||||
return typeof value === "bigint" ? value : null;
|
||||
};
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
isMapped,
|
||||
hasPointerArrayCoverage,
|
||||
readPointer,
|
||||
readUInt32,
|
||||
type NativeImage,
|
||||
} from "./Il2cppBinaryImage.js";
|
||||
import type { Il2cppMetadataDecoded } from "./Il2cppMetadataFile.js";
|
||||
@@ -136,6 +137,8 @@ export function* findModules(
|
||||
const indices = names.get(target);
|
||||
if (indices === undefined) continue;
|
||||
const address = mapping.address + BigInt(at);
|
||||
// Matching stored bytes nominate a candidate, not a resolved pointer.
|
||||
if (readPointer(image, address) !== target) continue;
|
||||
let recordOffset: number;
|
||||
try {
|
||||
recordOffset = mappedFileOffset(
|
||||
@@ -160,9 +163,10 @@ export function* findModules(
|
||||
let work = 0;
|
||||
for (const index of indices) {
|
||||
if (++work % 256 === 0) yield;
|
||||
const count = image.bytes.readUInt32LE(offset + 8);
|
||||
const count = readUInt32(image, address + 8n);
|
||||
if (count !== counts[index]) continue;
|
||||
const pointers = image.bytes.readBigUInt64LE(offset + 16);
|
||||
const pointers = readPointer(image, address + 16n);
|
||||
if (pointers === null) continue;
|
||||
if (!hasPointerArrayCoverage(image, pointers, count)) continue;
|
||||
const list = candidates.get(index) ?? [];
|
||||
if (list.some((candidate) => candidate.address === address)) continue;
|
||||
@@ -207,7 +211,6 @@ export function* findCodeRegistration(
|
||||
modules: readonly Il2cppCodeGenModule[],
|
||||
): Generator<void, { readonly field: bigint | null; readonly array: bigint }> {
|
||||
const expected = new Set(modules.map((module) => module.address));
|
||||
const total = modules.length;
|
||||
const matches: { field: bigint; array: bigint }[] = [];
|
||||
const matchedFields = new Set<bigint>();
|
||||
for (const mapping of image.container.mappings) {
|
||||
@@ -215,10 +218,13 @@ export function* findCodeRegistration(
|
||||
for (let at = start; at + 16 <= mapping.size; at += 8) {
|
||||
if ((at & 0xffff) < 8) yield;
|
||||
const offset = mapping.offset + at;
|
||||
if (image.bytes.readUInt32LE(offset) !== total) continue;
|
||||
const array = readPointer(image, mapping.address + BigInt(at + 8));
|
||||
if (array === null) continue;
|
||||
if (!isMapped(image, array, modules.length * 8)) continue;
|
||||
// The count owns four bytes; the following four only align its pointer.
|
||||
if (image.bytes.readUInt32LE(offset) !== modules.length) continue;
|
||||
const field = mapping.address + BigInt(at);
|
||||
if (readUInt32(image, field) !== modules.length) continue;
|
||||
const array = readPointer(image, field + 8n);
|
||||
if (array === null || !isMapped(image, array, modules.length * 8))
|
||||
continue;
|
||||
const seen = new Set<bigint>();
|
||||
for (let slot = 0; slot < modules.length; slot++) {
|
||||
if (slot % 256 === 0) yield;
|
||||
@@ -227,7 +233,6 @@ export function* findCodeRegistration(
|
||||
seen.add(pointer);
|
||||
}
|
||||
if (seen.size === modules.length) {
|
||||
const field = mapping.address + BigInt(at);
|
||||
// Exact load aliases can visit one virtual registration field twice.
|
||||
if (matchedFields.has(field)) continue;
|
||||
matchedFields.add(field);
|
||||
@@ -273,8 +278,11 @@ function* findModuleArrayByCode(
|
||||
const lastPosition = new Map<bigint, number>();
|
||||
for (let at = start; at + 8 <= mapping.size; at += 8) {
|
||||
if ((at & 0xffff) < 8) yield;
|
||||
const value = image.bytes.readBigUInt64LE(mapping.offset + at);
|
||||
if (!expected.has(value)) {
|
||||
const raw = image.bytes.readBigUInt64LE(mapping.offset + at);
|
||||
const value = expected.has(raw)
|
||||
? readPointer(image, mapping.address + BigInt(at))
|
||||
: null;
|
||||
if (value === null || !expected.has(value)) {
|
||||
runStart = at + 8;
|
||||
lastPosition.clear();
|
||||
continue;
|
||||
@@ -336,18 +344,23 @@ export function* findMetadataRegistration(
|
||||
const low = (offset: number) => image.bytes.readUInt32LE(offset);
|
||||
for (const mapping of image.container.mappings) {
|
||||
const start = Number((8n - (mapping.address % 8n)) % 8n);
|
||||
// The fields used below end at byte 112. The full 128-byte registration
|
||||
// must still have loader coverage, including any declared zero-fill tail.
|
||||
for (let at = start; at + 112 <= mapping.size; at += 8) {
|
||||
if ((at & 0xffff) < 8) yield;
|
||||
const base = mapping.offset + at;
|
||||
if (low(base + 80) !== typeDefinitions) continue;
|
||||
if (low(base + 96) !== typeDefinitions) continue;
|
||||
const count = low(base + 48);
|
||||
const address = mapping.address + BigInt(at);
|
||||
if (!hasPointerArrayCoverage(image, address, 16)) continue;
|
||||
if (readUInt32(image, address + 80n) !== typeDefinitions) continue;
|
||||
if (readUInt32(image, address + 96n) !== typeDefinitions) continue;
|
||||
const count = readUInt32(image, address + 48n);
|
||||
const types = readPointer(image, address + 56n);
|
||||
const fieldOffsets = readPointer(image, address + 88n);
|
||||
const typeSizes = readPointer(image, address + 104n);
|
||||
if (
|
||||
typeof count !== "number" ||
|
||||
count === 0 ||
|
||||
count > 0x7fffffff ||
|
||||
types === null ||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { Il2cppBinaryFormatFailure } from "./Il2cppBinaryFormat.js";
|
||||
import {
|
||||
readPointer,
|
||||
readSlot,
|
||||
readUInt32,
|
||||
isMapped,
|
||||
type NativeImage,
|
||||
@@ -46,12 +46,26 @@ export const createTypeNamer = (
|
||||
const fail = (message: string): never => {
|
||||
throw new Il2cppBinaryFormatFailure("invalid-type", message);
|
||||
};
|
||||
const pointer = (address: bigint, label: string) =>
|
||||
readPointer(image, address) ??
|
||||
fail(`${label} at 0x${address.toString(16)} is not file mapped`);
|
||||
const u32 = (address: bigint, label: string) =>
|
||||
readUInt32(image, address) ??
|
||||
fail(`${label} at 0x${address.toString(16)} is not file mapped`);
|
||||
const unresolved = (address: bigint, label: string): never => {
|
||||
throw new Il2cppBinaryFormatFailure(
|
||||
"unsupported-pointer-encoding",
|
||||
`${label} at 0x${address.toString(16)} is loader-unresolved; its stored bytes are not a resolved type value`,
|
||||
);
|
||||
};
|
||||
const pointer = (address: bigint, label: string): bigint => {
|
||||
const value = readSlot(image, address);
|
||||
if (value === "unknown") return unresolved(address, label);
|
||||
return (
|
||||
value ?? fail(`${label} at 0x${address.toString(16)} is not file mapped`)
|
||||
);
|
||||
};
|
||||
const u32 = (address: bigint, label: string): number => {
|
||||
const value = readUInt32(image, address);
|
||||
if (value === "unknown") return unresolved(address, label);
|
||||
return (
|
||||
value ?? fail(`${label} at 0x${address.toString(16)} is not file mapped`)
|
||||
);
|
||||
};
|
||||
let cachedBytes = 0;
|
||||
let work = 0;
|
||||
const guard = (bytes: number) => {
|
||||
|
||||
@@ -531,7 +531,7 @@ export class Il2cppMetadataProvider implements Il2cppMetadataPort {
|
||||
? "Field, parameter and return types are not recovered without binary_path; type indices refer to the native binary."
|
||||
: decoded.metadata_version === 27
|
||||
? "Field, parameter and return type names are null for metadata version 27: versions 27.0 to 27.2 share that header number but place the Il2CppType byref flag differently, so a name could gain or lose & without warning; type indices are still reported."
|
||||
: "Field, parameter and return type names are rendered from the binary's Il2CppType table in a C#-like form (keywords for primitives, Namespace.Outer+Inner, generic parameters by metadata name); nested generic arguments stay with the type component that declares them. Custom modifiers, array bounds and attributes are not reported.",
|
||||
: "Field, parameter and return type names are rendered from the binary's Il2CppType table in a C#-like form (keywords for primitives, Namespace.Outer+Inner, generic parameters by metadata name); nested generic arguments stay with the type component that declares them. An unresolved type-table slot retains its type index with a null name; a loader-unresolved value inside the type graph is refused rather than rendered from stored placeholder bytes. Custom modifiers, array bounds and attributes are not reported.",
|
||||
...(linked.binary !== null &&
|
||||
linked.binary.code_gen_modules_field === null
|
||||
? [
|
||||
|
||||
@@ -91,7 +91,6 @@ it("keeps ceiling dial recoveries and scrubs non-adjustable timeout teaching", a
|
||||
const files = [
|
||||
"docs/user-visible-errors.csv",
|
||||
"docs/mcp-contracts.md",
|
||||
"src/server/createServer.ts",
|
||||
"src/domain/analysisErrorPresentation.ts",
|
||||
".agents/skills/reverse-engineer-anything/SKILL.md",
|
||||
".agents/skills/reverse-engineer-anything/references/connection-and-recovery.md",
|
||||
@@ -128,17 +127,6 @@ it("keeps ceiling dial recoveries and scrubs non-adjustable timeout teaching", a
|
||||
expect(skillProse).toContain("explicitly recommends that change");
|
||||
expect(skillProse).toContain("not a recommendation to change it");
|
||||
expect(skillProse).not.toContain("names a request field");
|
||||
const instructions = prose(
|
||||
await readFile(join(root, "src/server/createServer.ts"), "utf8"),
|
||||
);
|
||||
expect(instructions).toContain("once only");
|
||||
expect(instructions).toContain("safe to repeat");
|
||||
expect(instructions).toContain("tell the user");
|
||||
expect(instructions).toContain("restart the MCP client");
|
||||
expect(instructions).toContain("explicitly recommends that change");
|
||||
expect(instructions).toContain("not a recommendation to change it");
|
||||
expect(instructions).not.toContain("names a request field");
|
||||
|
||||
const wakaru = await readFile(
|
||||
join(root, "src/javascript/recovery/RecoveryFiles.ts"),
|
||||
"utf8",
|
||||
|
||||
@@ -187,6 +187,7 @@ it("marks only opcode-selected pointer slots rebased, including target zero", ()
|
||||
kind: "rebase",
|
||||
target: BASE + 0x200n,
|
||||
});
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x100n, 4)).toBe(false);
|
||||
expect(fixups.decode(BASE + 0x118n, 0n)).toEqual({
|
||||
kind: "rebase",
|
||||
target: 0n,
|
||||
@@ -252,6 +253,7 @@ it("retains known prefix rebases and leaves unmatched locations unknown on an in
|
||||
expect(fixups.issues).toMatchObject([
|
||||
{ code: "rebase_stream_decode_failed" },
|
||||
]);
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x108n, 4)).toBe(true);
|
||||
});
|
||||
|
||||
it("does not treat 32-bit text rebases as 64-bit metadata pointers", () => {
|
||||
@@ -263,6 +265,10 @@ it("does not treat 32-bit text rebases as 64-bit metadata pointers", () => {
|
||||
kind: "unsupported",
|
||||
reason: "Legacy non-pointer rebase is not a 64-bit metadata pointer",
|
||||
});
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x100n, 4)).toBe(true);
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x103n, 4)).toBe(true);
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x104n, 4)).toBe(false);
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x108n, 4)).toBe(false);
|
||||
expect(fixups.decode(BASE + 0x108n, BASE + 0x300n)).toMatchObject({
|
||||
kind: "raw",
|
||||
target: BASE + 0x300n,
|
||||
@@ -438,6 +444,7 @@ describe("incomplete legacy bind applicability", () => {
|
||||
expect(fixups.issues).toMatchObject([
|
||||
{ code: "bind_stream_decode_failed" },
|
||||
]);
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x1008n, 4)).toBe(true);
|
||||
});
|
||||
|
||||
it("retains a positive bind prefix while leaving unmatched slots unknown", () => {
|
||||
@@ -1285,6 +1292,10 @@ describe("legacy pointer fixup streams", () => {
|
||||
kind: "raw",
|
||||
target: 0x99n,
|
||||
});
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x4014n, 4)).toBe(true);
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x4018n, 4)).toBe(false);
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x401cn, 4)).toBe(false);
|
||||
expect(fixups.hasUnresolvedLegacyBytes(BASE + 0x401cn, 8)).toBe(true);
|
||||
});
|
||||
|
||||
it("wraps a negative DO_BIND_ADD_ADDR_ULEB step and keeps decoding", () => {
|
||||
|
||||
@@ -65,6 +65,10 @@ export interface PointerFixups {
|
||||
readonly legacyBindStreams: number;
|
||||
/** Bytes in rebase streams that could not be completely decoded. */
|
||||
readonly unreadRebaseBytes: number;
|
||||
/** Unresolved legacy writes intersecting a scalar span, including incomplete
|
||||
* legacy applicability. Chained pointer values still require decode().
|
||||
*/
|
||||
hasUnresolvedLegacyBytes(address: bigint, size: number): boolean;
|
||||
/** Decode the pointer stored at `address` whose raw 64-bit value is `raw`. */
|
||||
decode(address: bigint, raw: bigint): PointerResolution;
|
||||
}
|
||||
@@ -508,6 +512,8 @@ const chainedFixups = (
|
||||
legacyRebaseStreams: rebases.examined,
|
||||
legacyBindStreams: binds.examined,
|
||||
unreadRebaseBytes: rebases.unreadBytes,
|
||||
hasUnresolvedLegacyBytes: (address, size) =>
|
||||
hasUnresolvedLegacyBytes(segments, rebases, binds, address, size),
|
||||
decode(address, raw) {
|
||||
const rawPointer = (): PointerResolution =>
|
||||
applicabilityComplete
|
||||
@@ -805,13 +811,13 @@ const readRebaseStream = (
|
||||
}
|
||||
};
|
||||
|
||||
const resolveLegacyRebase = (
|
||||
/** Read loader-owned membership without interpreting a stored pointer word. */
|
||||
const legacyRebaseState = (
|
||||
segments: readonly Segment[],
|
||||
membership: Uint8Array | null,
|
||||
address: bigint,
|
||||
raw: bigint,
|
||||
): PointerResolution | undefined => {
|
||||
if (membership === null) return undefined;
|
||||
): number | Extract<PointerResolution, { readonly kind: "unsupported" }> => {
|
||||
if (membership === null) return 0;
|
||||
let fileOffset: number | undefined;
|
||||
for (const segment of segments) {
|
||||
const relative = address - segment.address;
|
||||
@@ -830,10 +836,20 @@ const resolveLegacyRebase = (
|
||||
};
|
||||
fileOffset = Number(mapped);
|
||||
}
|
||||
if (fileOffset === undefined) return undefined;
|
||||
if (fileOffset === undefined) return 0;
|
||||
const at = Math.floor(fileOffset / 4);
|
||||
const shift = (fileOffset % 4) * 2;
|
||||
const state = ((membership[at] ?? 0) >> shift) & 3;
|
||||
return ((membership[at] ?? 0) >> shift) & 3;
|
||||
};
|
||||
|
||||
const resolveLegacyRebase = (
|
||||
segments: readonly Segment[],
|
||||
membership: Uint8Array | null,
|
||||
address: bigint,
|
||||
raw: bigint,
|
||||
): PointerResolution | undefined => {
|
||||
const state = legacyRebaseState(segments, membership, address);
|
||||
if (typeof state !== "number") return state;
|
||||
if (state === 0) return undefined;
|
||||
if (state === REBASE_POINTER) return { kind: "rebase", target: raw };
|
||||
return {
|
||||
@@ -1153,6 +1169,41 @@ const parseLegacyBinds = (
|
||||
};
|
||||
};
|
||||
|
||||
/** Query retained legacy facts; never expand bind runs or read scalar padding. */
|
||||
const hasUnresolvedLegacyBytes = (
|
||||
segments: readonly Segment[],
|
||||
rebases: RebaseFacts,
|
||||
binds: BindFacts,
|
||||
address: bigint,
|
||||
size: number,
|
||||
): boolean => {
|
||||
if (size <= 0) return false;
|
||||
if (!rebases.complete || !binds.complete) return true;
|
||||
const first = address - 7n;
|
||||
const end = address + BigInt(size);
|
||||
for (const run of binds.runs) {
|
||||
// An eight-byte bind overlaps [address,end) exactly when its start is
|
||||
// in [address-7,end). Find the first such member without expanding it.
|
||||
const index =
|
||||
first <= run.start
|
||||
? 0n
|
||||
: (first - run.start + run.stride - 1n) / run.stride;
|
||||
if (index < run.count && run.start + index * run.stride < end) return true;
|
||||
}
|
||||
if (rebases.membership !== null)
|
||||
for (let slot = first; slot < end; slot++) {
|
||||
const state = legacyRebaseState(segments, rebases.membership, slot);
|
||||
// Text rebases own four bytes; conflicting kinds may include a pointer.
|
||||
if (
|
||||
typeof state !== "number" ||
|
||||
state === REBASE_CONFLICT ||
|
||||
(state === REBASE_NON_POINTER && slot + 4n > address)
|
||||
)
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
/** Build pointer decoding for one slice from its fixup load commands. */
|
||||
export const parsePointerFixups = (
|
||||
bytes: Buffer,
|
||||
@@ -1187,6 +1238,7 @@ export const parsePointerFixups = (
|
||||
legacyRebaseStreams: 0,
|
||||
legacyBindStreams: 0,
|
||||
unreadRebaseBytes: 0,
|
||||
hasUnresolvedLegacyBytes: () => false,
|
||||
decode: (_address, raw) => rawPointer(raw),
|
||||
};
|
||||
const failures = [...rebases.failures, ...binds.failures];
|
||||
@@ -1201,6 +1253,8 @@ export const parsePointerFixups = (
|
||||
legacyRebaseStreams: rebases.examined,
|
||||
legacyBindStreams: binds.examined,
|
||||
unreadRebaseBytes: rebases.unreadBytes,
|
||||
hasUnresolvedLegacyBytes: (address, size) =>
|
||||
hasUnresolvedLegacyBytes(segments, rebases, binds, address, size),
|
||||
decode(address, raw) {
|
||||
const match = resolveLegacyBind(binds.runs, address);
|
||||
const rebase = resolveLegacyRebase(
|
||||
|
||||
@@ -19,6 +19,7 @@ const createInput = (
|
||||
legacyRebaseStreams: 0,
|
||||
legacyBindStreams: 0,
|
||||
unreadRebaseBytes: 0,
|
||||
hasUnresolvedLegacyBytes: () => false,
|
||||
decode: (address) =>
|
||||
address === 0x1000n
|
||||
? {
|
||||
|
||||
@@ -190,6 +190,7 @@ it("counts examined slots and reports an unresolved async slot", () => {
|
||||
legacyRebaseStreams: 0,
|
||||
legacyBindStreams: 0,
|
||||
unreadRebaseBytes: 0,
|
||||
hasUnresolvedLegacyBytes: () => false,
|
||||
},
|
||||
declarations: {
|
||||
facet: "swift_declarations",
|
||||
@@ -328,6 +329,7 @@ describe("Swift vtable coverage facts: unsupported pointer fixup coverage", () =
|
||||
legacyRebaseStreams: 0,
|
||||
legacyBindStreams: 0,
|
||||
unreadRebaseBytes: 0,
|
||||
hasUnresolvedLegacyBytes: () => false,
|
||||
},
|
||||
declarations: {
|
||||
facet: "swift_declarations",
|
||||
|
||||
@@ -0,0 +1,263 @@
|
||||
import { expect, it } from "vitest";
|
||||
import { parseNativeExports } from "./NativeExportParser.js";
|
||||
import {
|
||||
elfExportImage,
|
||||
peExportImage,
|
||||
} from "../../../tests/fixtures/binaryDiagnostics/nativeExports.js";
|
||||
import { freezeJsonSnapshot } from "../../domain/immutableJson.js";
|
||||
import { digestCanonicalValue } from "../../domain/canonicalDigest.js";
|
||||
import {
|
||||
analysisViewRequestSchema,
|
||||
projectAnalysisView,
|
||||
type AnalysisViewParent,
|
||||
} from "../../domain/analysisView/analysisView.js";
|
||||
|
||||
const artifact = {
|
||||
path: "/fixtures/exports",
|
||||
sha256: "a".repeat(64),
|
||||
bytes: 2048,
|
||||
};
|
||||
const parent = (report: unknown): AnalysisViewParent => ({
|
||||
evidenceId: `ev_${"b".repeat(64)}`,
|
||||
operation: "inspect_native_exports",
|
||||
normalizedResult: report,
|
||||
limitations: [],
|
||||
});
|
||||
const project = (report: unknown, input: unknown) => {
|
||||
const result = projectAnalysisView(
|
||||
parent(report),
|
||||
analysisViewRequestSchema.parse(input),
|
||||
);
|
||||
if (!result.ok) throw result.error;
|
||||
return result.value;
|
||||
};
|
||||
const elf = () => {
|
||||
const report = parseNativeExports(
|
||||
elfExportImage({
|
||||
symbols: [
|
||||
{ name: "same", version: 2, value: 0x1234n },
|
||||
{ name: "other" },
|
||||
{ name: "same", version: 0x8003, value: 0x1234n },
|
||||
{ name: "café" },
|
||||
],
|
||||
versionDefinitions: ["libfixture.so", "V1", "V2"],
|
||||
}),
|
||||
artifact,
|
||||
);
|
||||
if (report.format !== "elf") throw new Error("Expected ELF fixture");
|
||||
return report;
|
||||
};
|
||||
|
||||
it("keeps every ELF same-name/version alternative, source order and matching-page coordinates", () => {
|
||||
const report = elf();
|
||||
const input = {
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
names: ["same", "same"],
|
||||
fields: ["name", "value", "version"],
|
||||
limit: 1,
|
||||
};
|
||||
expect(project(report, input)).toMatchObject({
|
||||
items: [
|
||||
{
|
||||
source_index: 0,
|
||||
name: "same",
|
||||
value: "0x1234",
|
||||
version: { index: 2, name: "V1", hidden: false },
|
||||
},
|
||||
],
|
||||
exports: { export_count: 4, coverage: { status: "complete" } },
|
||||
coverage: { examined: 1, total: 2, next_offset: 1, exhausted: false },
|
||||
});
|
||||
expect(project(report, { ...input, offset: 1 })).toMatchObject({
|
||||
items: [
|
||||
{ source_index: 2, version: { index: 3, name: "V2", hidden: true } },
|
||||
],
|
||||
coverage: { total: 2, next_offset: null, exhausted: true },
|
||||
});
|
||||
expect(
|
||||
project(report, { ...input, indices: [2, 2, 99], fields: [] }),
|
||||
).toMatchObject({ items: [{ source_index: 2 }], coverage: { total: 1 } });
|
||||
expect(project(report, { ...input, names: [] })).toMatchObject({
|
||||
items: [],
|
||||
coverage: { status: "empty", total: 0 },
|
||||
});
|
||||
});
|
||||
|
||||
it("qualifies projected names with exact encoding facts and normalizes historical omissions", () => {
|
||||
const report = elf();
|
||||
expect(
|
||||
project(report, {
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
names: ["café"],
|
||||
fields: ["name"],
|
||||
}),
|
||||
).toMatchObject({
|
||||
items: [
|
||||
{
|
||||
source_index: 3,
|
||||
name: "café",
|
||||
name_bytes_base64: "Y2Fmw6k=",
|
||||
name_utf8_decodable: true,
|
||||
},
|
||||
],
|
||||
});
|
||||
const historical = {
|
||||
...report,
|
||||
exports: report.exports.map(
|
||||
({ name_bytes_base64: _bytes, name_utf8_decodable: _utf8, ...row }) =>
|
||||
row,
|
||||
),
|
||||
};
|
||||
freezeJsonSnapshot(historical);
|
||||
const input = { ...parent(historical), sealedResult: true };
|
||||
for (let attempt = 0; attempt < 2; attempt++) {
|
||||
const result = projectAnalysisView(
|
||||
input,
|
||||
analysisViewRequestSchema.parse({
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
names: ["café"],
|
||||
fields: ["name"],
|
||||
}),
|
||||
);
|
||||
if (!result.ok) throw result.error;
|
||||
expect(result.value).toMatchObject({
|
||||
items: [
|
||||
{ name: "café", name_bytes_base64: null, name_utf8_decodable: null },
|
||||
],
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("does not fill historical name defaults into a projection that omits names", () => {
|
||||
const result = project(elf(), {
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
names: ["same"],
|
||||
fields: ["value"],
|
||||
});
|
||||
expect(result).toHaveProperty("items", [
|
||||
{ source_index: 0, value: "0x1234" },
|
||||
{ source_index: 2, value: "0x1234" },
|
||||
]);
|
||||
const { view_digest, ...content } = result;
|
||||
expect(view_digest).toBe(digestCanonicalValue(content, "Analysis view"));
|
||||
});
|
||||
|
||||
it("preserves PE ordinal-only, forwarder, alias and same-address distinctions", () => {
|
||||
const report = parseNativeExports(
|
||||
peExportImage({
|
||||
ordinalBase: 65536,
|
||||
functions: [
|
||||
{ rva: 0x2000 },
|
||||
{ forwarder: "other.Target" },
|
||||
{ rva: 0x2000 },
|
||||
],
|
||||
names: [
|
||||
{ name: "AliasA", index: 0 },
|
||||
{ name: "AliasB", index: 0 },
|
||||
{ name: "Forwarded", index: 1 },
|
||||
],
|
||||
}),
|
||||
artifact,
|
||||
);
|
||||
const result = project(report, {
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
offset: 1,
|
||||
limit: 9,
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
items: [
|
||||
{
|
||||
source_index: 1,
|
||||
name: "AliasB",
|
||||
ordinal: 65536,
|
||||
address: "0x180002000",
|
||||
forwarder: null,
|
||||
},
|
||||
{
|
||||
source_index: 2,
|
||||
name: "Forwarded",
|
||||
ordinal: 65537,
|
||||
address: null,
|
||||
forwarder: "other.Target",
|
||||
},
|
||||
{
|
||||
source_index: 3,
|
||||
name: null,
|
||||
ordinal: 65538,
|
||||
address: "0x180002000",
|
||||
forwarder: null,
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["partial", "unsupported"] as const)(
|
||||
"keeps %s source coverage in empty pages and exposes complete issues",
|
||||
(status) => {
|
||||
const report = elf();
|
||||
report.coverage = {
|
||||
...report.coverage,
|
||||
status,
|
||||
reason: "Selected tables remain incomplete",
|
||||
};
|
||||
report.issues = [
|
||||
{
|
||||
code: "table_unavailable",
|
||||
location: null,
|
||||
message: "Unobserved table",
|
||||
},
|
||||
];
|
||||
expect(
|
||||
project(report, {
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
names: ["missing"],
|
||||
}),
|
||||
).toMatchObject({
|
||||
items: [],
|
||||
coverage: { status: "empty", total: 0 },
|
||||
exports: {
|
||||
coverage: { status, reason: "Selected tables remain incomplete" },
|
||||
issue_count: 1,
|
||||
},
|
||||
unknowns: ["Selected tables remain incomplete"],
|
||||
});
|
||||
expect(
|
||||
project(report, { kind: "native-exports", collection: "issues" }),
|
||||
).toMatchObject({
|
||||
items: [
|
||||
{
|
||||
source_index: 0,
|
||||
code: "table_unavailable",
|
||||
location: null,
|
||||
message: "Unobserved table",
|
||||
},
|
||||
],
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("revalidates mutable imports and refuses inconsistent coverage and format-incompatible fields", () => {
|
||||
const report = elf();
|
||||
const input = { ...parent(report), sealedResult: true };
|
||||
expect(projectAnalysisView(input, { kind: "summary" }).ok).toBe(true);
|
||||
Object.freeze(report);
|
||||
report.coverage.exports = 99;
|
||||
expect(projectAnalysisView(input, { kind: "summary" }).ok).toBe(false);
|
||||
expect(
|
||||
projectAnalysisView(
|
||||
parent(elf()),
|
||||
analysisViewRequestSchema.parse({
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
fields: ["ordinal"],
|
||||
}),
|
||||
).ok,
|
||||
).toBe(false);
|
||||
expect(projectAnalysisView(parent({}), { kind: "summary" }).ok).toBe(false);
|
||||
});
|
||||
+22
-19
@@ -1,3 +1,8 @@
|
||||
import {
|
||||
legacyMachoPlatform,
|
||||
type MachoVersionMinPlatform,
|
||||
} from "../../domain/apple/machoPlatform.js";
|
||||
|
||||
interface ParsedSection {
|
||||
readonly segment: string;
|
||||
readonly name: string;
|
||||
@@ -70,6 +75,14 @@ const DYLIB_FIELDS = [
|
||||
const DYLIB_ENCODING_FIELDS = ["time stamp", "options"];
|
||||
const EMPTY_VALUE_FIELDS = new Set(["options", "segname", "sectname"]);
|
||||
|
||||
const VERSION_MIN_PLATFORMS: ReadonlyMap<string, MachoVersionMinPlatform> =
|
||||
new Map([
|
||||
["LC_VERSION_MIN_MACOSX", 1],
|
||||
["LC_VERSION_MIN_IPHONEOS", 2],
|
||||
["LC_VERSION_MIN_TVOS", 3],
|
||||
["LC_VERSION_MIN_WATCHOS", 4],
|
||||
]);
|
||||
|
||||
/**
|
||||
* Non-verbose otool prints LC_BUILD_VERSION platform and tool codes as
|
||||
* numbers. Name the codes declared in the SDK's <mach-o/loader.h>, without
|
||||
@@ -408,8 +421,16 @@ const collectBuild = (
|
||||
}
|
||||
if (kind.startsWith("LC_VERSION_MIN_")) {
|
||||
const platform = kind.slice("LC_VERSION_MIN_".length) || null;
|
||||
const devicePlatform = VERSION_MIN_PLATFORMS.get(kind);
|
||||
const derivedPlatform =
|
||||
devicePlatform === undefined
|
||||
? undefined
|
||||
: legacyMachoPlatform(devicePlatform, state.cpuType);
|
||||
state.builds.push({
|
||||
platform: legacySimulatorPlatform(platform, state.cpuType),
|
||||
platform:
|
||||
derivedPlatform === undefined || derivedPlatform === devicePlatform
|
||||
? platform
|
||||
: buildCodeName(BUILD_PLATFORM_NAMES, String(derivedPlatform)),
|
||||
minimum_os: stringField(fields, "version"),
|
||||
sdk: stringField(fields, "sdk"),
|
||||
tools: [],
|
||||
@@ -417,24 +438,6 @@ const collectBuild = (
|
||||
}
|
||||
};
|
||||
|
||||
// Apple dyld interprets legacy iOS/tvOS/watchOS minimum-version commands on
|
||||
// Intel slices as simulator platforms. Newer commands encode this directly.
|
||||
const legacySimulatorPlatform = (
|
||||
platform: string | null,
|
||||
cpuType: number | null,
|
||||
): string | null => {
|
||||
if (![7, 0x01000007].includes(cpuType ?? -1)) return platform;
|
||||
return (
|
||||
(
|
||||
{
|
||||
IPHONEOS: "IOSSIMULATOR",
|
||||
TVOS: "TVOSSIMULATOR",
|
||||
WATCHOS: "WATCHOSSIMULATOR",
|
||||
} as const
|
||||
)[platform as "IPHONEOS" | "TVOS" | "WATCHOS"] ?? platform
|
||||
);
|
||||
};
|
||||
|
||||
const collectDependency = (
|
||||
kind: string,
|
||||
fields: Readonly<Record<string, string | number | null>>,
|
||||
|
||||
@@ -2,15 +2,57 @@ import { AnalysisOutputError } from "../../domain/analysisErrorCore.js";
|
||||
import {
|
||||
RECORDED_LINUX_CORE_PROFILES,
|
||||
type RecordedCrash,
|
||||
type RecordedCrashArchitecture,
|
||||
} from "../../domain/native/recordedCrash.js";
|
||||
|
||||
// Pinned pyelftools 0.33 display projections depend on the machine. Validate
|
||||
// them separately from the original numeric p_type identity bound below.
|
||||
const baseSegmentKinds = new Map([
|
||||
[0, "PT_NULL"],
|
||||
[1, "PT_LOAD"],
|
||||
[2, "PT_DYNAMIC"],
|
||||
[3, "PT_INTERP"],
|
||||
[4, "PT_NOTE"],
|
||||
[5, "PT_SHLIB"],
|
||||
[6, "PT_PHDR"],
|
||||
[7, "PT_TLS"],
|
||||
[0x60000000, "PT_LOOS"],
|
||||
[0x6fffffff, "PT_HIOS"],
|
||||
[0x6474e550, "PT_GNU_EH_FRAME"],
|
||||
[0x6474e551, "PT_GNU_STACK"],
|
||||
[0x6474e552, "PT_GNU_RELRO"],
|
||||
[0x6474e553, "PT_GNU_PROPERTY"],
|
||||
]);
|
||||
const segmentKinds: Readonly<
|
||||
Record<RecordedCrashArchitecture, ReadonlyMap<number, string>>
|
||||
> = {
|
||||
"x86_64-little-endian": baseSegmentKinds,
|
||||
"aarch64-little-endian": new Map([
|
||||
...baseSegmentKinds,
|
||||
[0x70000000, "PT_AARCH64_ARCHEXT"],
|
||||
[0x70000001, "PT_AARCH64_UNWIND"],
|
||||
]),
|
||||
};
|
||||
const matchesKind = (
|
||||
raw: number,
|
||||
reported: string | number,
|
||||
kinds: ReadonlyMap<number, string>,
|
||||
): boolean => {
|
||||
const known = kinds.get(raw);
|
||||
return typeof reported === "string"
|
||||
? reported === known
|
||||
: known === undefined && reported === raw;
|
||||
};
|
||||
|
||||
const segmentMatches = (
|
||||
segment: RecordedCrash["segments"][number],
|
||||
raw: Buffer,
|
||||
kinds: ReadonlyMap<number, string>,
|
||||
): boolean => {
|
||||
const type = raw.readUInt32LE(0);
|
||||
return (
|
||||
type === segment.type_id &&
|
||||
matchesKind(type, segment.type, kinds) &&
|
||||
BigInt(segment.flags) === BigInt(raw.readUInt32LE(4)) &&
|
||||
segment.file_backing ===
|
||||
(type === 0 || raw.readBigUInt64LE(32) === 0n ? "none" : "file") &&
|
||||
@@ -78,7 +120,8 @@ export function validateRecordedCrashStructure(
|
||||
)
|
||||
fail();
|
||||
const raw = bytesAt(offset, entryBytes);
|
||||
if (!segmentMatches(segment, raw)) fail();
|
||||
if (!segmentMatches(segment, raw, segmentKinds[report.architecture]))
|
||||
fail();
|
||||
}
|
||||
for (const note of report.notes) {
|
||||
const raw = bytesAt(BigInt(note.location.offset), 12);
|
||||
|
||||
@@ -540,7 +540,11 @@ it.skipIf(process.platform === "win32")(
|
||||
await writeFile(join(target, "created.txt"), "unobserved change");
|
||||
const after = await snapshotRoots(scenario);
|
||||
expect(before.files).toEqual(after.files);
|
||||
expect(after.coverage.enumeration_truncated).toBe(false);
|
||||
expect(after.coverage.enumeration_truncated).toBe(true);
|
||||
expect(after.coverage.enumeration_omissions).toEqual([
|
||||
{ path: "root_0:.", reason: "symlink_not_followed", system_code: null },
|
||||
]);
|
||||
expect(after.completeRoots).toEqual([]);
|
||||
expect(after.coverage.unobserved_subtrees).toEqual([
|
||||
{ path: "root_0:.", reason: "symlink_not_followed" },
|
||||
]);
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
import { expect, it } from "vitest";
|
||||
import { AnalysisCleanupLease } from "../../artifacts/AnalysisCleanupLease.js";
|
||||
import { analysisErrorProjectionSchema } from "../../contracts/errorSchemas.js";
|
||||
import { AnalysisAccessDeniedError } from "../../domain/analysisErrorCore.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { emptyProcessCapture } from "../../domain/process/processCapture.fixture.js";
|
||||
import { parseProcessScenario } from "../../domain/process/processScenario.js";
|
||||
import { ProcessCaptureSnapshotFailure } from "./FilesystemSnapshot.js";
|
||||
import { ProcessCaptureError } from "./ProcessCaptureError.js";
|
||||
import { prepareProcessCapture } from "./ProcessCaptureLifecycle.js";
|
||||
|
||||
it.each(["EACCES", "EPERM"] as const)(
|
||||
"retains %s admission facts through snapshot finalization and exact cleanup retry",
|
||||
async (systemCode) => {
|
||||
const selected = "/fixture/denied/selected";
|
||||
const native = Object.assign(new Error("selected root lookup denied"), {
|
||||
code: systemCode,
|
||||
});
|
||||
const primary = new AnalysisAccessDeniedError(
|
||||
"capture_process_scenario",
|
||||
selected,
|
||||
systemCode,
|
||||
{ cause: native },
|
||||
);
|
||||
const snapshot = {
|
||||
files: [],
|
||||
completeRoots: ["root_0"],
|
||||
truncated: true,
|
||||
coverage: {
|
||||
...emptyProcessCapture().truncation_details.filesystem_before,
|
||||
enumeration_truncated: true,
|
||||
enumeration_reasons: ["entry_unavailable" as const],
|
||||
enumeration_omissions: [
|
||||
{
|
||||
path: "root_1:.",
|
||||
reason: "entry_unavailable" as const,
|
||||
system_code: systemCode,
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
let closes = 0;
|
||||
const lease = new AnalysisCleanupLease(async () => {
|
||||
closes++;
|
||||
}, ["fixture:earlier-snapshot-descriptor"]);
|
||||
const snapshotFailure = new ProcessCaptureSnapshotFailure({
|
||||
cause: primary,
|
||||
snapshot,
|
||||
cleanupLeases: [lease],
|
||||
});
|
||||
const scenario = parseProcessScenario({
|
||||
executable: process.execPath,
|
||||
filesystem_observation_paths: ["/fixture/absent", selected],
|
||||
});
|
||||
const failure = await prepareProcessCapture(
|
||||
scenario,
|
||||
undefined,
|
||||
async () => {
|
||||
throw snapshotFailure;
|
||||
},
|
||||
{
|
||||
createTemporaryRoot: async () => {
|
||||
throw new Error("admission must precede allocation");
|
||||
},
|
||||
cleanup: async () => {
|
||||
throw new Error("no temporary root was allocated");
|
||||
},
|
||||
},
|
||||
).catch((cause: unknown) => cause);
|
||||
if (!(failure instanceof ProcessCaptureError))
|
||||
throw new Error("Expected retained snapshot finalization failure");
|
||||
expect(primary.cause).toBe(native);
|
||||
expect(closes).toBe(0);
|
||||
expect(projectAnalysisError(failure)).toMatchObject({
|
||||
code: "cleanup_incomplete",
|
||||
details: {
|
||||
resources: ["fixture:earlier-snapshot-descriptor"],
|
||||
partial_observation: {
|
||||
observations: {
|
||||
target_pid: { state: "unavailable" },
|
||||
filesystem_snapshots: {
|
||||
before: { state: "available", value: snapshot },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
await lease.release();
|
||||
await lease.release();
|
||||
expect(closes).toBe(1);
|
||||
expect(
|
||||
analysisErrorProjectionSchema.parse(projectAnalysisError(failure)),
|
||||
).toMatchObject({
|
||||
code: "access_denied",
|
||||
category: "unavailable",
|
||||
details: {
|
||||
path: selected,
|
||||
system_code: systemCode,
|
||||
boundary: "filesystem-read",
|
||||
partial_observation: {
|
||||
observations: {
|
||||
filesystem_snapshots: {
|
||||
before: { state: "available", value: snapshot },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
},
|
||||
);
|
||||
@@ -7,6 +7,7 @@ import type { ProcessScenario } from "../../domain/process/processScenario.js";
|
||||
import type { FilesystemCoverage } from "../../domain/process/processCaptureCoverage.js";
|
||||
import type { Stats } from "node:fs";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import { AnalysisAccessDeniedError } from "../../domain/analysisErrorCore.js";
|
||||
import type {
|
||||
FileState,
|
||||
ProcessFilesystemSnapshot,
|
||||
@@ -26,7 +27,6 @@ import {
|
||||
withProcessCaptureFile,
|
||||
type ProcessCaptureCleanupRetainer,
|
||||
} from "./ProcessCaptureRead.js";
|
||||
import { AnalysisAccessDeniedError } from "../../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
|
||||
|
||||
@@ -121,21 +121,6 @@ type PathLookup =
|
||||
readonly cause: unknown;
|
||||
};
|
||||
|
||||
const rootAdmissionFailure = (
|
||||
path: string,
|
||||
lookup: Exclude<PathLookup, { readonly state: "present" }>,
|
||||
): unknown =>
|
||||
lookup.system_code === "EACCES" || lookup.system_code === "EPERM"
|
||||
? new AnalysisAccessDeniedError(
|
||||
"capture_process",
|
||||
path,
|
||||
lookup.system_code,
|
||||
{
|
||||
cause: lookup.cause,
|
||||
},
|
||||
)
|
||||
: lookup.cause;
|
||||
|
||||
/**
|
||||
* Look up one path without following its final component. ENOENT and ENOTDIR
|
||||
* both prove that no entry exists at the path; other unavailable lookups leave
|
||||
@@ -180,7 +165,6 @@ export class ProcessCaptureSnapshotFailure extends ProcessCaptureError {
|
||||
readonly snapshot: ProcessFilesystemSnapshot | undefined;
|
||||
readonly cleanupLeases: readonly AnalysisCleanupLease[];
|
||||
readonly readFailures: readonly ProcessCaptureReadFailure<unknown>[];
|
||||
readonly primaryFailure: AnalysisAccessDeniedError | undefined;
|
||||
|
||||
constructor(options: {
|
||||
readonly cause: unknown;
|
||||
@@ -228,6 +212,9 @@ export class ProcessCaptureSnapshotFailure extends ProcessCaptureError {
|
||||
"snapshot handle cleanup failed";
|
||||
super(message, {
|
||||
cause: primary,
|
||||
...(options.cause instanceof AnalysisAccessDeniedError
|
||||
? { cleanupPrimary: options.cause }
|
||||
: {}),
|
||||
...(cancellation
|
||||
? { userCategory: "cancelled" as const, reason: "cancelled" as const }
|
||||
: {}),
|
||||
@@ -249,24 +236,21 @@ export class ProcessCaptureSnapshotFailure extends ProcessCaptureError {
|
||||
this.snapshot = options.snapshot;
|
||||
this.cleanupLeases = cleanupLeases;
|
||||
this.readFailures = readFailures;
|
||||
this.primaryFailure =
|
||||
options.cause instanceof AnalysisAccessDeniedError
|
||||
? options.cause
|
||||
: undefined;
|
||||
if (cleanupLeases.length > 0)
|
||||
new AnalysisCleanupLeaseGroup(this, cleanupLeases);
|
||||
}
|
||||
|
||||
/** The snapshot owns observations; the admitted read owns its failure identity. */
|
||||
failureForCaller(): AnalysisError {
|
||||
if (this.primaryFailure === undefined) return this;
|
||||
const primary = this.cleanupPrimary;
|
||||
if (!(primary instanceof AnalysisAccessDeniedError)) return this;
|
||||
return this.cleanupIncomplete
|
||||
? analysisErrorWithCleanupFailure(
|
||||
this.primaryFailure,
|
||||
primary,
|
||||
this,
|
||||
"capture_process",
|
||||
).bindCleanupDependencies([this.primaryFailure, this])
|
||||
: this.primaryFailure;
|
||||
).bindCleanupDependencies([primary, this])
|
||||
: primary;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -527,7 +511,16 @@ export const snapshotRoots = async (
|
||||
path: `${rootAlias}:${relativePath}`,
|
||||
reason: "symlink_not_followed",
|
||||
});
|
||||
return true;
|
||||
truncated = true;
|
||||
enumerationReasons.add("symlink_not_followed");
|
||||
enumerationOmissions.push({
|
||||
path: `${rootAlias}:${relativePath}`,
|
||||
reason: "symlink_not_followed",
|
||||
system_code: null,
|
||||
});
|
||||
// A link leaves its descendants unknown, while a physically enumerated
|
||||
// parent still proves absence of unrelated sibling names.
|
||||
return depth > 0;
|
||||
}
|
||||
if (stats.isFile()) {
|
||||
if (remainingBytes < stats.size) {
|
||||
@@ -677,7 +670,8 @@ export const snapshotRoots = async (
|
||||
files_limit: scenario.limits.files,
|
||||
depth_limit: scenario.limits.filesystem_depth,
|
||||
enumeration_truncated:
|
||||
completeRoots.length !== scenario.filesystem_observation_paths.length,
|
||||
completeRoots.length !== scenario.filesystem_observation_paths.length ||
|
||||
enumerationOmissions.length > 0,
|
||||
enumeration_reasons: [...enumerationReasons],
|
||||
enumeration_omissions: enumerationOmissions,
|
||||
unobserved_subtrees: unobservedSubtrees,
|
||||
@@ -695,8 +689,24 @@ export const snapshotRoots = async (
|
||||
const alias = `root_${String(index)}`;
|
||||
const lookup = await lookupPath(root, signal);
|
||||
if (lookup.state !== "present") {
|
||||
if (admission === undefined && lookup.system_code !== "ENOENT")
|
||||
throw rootAdmissionFailure(root, lookup);
|
||||
if (admission === undefined && lookup.system_code !== "ENOENT") {
|
||||
if (
|
||||
lookup.system_code === "EACCES" ||
|
||||
lookup.system_code === "EPERM"
|
||||
) {
|
||||
omitEntry(`${alias}:.`, {
|
||||
reason: "entry_unavailable",
|
||||
system_code: lookup.system_code,
|
||||
});
|
||||
throw new AnalysisAccessDeniedError(
|
||||
"capture_process_scenario",
|
||||
root,
|
||||
lookup.system_code,
|
||||
{ cause: lookup.cause },
|
||||
);
|
||||
}
|
||||
throw lookup.cause;
|
||||
}
|
||||
if (lookup.state === "absent") completeRoots.push(alias);
|
||||
else
|
||||
omitEntry(`${alias}:.`, {
|
||||
|
||||
@@ -940,6 +940,7 @@ const registerConfiguredAnalysisTools = (
|
||||
nativeExports,
|
||||
toolLogger,
|
||||
recordEvidence,
|
||||
evidenceById,
|
||||
withAdmittedAnalysis,
|
||||
);
|
||||
registerElfNotesTool(
|
||||
|
||||
@@ -6,6 +6,13 @@ import type { Logger } from "pino";
|
||||
import { createEvidenceToolHandler } from "./contractToolHandler.js";
|
||||
import type { WithAdmittedAnalysis } from "./analysisAdmission.js";
|
||||
import { applicationToolRegistrationOptions } from "./toolRegistrationOptions.js";
|
||||
import type { EvidenceLookup } from "../application/EvidenceInputResolver.js";
|
||||
import {
|
||||
requireAnalysisViewRetention,
|
||||
summarizeRetainedAnalysis,
|
||||
} from "../application/analysisView/AnalysisViewService.js";
|
||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||
import { err } from "../domain/result.js";
|
||||
|
||||
/** Register the explicit-file export-table workflow and retain its complete Evidence. */
|
||||
export const registerNativeExportsTool = (
|
||||
@@ -13,12 +20,34 @@ export const registerNativeExportsTool = (
|
||||
service: NativeExportsService,
|
||||
logger: Logger,
|
||||
recordEvidence?: EvidenceWriter["recordEvidence"],
|
||||
evidenceById?: EvidenceLookup,
|
||||
withAdmittedAnalysis?: WithAdmittedAnalysis,
|
||||
): void => {
|
||||
const contract = toolContract("inspect_native_exports");
|
||||
const handler = createEvidenceToolHandler(
|
||||
server,
|
||||
(input, options) => service.inspect(input, options),
|
||||
async (rawInput, options) => {
|
||||
const parsed = contract.inputSchema.safeParse(rawInput);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
contract.name,
|
||||
parsed.error.issues,
|
||||
rawInput,
|
||||
{ cause: parsed.error },
|
||||
),
|
||||
);
|
||||
const { detail, ...input } = parsed.data;
|
||||
const retention = { recordEvidence, evidenceById };
|
||||
if (detail === "summary") {
|
||||
const ready = requireAnalysisViewRetention(contract.name, retention);
|
||||
if (!ready.ok) return ready;
|
||||
}
|
||||
const inspected = await service.inspect(input, options);
|
||||
return detail === "summary" && inspected.ok
|
||||
? summarizeRetainedAnalysis(inspected.value, retention)
|
||||
: inspected;
|
||||
},
|
||||
logger,
|
||||
recordEvidence,
|
||||
withAdmittedAnalysis,
|
||||
|
||||
@@ -240,8 +240,8 @@ export class SigrokRecordingProvider implements SignalRecordingPort {
|
||||
},
|
||||
},
|
||||
);
|
||||
// Some selected-channel refusals are diagnostics, not process failures:
|
||||
// sigrok can emit fallback annotations and still exit successfully.
|
||||
// Selected-channel and input-option refusals can be diagnostics rather
|
||||
// than process failures: fallback annotations may still exit successfully.
|
||||
const rejection =
|
||||
selection === undefined
|
||||
? undefined
|
||||
@@ -335,6 +335,8 @@ export class SigrokRecordingProvider implements SignalRecordingPort {
|
||||
phase = "producer";
|
||||
const args = [
|
||||
"--dont-scan",
|
||||
// An explicit relative path preserves filename hints while keeping
|
||||
// a recording named "-" distinct from sigrok's stdin operand.
|
||||
`--input-file=./${snapshotName}`,
|
||||
`--protocol-decoders=${input.decoders}`,
|
||||
"--protocol-decoder-samplenum",
|
||||
|
||||
@@ -14,6 +14,8 @@ export type SigrokSelection =
|
||||
>
|
||||
| Pick<InspectSignalDecodersInput, "decoder">;
|
||||
|
||||
type SigrokDiagnosticMatcher = (message: string) => boolean;
|
||||
|
||||
const rejectedSelector = (
|
||||
field: "decoder" | "decoders" | "annotations" | "input_format",
|
||||
producerMessage: string,
|
||||
@@ -32,6 +34,8 @@ const CHANNEL_RECOVERY =
|
||||
"Set each decoder channel target to a channel present in the recording; inspect_signal_decoders shows the selected decoder's channel IDs.";
|
||||
const INPUT_FORMAT_RECOVERY =
|
||||
"Use a supported sigrok input format name, or omit input_format to let sigrok-cli autodetect the recording.";
|
||||
const INPUT_OPTION_RECOVERY =
|
||||
"Use options supported by the selected sigrok input format, or remove the rejected option.";
|
||||
|
||||
/**
|
||||
* sigrok-cli loads selected decoders before decoding or showing help, so an
|
||||
@@ -40,13 +44,13 @@ const INPUT_FORMAT_RECOVERY =
|
||||
* whose own imports or code fail to load.
|
||||
*/
|
||||
const unknownDecoderRefusal = (
|
||||
lines: ReadonlySet<string>,
|
||||
hasDiagnostic: SigrokDiagnosticMatcher,
|
||||
id: string,
|
||||
): string | undefined => {
|
||||
const producerMessage = `Failed to load protocol decoder ${id}.`;
|
||||
return id !== "" &&
|
||||
lines.has(producerMessage) &&
|
||||
lines.has(
|
||||
hasDiagnostic(producerMessage) &&
|
||||
hasDiagnostic(
|
||||
`srd: ModuleNotFoundError: Failed to load decoder ${id}: import by name failed: No module named '${id}'`,
|
||||
)
|
||||
? producerMessage
|
||||
@@ -55,12 +59,12 @@ const unknownDecoderRefusal = (
|
||||
|
||||
/**
|
||||
* The sigrok-cli 0.7.2 channel mapper reports missing and unknown targets as
|
||||
* plain stderr lines, even though it continues with the other mappings. Match
|
||||
* plain stderr diagnostics, even though it continues with the other mappings. Match
|
||||
* the exact selected channel key or target value rather than searching the
|
||||
* selector text for a substring.
|
||||
*/
|
||||
const missingSelectedChannelRefusal = (
|
||||
lines: ReadonlySet<string>,
|
||||
hasDiagnostic: SigrokDiagnosticMatcher,
|
||||
decoders: string,
|
||||
): string | undefined => {
|
||||
const selectedValues = new Set<string>();
|
||||
@@ -69,14 +73,14 @@ const missingSelectedChannelRefusal = (
|
||||
const separator = assignment.indexOf("=");
|
||||
if (separator < 0 && assignment !== "") {
|
||||
const producerMessage = `Channel name for "${assignment}" missing.`;
|
||||
if (lines.has(`cli: ${producerMessage}`)) return producerMessage;
|
||||
if (hasDiagnostic(`cli: ${producerMessage}`)) return producerMessage;
|
||||
}
|
||||
if (separator > 0) selectedValues.add(assignment.slice(separator + 1));
|
||||
}
|
||||
}
|
||||
for (const channel of selectedValues) {
|
||||
const producerMessage = `No channel with name "${channel}" found.`;
|
||||
if (lines.has(`cli: ${producerMessage}`)) return producerMessage;
|
||||
if (hasDiagnostic(`cli: ${producerMessage}`)) return producerMessage;
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
@@ -89,42 +93,54 @@ export const parseSigrokSelectorRejection = (
|
||||
stderr: string,
|
||||
selection: SigrokSelection,
|
||||
): SigrokSelectorRejection | undefined => {
|
||||
const lines = new Set(stderr.split("\n"));
|
||||
// Selected option keys may contain LF, which the tagged producer prints verbatim.
|
||||
// Match the complete caption at diagnostic boundaries without splitting it.
|
||||
const hasDiagnostic: SigrokDiagnosticMatcher = (message) =>
|
||||
stderr === message ||
|
||||
stderr.startsWith(`${message}\n`) ||
|
||||
stderr.endsWith(`\n${message}`) ||
|
||||
stderr.includes(`\n${message}\n`);
|
||||
if ("decoder" in selection) {
|
||||
const refusal =
|
||||
selection.decoder === null
|
||||
? undefined
|
||||
: unknownDecoderRefusal(lines, selection.decoder);
|
||||
: unknownDecoderRefusal(hasDiagnostic, selection.decoder);
|
||||
return refusal === undefined
|
||||
? undefined
|
||||
: rejectedSelector("decoder", refusal, HELP_RECOVERY);
|
||||
}
|
||||
const inputFormat = selection.input_format?.split(":")[0] ?? "";
|
||||
const [inputFormat = "", ...inputOptions] =
|
||||
selection.input_format?.split(":") ?? [];
|
||||
const inputFormatRefusal = `Error: unknown input module '${inputFormat}'.`;
|
||||
if (inputFormat !== "" && lines.has(`cli: ${inputFormatRefusal}`))
|
||||
if (inputFormat !== "" && hasDiagnostic(`cli: ${inputFormatRefusal}`))
|
||||
return rejectedSelector(
|
||||
"input_format",
|
||||
inputFormatRefusal,
|
||||
INPUT_FORMAT_RECOVERY,
|
||||
);
|
||||
for (const assignment of selection.input_format?.split(":").slice(1) ?? []) {
|
||||
const key = assignment.split("=", 1)[0] ?? "";
|
||||
const refusal = `Unknown keyword: ${key}.`;
|
||||
if (key !== "" && lines.has(refusal))
|
||||
// sigrok-cli 0.7.2 parses format:option=value with colon separators and
|
||||
// warns on unknown option keys, then continues with the supported options.
|
||||
for (const option of inputOptions) {
|
||||
const key = option.split("=", 1)[0] ?? "";
|
||||
const producerMessage = `Unknown keyword: ${key}.`;
|
||||
if (key !== "" && hasDiagnostic(producerMessage))
|
||||
return rejectedSelector(
|
||||
"input_format",
|
||||
refusal,
|
||||
"Use a supported option for the selected sigrok input format, or omit that option.",
|
||||
producerMessage,
|
||||
INPUT_OPTION_RECOVERY,
|
||||
);
|
||||
}
|
||||
// sigrok-cli stacks decoders with "," and appends options with ":".
|
||||
for (const entry of selection.decoders.split(",")) {
|
||||
const refusal = unknownDecoderRefusal(lines, entry.split(":")[0] ?? "");
|
||||
const refusal = unknownDecoderRefusal(
|
||||
hasDiagnostic,
|
||||
entry.split(":")[0] ?? "",
|
||||
);
|
||||
if (refusal !== undefined)
|
||||
return rejectedSelector("decoders", refusal, DECODE_RECOVERY);
|
||||
}
|
||||
const missingChannel = missingSelectedChannelRefusal(
|
||||
lines,
|
||||
hasDiagnostic,
|
||||
selection.decoders,
|
||||
);
|
||||
if (missingChannel !== undefined)
|
||||
@@ -135,7 +151,7 @@ export const parseSigrokSelectorRejection = (
|
||||
if (decoder === "" || classes === undefined || rest.length > 0) continue;
|
||||
for (const annotation of classes.split(":")) {
|
||||
const producerMessage = `Annotation '${annotation}' not found for protocol decoder '${decoder}'.`;
|
||||
if (annotation !== "" && lines.has(producerMessage))
|
||||
if (annotation !== "" && hasDiagnostic(producerMessage))
|
||||
return rejectedSelector(
|
||||
"annotations",
|
||||
producerMessage,
|
||||
|
||||
+49
-21
@@ -421,11 +421,58 @@ it("matches a CLI-selected ./ module exactly against its glue reference", () =>
|
||||
]);
|
||||
});
|
||||
|
||||
it.each(["\n", "\r", "\r\n", "\u2028", "\u2029"])(
|
||||
"retains complete .wasm query and fragment literals containing %j",
|
||||
(separator) => {
|
||||
const candidates = ["/artifacts/module.wasm", "/other/module.wasm"];
|
||||
for (const qualifier of ["?query=", "#fragment="]) {
|
||||
const value = `https://example.invalid/module.wasm${qualifier}a${separator}b`;
|
||||
const result = associateWasmGlue(
|
||||
`fetch(${JSON.stringify(value)});`,
|
||||
"/artifacts/glue.js",
|
||||
candidates,
|
||||
);
|
||||
expect(result).toEqual({
|
||||
parse_status: "complete",
|
||||
references: [
|
||||
{
|
||||
value,
|
||||
line: 1,
|
||||
column: 6,
|
||||
candidate_paths: candidates,
|
||||
association: "basename-candidates",
|
||||
evidence_kind: "static-literal",
|
||||
},
|
||||
],
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it.each(["\n", "\r", "\r\n", "\u2028", "\u2029"])(
|
||||
"does not accept a path-ending separator without a query or fragment: %j",
|
||||
(separator) => {
|
||||
const value = `https://example.invalid/module.wasm${separator}`;
|
||||
expect(
|
||||
associateWasmGlue(`fetch(${JSON.stringify(value)});`, "/glue.js", []),
|
||||
).toEqual({ parse_status: "complete", references: [] });
|
||||
},
|
||||
);
|
||||
|
||||
it("tests long multi-line glue literals for a .wasm suffix in linear time", () => {
|
||||
const value = `${".wasm".repeat(32_000)}\nend`;
|
||||
const value = `${".wasm#".repeat(32_000)}\nend`;
|
||||
const started = performance.now();
|
||||
const result = associateWasmGlue(
|
||||
`const s = ${JSON.stringify(value)};`,
|
||||
"/glue.js",
|
||||
[],
|
||||
);
|
||||
expect(result.parse_status).toBe("complete");
|
||||
expect(result.references).toHaveLength(1);
|
||||
expect(result.references[0]?.value).toBe(value);
|
||||
const negative = `${".wasm!".repeat(32_000)}\nend`;
|
||||
expect(
|
||||
associateWasmGlue(`const s = ${JSON.stringify(value)};`, "/glue.js", []),
|
||||
associateWasmGlue(`const s = ${JSON.stringify(negative)};`, "/glue.js", []),
|
||||
).toEqual({ parse_status: "complete", references: [] });
|
||||
expect(performance.now() - started).toBeLessThan(1_000);
|
||||
expect(
|
||||
@@ -451,25 +498,6 @@ it("retains earlier module names for objdump framing when a later optional subse
|
||||
).toHaveLength(7);
|
||||
});
|
||||
|
||||
it.each(["\n", "\r", "\u2028", "\u2029"])(
|
||||
"retains glue references with a line separator in a query or fragment: %j",
|
||||
(separator) => {
|
||||
for (const delimiter of ["?", "#"]) {
|
||||
const value = `https://example.test/module.wasm${delimiter}before${separator}after`;
|
||||
expect(
|
||||
associateWasmGlue(`fetch(${JSON.stringify(value)});`, "/glue.js", [
|
||||
"/module.wasm",
|
||||
]).references,
|
||||
).toMatchObject([
|
||||
{
|
||||
value,
|
||||
candidate_paths: ["/module.wasm"],
|
||||
association: "basename-candidates",
|
||||
},
|
||||
]);
|
||||
}
|
||||
},
|
||||
);
|
||||
it.each([
|
||||
'import source module from "./module.wasm";',
|
||||
'const module = await import.source("./module.wasm");',
|
||||
|
||||
@@ -4,7 +4,10 @@ import { parseJavaScriptSource } from "../domain/javascript/javascriptSourcePars
|
||||
import { traverseJavaScriptAst } from "../domain/javascript/javascriptSemanticTraversal.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
|
||||
/** Query/fragment contents do not change the path suffix; one bounded scan is enough. */
|
||||
/**
|
||||
* Only the path before the first query/fragment delimiter determines the suffix.
|
||||
* Query/fragment content can contain any line terminator; one bounded scan suffices.
|
||||
*/
|
||||
const endsWithWasmReference = (value: string): boolean => {
|
||||
const delimiter = value.search(/[?#]/u);
|
||||
return /\.wasm$/iu.test(delimiter < 0 ? value : value.slice(0, delimiter));
|
||||
|
||||
@@ -0,0 +1,193 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect } from "vitest";
|
||||
|
||||
import { analysisErrorProjectionSchema } from "../../src/contracts/errorSchemas.js";
|
||||
import { apkSignatureResultSchema } from "../../src/domain/apkSignature/apkSignatureReport.js";
|
||||
import { parseEvidence } from "../../src/domain/evidence.js";
|
||||
import { zip64Tail } from "../fixtures/apkSignature/zip64Tail.js";
|
||||
import { parseMcpToolError } from "../fixtures/mcpToolError.js";
|
||||
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
|
||||
import { cliTest, type TestCli } from "../support/cli/cliFixture.js";
|
||||
|
||||
const inspectBoth = async (cli: TestCli, path: string) => {
|
||||
const output = await cli.sharedCliRun!({
|
||||
arguments: [
|
||||
"inspect-apk-signature",
|
||||
path,
|
||||
"--detail",
|
||||
"complete",
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
const call = await cli.sharedMcpCall!();
|
||||
const response = await call("inspect_apk_signature", {
|
||||
path,
|
||||
detail: "complete",
|
||||
});
|
||||
return { output, response };
|
||||
};
|
||||
|
||||
const expectUnsignedZip64 = async (
|
||||
cli: TestCli,
|
||||
path: string,
|
||||
bytes: Uint8Array,
|
||||
) => {
|
||||
await writeFile(path, bytes);
|
||||
const { output, response } = await inspectBoth(cli, path);
|
||||
expect(output.exitCode, output.stderr).toBe(0);
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
const sha256 = createHash("sha256").update(bytes).digest("hex");
|
||||
for (const value of [output.json, response.structuredContent]) {
|
||||
const evidence = parseEvidence(value);
|
||||
expect(evidence.subject).toMatchObject({
|
||||
local_path: path,
|
||||
digest: { sha256 },
|
||||
});
|
||||
expect(
|
||||
apkSignatureResultSchema.parse(evidence.normalized_result),
|
||||
).toMatchObject({
|
||||
artifact: { path, bytes: bytes.length, sha256 },
|
||||
schemes: {
|
||||
v1: { present: false },
|
||||
v2: { present: false },
|
||||
v3: { present: false },
|
||||
"v3.1": { present: false },
|
||||
},
|
||||
runtime_execution: "not-performed",
|
||||
});
|
||||
}
|
||||
expect(await readFile(path)).toEqual(Buffer.from(bytes));
|
||||
};
|
||||
|
||||
const expectRefusal = async (
|
||||
cli: TestCli,
|
||||
path: string,
|
||||
bytes: Uint8Array,
|
||||
expected: Record<string, unknown>,
|
||||
) => {
|
||||
await writeFile(path, bytes);
|
||||
const { output, response } = await inspectBoth(cli, path);
|
||||
expect(output.exitCode, output.stderr).toBe(1);
|
||||
const cliError = analysisErrorProjectionSchema.parse(output.json);
|
||||
expect(cliError).toMatchObject(expected);
|
||||
const mcpError = parseMcpToolError(response).error;
|
||||
expect(mcpError).toMatchObject(expected);
|
||||
expect(mcpError.message).toBe(cliError.message);
|
||||
expect(await readFile(path)).toEqual(Buffer.from(bytes));
|
||||
};
|
||||
|
||||
cliTest(
|
||||
"classifies damaged Zip64 framing as input errors through CLI and stdio MCP",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-apk-zip64-framing-");
|
||||
const path = join(root, "selected.apk");
|
||||
const fixture = await zip64Tail();
|
||||
const { recordOffset, locatorOffset, eocdOffset } = fixture;
|
||||
await expectUnsignedZip64(cli, path, fixture.bytes);
|
||||
const cases = [
|
||||
{
|
||||
at: locatorOffset,
|
||||
value: 0x07064b51,
|
||||
diagnosticOffset: eocdOffset,
|
||||
detail: "missing Zip64 EOCD locator",
|
||||
},
|
||||
{
|
||||
at: recordOffset,
|
||||
value: 0x06064b51,
|
||||
diagnosticOffset: recordOffset,
|
||||
detail: "invalid Zip64 EOCD signature",
|
||||
},
|
||||
{
|
||||
at: locatorOffset + 8,
|
||||
value: recordOffset + 1,
|
||||
diagnosticOffset: locatorOffset + 8,
|
||||
detail: "Zip64 EOCD record extends beyond its locator",
|
||||
},
|
||||
{
|
||||
at: locatorOffset + 16,
|
||||
value: 0,
|
||||
diagnosticOffset: locatorOffset + 16,
|
||||
detail: "Zip64 locator declares zero disks",
|
||||
},
|
||||
{
|
||||
at: locatorOffset + 4,
|
||||
value: 1,
|
||||
diagnosticOffset: locatorOffset + 4,
|
||||
detail: "Zip64 EOCD disk is outside the declared disk count",
|
||||
},
|
||||
];
|
||||
for (const scenario of cases) {
|
||||
const bytes = Uint8Array.from(fixture.bytes);
|
||||
new DataView(bytes.buffer).setUint32(scenario.at, scenario.value, true);
|
||||
await expectRefusal(cli, path, bytes, {
|
||||
code: "invalid_request",
|
||||
category: "invalid_input",
|
||||
retryable: false,
|
||||
recovery: {
|
||||
kind: "inspect_input_contract",
|
||||
operation: "inspect_apk_signature",
|
||||
},
|
||||
details: {
|
||||
issues: [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_value",
|
||||
message: `Damaged ZIP/APK at byte ${String(scenario.diagnosticOffset)}: ${scenario.detail}.`,
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
}
|
||||
// Restore the exact original bytes at the same caller-selected path.
|
||||
await expectUnsignedZip64(cli, path, fixture.bytes);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"keeps Zip64 representation limits and non-ZIP targets distinct from damaged input",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-apk-zip64-unsupported-");
|
||||
const path = join(root, "selected.apk");
|
||||
const fixture = await zip64Tail();
|
||||
for (const at of [
|
||||
fixture.locatorOffset + 8,
|
||||
fixture.recordOffset + 32,
|
||||
fixture.recordOffset + 40,
|
||||
fixture.recordOffset + 48,
|
||||
]) {
|
||||
const bytes = Uint8Array.from(fixture.bytes);
|
||||
new DataView(bytes.buffer).setBigUint64(at, 1n << 53n, true);
|
||||
await expectRefusal(cli, path, bytes, {
|
||||
code: "unsupported_target",
|
||||
category: "unsupported_target",
|
||||
message: expect.stringContaining("safe range"),
|
||||
details: { path },
|
||||
});
|
||||
}
|
||||
const split = Uint8Array.from(fixture.bytes);
|
||||
const view = new DataView(split.buffer);
|
||||
view.setUint32(fixture.locatorOffset + 4, 1, true);
|
||||
view.setUint32(fixture.locatorOffset + 16, 2, true);
|
||||
view.setBigUint64(fixture.locatorOffset + 8, 1000000n, true);
|
||||
// Only locator metadata is authored here; this is not a real split-archive oracle.
|
||||
await expectRefusal(cli, path, split, {
|
||||
code: "unsupported_target",
|
||||
category: "unsupported_target",
|
||||
message: expect.stringContaining("multi-disk Zip64"),
|
||||
details: { path },
|
||||
});
|
||||
await expectRefusal(
|
||||
cli,
|
||||
path,
|
||||
new TextEncoder().encode("not a ZIP archive at all"),
|
||||
{
|
||||
code: "unsupported_target",
|
||||
category: "unsupported_target",
|
||||
message: expect.stringContaining("not a ZIP/APK"),
|
||||
details: { path },
|
||||
},
|
||||
);
|
||||
},
|
||||
);
|
||||
@@ -0,0 +1,168 @@
|
||||
import { rm, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect } from "vitest";
|
||||
import { z } from "zod";
|
||||
import { readEvidenceBundle } from "../../../src/application/EvidenceBundleFiles.js";
|
||||
import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
import { evidenceSchema } from "../../../src/domain/evidence.js";
|
||||
import { digestCanonicalValue } from "../../../src/domain/canonicalDigest.js";
|
||||
import { analysisViewResultSchema } from "../../../src/domain/analysisView/analysisView.js";
|
||||
import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js";
|
||||
import {
|
||||
elfExportImage,
|
||||
peExportImage,
|
||||
} from "../../fixtures/binaryDiagnostics/nativeExports.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
|
||||
const exportsContract = toolContract("inspect_native_exports");
|
||||
const viewContract = toolContract("inspect_analysis_view");
|
||||
|
||||
cliTest.for(["pe", "elf"] as const)(
|
||||
"queries retained %s exports through CLI and actual stdio MCP after the source is deleted",
|
||||
{ timeout: 60_000 },
|
||||
async (format, { cli }) => {
|
||||
const root = await createTestTempDirectory("rea-export-views-");
|
||||
const path = join(root, `library.${format}`);
|
||||
const bytes =
|
||||
format === "pe"
|
||||
? peExportImage({
|
||||
functions: [{ rva: 0x2000 }, { forwarder: "other.Target" }],
|
||||
names: [
|
||||
{ name: "selected", index: 0 },
|
||||
{ name: "selected", index: 1 },
|
||||
],
|
||||
})
|
||||
: elfExportImage({
|
||||
symbols: [
|
||||
{ name: "selected", version: 2 },
|
||||
{ name: "selected", version: 0x8002 },
|
||||
],
|
||||
versionDefinitions: ["fixture", "V1"],
|
||||
});
|
||||
await writeFile(path, bytes);
|
||||
const run = cli.sharedCliRun?.bind(cli) ?? cli.run.bind(cli);
|
||||
const complete = await run({
|
||||
arguments: ["inspect-native-exports", path, "--json"],
|
||||
});
|
||||
expect(complete.exitCode, complete.stderr).toBe(0);
|
||||
const parent = exportsContract.outputSchema.parse(complete.json);
|
||||
const bundle = join(root, "complete.bundle.json");
|
||||
const summarized = await run({
|
||||
arguments: [
|
||||
"inspect-native-exports",
|
||||
path,
|
||||
"--detail",
|
||||
"summary",
|
||||
"--evidence_output",
|
||||
bundle,
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
expect(summarized.exitCode, summarized.stderr).toBe(0);
|
||||
const delivered = z
|
||||
.object({ summary: evidenceSchema })
|
||||
.parse(summarized.json).summary;
|
||||
expect(delivered.operation).toBe("inspect_analysis_view");
|
||||
expect(delivered.normalized_result).toMatchObject({
|
||||
parent_evidence_id: parent.evidence_id,
|
||||
summary: { format, export_count: 2, coverage: { status: "complete" } },
|
||||
});
|
||||
const loaded = await readEvidenceBundle(bundle);
|
||||
if (!loaded.ok) throw loaded.error;
|
||||
expect(loaded.value.records).toEqual([parent]);
|
||||
|
||||
const call =
|
||||
cli.sharedMcpCall === undefined
|
||||
? (await connectLocalToolsMcp()).call
|
||||
: await cli.sharedMcpCall();
|
||||
const response = await call("inspect_native_exports", {
|
||||
path,
|
||||
detail: "summary",
|
||||
});
|
||||
expect(response.isError).not.toBe(true);
|
||||
expect(
|
||||
exportsContract.outputSchema.parse(response.structuredContent)
|
||||
.normalized_result,
|
||||
).toEqual(delivered.normalized_result);
|
||||
await rm(path);
|
||||
const source = {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: parent.evidence_id,
|
||||
};
|
||||
const request = {
|
||||
source,
|
||||
view: {
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
names: ["selected"],
|
||||
fields: ["name"],
|
||||
offset: 1,
|
||||
limit: 1,
|
||||
},
|
||||
};
|
||||
const viewed = await call("inspect_analysis_view", request);
|
||||
expect(viewed.isError).not.toBe(true);
|
||||
const page = viewContract.outputSchema.parse(viewed.structuredContent);
|
||||
expect(page.evidence_links).toContain(parent.evidence_id);
|
||||
expect(page.normalized_result).toMatchObject({
|
||||
kind: "native-exports",
|
||||
items: [{ source_index: 1, name: "selected" }],
|
||||
exports: { coverage: { status: "complete" } },
|
||||
coverage: { examined: 1, total: 2, next_offset: null, exhausted: true },
|
||||
});
|
||||
const offline = await run({
|
||||
arguments: [
|
||||
"inspect-analysis-view",
|
||||
JSON.stringify(request),
|
||||
"--evidence-bundle",
|
||||
bundle,
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
expect(offline.exitCode, offline.stderr).toBe(0);
|
||||
expect(viewContract.outputSchema.parse(offline.json)).toEqual(page);
|
||||
const projected = await call("inspect_analysis_view", {
|
||||
...request,
|
||||
view: {
|
||||
...request.view,
|
||||
fields: [format === "elf" ? "value" : "ordinal"],
|
||||
},
|
||||
});
|
||||
expect(projected.isError).not.toBe(true);
|
||||
const projectedView = analysisViewResultSchema.parse(
|
||||
viewContract.outputSchema.parse(projected.structuredContent)
|
||||
.normalized_result,
|
||||
);
|
||||
expect(projectedView).toHaveProperty("items", [
|
||||
{
|
||||
source_index: 1,
|
||||
...(format === "elf"
|
||||
? { value: expect.any(String) }
|
||||
: { ordinal: expect.any(Number) }),
|
||||
},
|
||||
]);
|
||||
const { view_digest, ...content } = projectedView;
|
||||
expect(view_digest).toBe(digestCanonicalValue(content, "Analysis view"));
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"refuses CLI summary without a durable Evidence destination before reading",
|
||||
async ({ cli }) => {
|
||||
const result = await cli.run({
|
||||
arguments: [
|
||||
"inspect-native-exports",
|
||||
"/rea-nonexistent/export.so",
|
||||
"--detail",
|
||||
"summary",
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.json).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: { issues: [{ path: ["evidence_output"] }] },
|
||||
});
|
||||
},
|
||||
);
|
||||
@@ -34,6 +34,7 @@ import {
|
||||
createElfFixture,
|
||||
createIl2cppBinaryFixture,
|
||||
createIl2cppDyldBindFixture,
|
||||
createIl2cppRegistrationTailFixture,
|
||||
createUniversalFixture,
|
||||
FIXTURE_BINARY_BASE,
|
||||
FIXTURE_METHOD_ADDRESSES,
|
||||
@@ -899,6 +900,136 @@ cliTest(
|
||||
30000,
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"retains unknown dyld-bound type names and source identity through CLI and MCP",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-il2cpp-type-bind-");
|
||||
const path = join(root, "global-metadata.dat");
|
||||
const binaryPath = join(root, "GameAssembly.dylib");
|
||||
await writeFile(path, createLinkedIl2cppMetadataFixture().bytes);
|
||||
const inspect = async (bytes: Buffer) => {
|
||||
await writeFile(binaryPath, bytes);
|
||||
const output = await runFileCli(cli, {
|
||||
arguments: [
|
||||
"inspect-il2cpp-metadata",
|
||||
path,
|
||||
"--binary",
|
||||
binaryPath,
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
expect(output.exitCode, output.stderr).toBe(0);
|
||||
const report = parseCliMetadata(output.json);
|
||||
expect(report.binary?.artifact).toEqual({
|
||||
path: binaryPath,
|
||||
sha256: createHash("sha256").update(bytes).digest("hex"),
|
||||
bytes: bytes.length,
|
||||
});
|
||||
expect(await readFile(binaryPath)).toEqual(bytes);
|
||||
return report;
|
||||
};
|
||||
const control = await inspect(createIl2cppBinaryFixture());
|
||||
expect(control.return_types[0]?.name).toBe(FIXTURE_RETURN_TYPE_NAME);
|
||||
const { client, call } = await connectLocalToolsMcp();
|
||||
for (const raw of [
|
||||
0n,
|
||||
FIXTURE_BINARY_BASE + 0x980n,
|
||||
FIXTURE_BINARY_BASE + 0x9b0n,
|
||||
]) {
|
||||
const report = await inspect(createIl2cppDyldBindFixture(raw, 0x900));
|
||||
for (const collection of [
|
||||
"return_types",
|
||||
"field_types",
|
||||
"parameter_types",
|
||||
] as const)
|
||||
expect(report[collection]).toEqual(
|
||||
control[collection].map((row) =>
|
||||
row.type === 0 ? { ...row, name: null } : row,
|
||||
),
|
||||
);
|
||||
expect(report.methods).toEqual(control.methods);
|
||||
expect(report.method_addresses).toEqual(control.method_addresses);
|
||||
expect(report.limitations).toContainEqual(
|
||||
expect.stringContaining("unresolved type-table slot"),
|
||||
);
|
||||
// One transport round trip covers the nullable name contract; all stored
|
||||
// placeholder variants above cross the real CLI workflow.
|
||||
if (raw === 0n) {
|
||||
const response = await call("inspect_il2cpp_metadata", {
|
||||
path,
|
||||
binary_path: binaryPath,
|
||||
detail: "complete",
|
||||
});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
expect(parseCliMetadata(response.structuredContent)).toEqual(report);
|
||||
}
|
||||
}
|
||||
await client.ping();
|
||||
},
|
||||
30000,
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"links ABI count padding and declared zero-fill registration tails through CLI and MCP",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory(
|
||||
"rea-il2cpp-registration-layout-",
|
||||
);
|
||||
const path = join(root, "global-metadata.dat");
|
||||
await writeFile(path, createLinkedIl2cppMetadataFixture().bytes);
|
||||
const { client, call } = await connectLocalToolsMcp();
|
||||
for (const tailBytes of [0, 8, 16] as const) {
|
||||
const bytes = createIl2cppRegistrationTailFixture(tailBytes);
|
||||
bytes.fill(0xff, 0x604, 0x608);
|
||||
const binaryPath = join(root, `registration-${tailBytes}.dylib`);
|
||||
await writeFile(binaryPath, bytes);
|
||||
const output = await runFileCli(cli, {
|
||||
arguments: [
|
||||
"inspect-il2cpp-metadata",
|
||||
path,
|
||||
"--binary",
|
||||
binaryPath,
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
expect(output.exitCode, output.stderr).toBe(0);
|
||||
const report = parseCliMetadata(output.json);
|
||||
expect(report.binary).toMatchObject({
|
||||
code_gen_modules_field: "0x100000600",
|
||||
types_field: "0x100000fb0",
|
||||
types_count: 4,
|
||||
artifact: {
|
||||
path: binaryPath,
|
||||
sha256: createHash("sha256").update(bytes).digest("hex"),
|
||||
bytes: bytes.length,
|
||||
},
|
||||
});
|
||||
expect(report.return_types.map((row) => row.name)).toEqual(
|
||||
Array.from({ length: 5 }, () => FIXTURE_RETURN_TYPE_NAME),
|
||||
);
|
||||
expect(report.method_addresses.map((row) => row.state)).toEqual([
|
||||
"code",
|
||||
"code",
|
||||
"code",
|
||||
"code",
|
||||
"no-body",
|
||||
]);
|
||||
if (tailBytes === 16) {
|
||||
const response = await call("inspect_il2cpp_metadata", {
|
||||
path,
|
||||
binary_path: binaryPath,
|
||||
detail: "complete",
|
||||
});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
expect(parseCliMetadata(response.structuredContent)).toEqual(report);
|
||||
}
|
||||
expect(await readFile(binaryPath)).toEqual(bytes);
|
||||
}
|
||||
await client.ping();
|
||||
},
|
||||
30000,
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"keeps nested IL2CPP generic arguments on their declaring type components",
|
||||
async ({ cli }) => {
|
||||
|
||||
@@ -526,3 +526,31 @@ cliTest(
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"managed CLI retains target frameworks with multi-byte SerString lengths",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-managed-serstring-");
|
||||
for (const length of [0x80, 0x4000]) {
|
||||
const targetFramework = "A".repeat(length);
|
||||
const path = join(root, `framework-${length}.dll`);
|
||||
await writeFile(path, buildManagedPeFixture({ targetFramework }));
|
||||
|
||||
const result = await cli.run({
|
||||
arguments: ["inspect-managed-artifact", path, "--json"],
|
||||
});
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.json).toMatchObject({
|
||||
operation: "inspect_managed_artifact",
|
||||
normalized_result: {
|
||||
target_frameworks: [targetFramework],
|
||||
attributes: [
|
||||
expect.objectContaining({ decoded_fixed_string: targetFramework }),
|
||||
],
|
||||
coverage: { state: "complete", issues: [] },
|
||||
},
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
@@ -0,0 +1,301 @@
|
||||
import { create, toBinary } from "@bufbuild/protobuf";
|
||||
import {
|
||||
DescriptorProtoSchema,
|
||||
FieldDescriptorProtoSchema,
|
||||
FileDescriptorProtoSchema,
|
||||
FileDescriptorSetSchema,
|
||||
FileOptionsSchema,
|
||||
} from "@bufbuild/protobuf/wkt";
|
||||
import { BinaryWriter, WireType } from "@bufbuild/protobuf/wire";
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { analysisViewResultSchema } from "../../../src/domain/analysisView/analysisView.js";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { webNetworkCaptureSchema } from "../../../src/domain/webNetworkCapture.js";
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import { protobufHar } from "../../fixtures/protobufCapture.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
|
||||
const sha256 = (bytes: Uint8Array) =>
|
||||
createHash("sha256").update(bytes).digest("hex");
|
||||
const body = Buffer.from("0a020801", "hex"); // Message { payload { value=1 } }
|
||||
const descriptorDefinition = () =>
|
||||
create(FileDescriptorSetSchema, {
|
||||
file: [
|
||||
{
|
||||
name: "shared.proto",
|
||||
package: "capture",
|
||||
syntax: "proto3",
|
||||
messageType: [
|
||||
{
|
||||
name: "Payload",
|
||||
field: [{ name: "value", number: 1, type: 5, label: 1 }],
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "entry.proto",
|
||||
package: "entry",
|
||||
syntax: "proto3",
|
||||
dependency: ["shared.proto"],
|
||||
messageType: [
|
||||
{
|
||||
name: "Message",
|
||||
field: [
|
||||
{
|
||||
name: "payload",
|
||||
number: 1,
|
||||
type: 11,
|
||||
typeName: ".capture.Payload",
|
||||
label: 1,
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
const conflictingFiles = (change: string, reverse: boolean) => {
|
||||
const definition = descriptorDefinition();
|
||||
const shared = definition.file[0]!;
|
||||
const duplicate = structuredClone(shared);
|
||||
let captured = body;
|
||||
if (change === "declarations") duplicate.messageType[0]!.name = "Other";
|
||||
else if (change === "package") duplicate.package = "other";
|
||||
else if (change === "imports") {
|
||||
definition.file.unshift(
|
||||
create(FileDescriptorProtoSchema, { name: "a.proto", syntax: "proto3" }),
|
||||
create(FileDescriptorProtoSchema, { name: "b.proto", syntax: "proto3" }),
|
||||
);
|
||||
shared.messageType = [];
|
||||
duplicate.messageType = [];
|
||||
shared.dependency = ["a.proto"];
|
||||
duplicate.dependency = ["b.proto"];
|
||||
definition.file.at(-1)!.messageType[0]!.field[0] = create(
|
||||
FieldDescriptorProtoSchema,
|
||||
{ name: "value", number: 1, type: 5, label: 1 },
|
||||
);
|
||||
captured = Buffer.from([8, 1]);
|
||||
} else if (change === "presence") {
|
||||
shared.options = create(FileOptionsSchema);
|
||||
duplicate.options = create(FileOptionsSchema, { javaMultipleFiles: false });
|
||||
} else if (change === "opaque-file") {
|
||||
duplicate.$unknown = [
|
||||
{ no: 50000, wireType: WireType.Varint, data: Uint8Array.of(1) },
|
||||
];
|
||||
} else {
|
||||
shared.options = create(FileOptionsSchema);
|
||||
duplicate.options = create(FileOptionsSchema);
|
||||
shared.options.$unknown = [
|
||||
{ no: 50000, wireType: WireType.Varint, data: Uint8Array.of(1) },
|
||||
];
|
||||
duplicate.options.$unknown = [
|
||||
{ no: 50000, wireType: WireType.Varint, data: Uint8Array.of(2) },
|
||||
];
|
||||
}
|
||||
const sharedIndex = definition.file.indexOf(shared);
|
||||
definition.file.splice(
|
||||
sharedIndex,
|
||||
1,
|
||||
...(reverse ? [duplicate, shared] : [shared, duplicate]),
|
||||
);
|
||||
return { bytes: toBinary(FileDescriptorSetSchema, definition), captured };
|
||||
};
|
||||
|
||||
const retainedDetailsSchema = z.object({
|
||||
partial_observation: z.strictObject({
|
||||
kind: z.literal("retained-evidence"),
|
||||
evidence_id: z.string(),
|
||||
}),
|
||||
});
|
||||
|
||||
cliTest(
|
||||
"refuses conflicting descriptor file identities before registry import lookup through CLI/MCP",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-protobuf-file-conflicts-");
|
||||
const call = await cli.sharedMcpCall!();
|
||||
for (const change of [
|
||||
"declarations",
|
||||
"package",
|
||||
"imports",
|
||||
"presence",
|
||||
"opaque-file",
|
||||
"opaque-option",
|
||||
])
|
||||
for (const reverse of [false, true]) {
|
||||
const { bytes, captured } = conflictingFiles(change, reverse);
|
||||
const descriptor = join(root, `${change}-${reverse}.pb`);
|
||||
const capture = join(root, `${change}-${reverse}.har`);
|
||||
const text = protobufHar(captured, "application/x-protobuf");
|
||||
await writeFile(descriptor, bytes);
|
||||
await writeFile(capture, text);
|
||||
const expected = {
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
issues: [
|
||||
{
|
||||
path: ["protobuf", "schema", "descriptor_set_path"],
|
||||
reason: "invalid_format",
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
if (change === "declarations") {
|
||||
const output = await cli.sharedCliRun!({
|
||||
arguments: [
|
||||
"inspect-web-network-capture",
|
||||
capture,
|
||||
"har",
|
||||
"--record",
|
||||
"0",
|
||||
"--protobuf-schema",
|
||||
descriptor,
|
||||
"--protobuf-message",
|
||||
"entry.Message",
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
expect(output.exitCode).toBe(1);
|
||||
expect(output.json).toMatchObject(expected);
|
||||
}
|
||||
const response = await call("inspect_web_network_capture", {
|
||||
capture_path: capture,
|
||||
format: "har",
|
||||
record_ordinals: [0],
|
||||
protobuf: {
|
||||
schema: {
|
||||
descriptor_set_path: descriptor,
|
||||
message_type: "entry.Message",
|
||||
},
|
||||
},
|
||||
});
|
||||
const error = parseMcpToolError(response).error;
|
||||
expect(error, `${change}-${reverse}`).toMatchObject(expected);
|
||||
if (change === "declarations") {
|
||||
const source = retainedDetailsSchema.parse(
|
||||
error.details,
|
||||
).partial_observation;
|
||||
const retained = await call("inspect_analysis_view", {
|
||||
source,
|
||||
view: { kind: "summary" },
|
||||
});
|
||||
expect(retained.isError, JSON.stringify(retained)).not.toBe(true);
|
||||
expect(
|
||||
analysisViewResultSchema.parse(
|
||||
parseEvidence(retained.structuredContent).normalized_result,
|
||||
),
|
||||
).toMatchObject({
|
||||
summary: {
|
||||
artifact: { path: capture, sha256: sha256(Buffer.from(text)) },
|
||||
retained_records: 1,
|
||||
protobuf_coverage: { status: "partial", inspected_records: 0 },
|
||||
},
|
||||
});
|
||||
}
|
||||
expect(await readFile(descriptor)).toEqual(Buffer.from(bytes));
|
||||
expect(await readFile(capture, "utf8")).toBe(text);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
/** Reorder known FileDescriptorProto fields without changing their facts. */
|
||||
const reorderedSharedFile = () => {
|
||||
const shared = descriptorDefinition().file[0]!;
|
||||
return new BinaryWriter()
|
||||
.tag(12, WireType.LengthDelimited)
|
||||
.string(shared.syntax)
|
||||
.tag(4, WireType.LengthDelimited)
|
||||
.bytes(toBinary(DescriptorProtoSchema, shared.messageType[0]!))
|
||||
.tag(2, WireType.LengthDelimited)
|
||||
.string(shared.package)
|
||||
.tag(1, WireType.LengthDelimited)
|
||||
.string(shared.name)
|
||||
.finish();
|
||||
};
|
||||
|
||||
cliTest(
|
||||
"coalesces equal descriptor file facts while preserving original descriptor identity through CLI/MCP",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-protobuf-file-repeat-");
|
||||
const capture = join(root, "capture.har");
|
||||
await writeFile(capture, protobufHar(body, "application/x-protobuf"));
|
||||
const call = await cli.sharedMcpCall!();
|
||||
for (const mode of ["identical", "reordered", "opaque"]) {
|
||||
const definition = descriptorDefinition();
|
||||
const shared = definition.file[0]!;
|
||||
if (mode === "opaque") {
|
||||
shared.options = create(FileOptionsSchema);
|
||||
shared.options.$unknown = [
|
||||
{ no: 50000, wireType: WireType.Varint, data: Uint8Array.of(7) },
|
||||
];
|
||||
}
|
||||
const duplicate =
|
||||
mode === "reordered"
|
||||
? reorderedSharedFile()
|
||||
: toBinary(FileDescriptorProtoSchema, shared);
|
||||
const writer = new BinaryWriter();
|
||||
writer
|
||||
.tag(1, WireType.LengthDelimited)
|
||||
.bytes(toBinary(FileDescriptorProtoSchema, shared));
|
||||
writer.tag(1, WireType.LengthDelimited).bytes(duplicate);
|
||||
writer
|
||||
.tag(1, WireType.LengthDelimited)
|
||||
.bytes(toBinary(FileDescriptorProtoSchema, definition.file[1]!));
|
||||
// Repeated imported facts after their consumer must coalesce as well.
|
||||
writer.tag(1, WireType.LengthDelimited).bytes(duplicate);
|
||||
const bytes = writer.finish();
|
||||
const descriptor = join(root, `${mode}.pb`);
|
||||
await writeFile(descriptor, bytes);
|
||||
const output = await cli.sharedCliRun!({
|
||||
arguments: [
|
||||
"inspect-web-network-capture",
|
||||
capture,
|
||||
"har",
|
||||
"--record",
|
||||
"0",
|
||||
"--protobuf-schema",
|
||||
descriptor,
|
||||
"--protobuf-message",
|
||||
"entry.Message",
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
expect(output.exitCode, output.stdout + output.stderr).toBe(0);
|
||||
const evidence = parseEvidence(output.json);
|
||||
const report = webNetworkCaptureSchema.parse(evidence.normalized_result);
|
||||
expect(report.protobuf_coverage?.status).toBe("complete");
|
||||
const decoded = report.records[0]?.protobuf?.[0];
|
||||
expect(decoded?.schema).toMatchObject({
|
||||
descriptor_set_path: descriptor,
|
||||
descriptor_set_sha256: sha256(bytes),
|
||||
descriptor_set_bytes: bytes.length,
|
||||
});
|
||||
expect(decoded?.fields).toMatchObject([
|
||||
{ number: 1, schema_name: "payload", schema_type: "capture.Payload" },
|
||||
{ number: 1, schema_name: "value", schema_value: "1" },
|
||||
]);
|
||||
const response = await call("inspect_web_network_capture", {
|
||||
capture_path: capture,
|
||||
format: "har",
|
||||
record_ordinals: [0],
|
||||
protobuf: {
|
||||
schema: {
|
||||
descriptor_set_path: descriptor,
|
||||
message_type: "entry.Message",
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
expect(parseEvidence(response.structuredContent).evidence_id).toBe(
|
||||
evidence.evidence_id,
|
||||
);
|
||||
expect(await readFile(descriptor)).toEqual(Buffer.from(bytes));
|
||||
}
|
||||
},
|
||||
);
|
||||
@@ -0,0 +1,262 @@
|
||||
import { create, toBinary } from "@bufbuild/protobuf";
|
||||
import {
|
||||
FieldDescriptorProtoSchema,
|
||||
FileDescriptorSetSchema,
|
||||
OneofDescriptorProtoSchema,
|
||||
} from "@bufbuild/protobuf/wkt";
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { analysisViewResultSchema } from "../../../src/domain/analysisView/analysisView.js";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { webNetworkCaptureSchema } from "../../../src/domain/webNetworkCapture.js";
|
||||
import { protobufHar } from "../../fixtures/protobufCapture.js";
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
|
||||
const sha256 = (bytes: string | Uint8Array) =>
|
||||
createHash("sha256").update(bytes).digest("hex");
|
||||
// Two occurrences of first=1, second=3, then child { value=4 }.
|
||||
const body = Buffer.from("0801080210031a020804", "hex");
|
||||
const descriptorDefinition = () =>
|
||||
create(FileDescriptorSetSchema, {
|
||||
file: [
|
||||
{
|
||||
name: "identity.proto",
|
||||
package: "capture",
|
||||
syntax: "proto3",
|
||||
messageType: [
|
||||
{
|
||||
name: "Message",
|
||||
field: [
|
||||
{ name: "first", number: 1, type: 5, label: 3 },
|
||||
{ name: "second", number: 2, type: 5, label: 1 },
|
||||
{
|
||||
name: "child",
|
||||
number: 3,
|
||||
type: 11,
|
||||
typeName: ".capture.Message.Child",
|
||||
label: 1,
|
||||
},
|
||||
],
|
||||
nestedType: [
|
||||
{
|
||||
name: "Child",
|
||||
field: [{ name: "value", number: 1, type: 5, label: 1 }],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
const retainedDetailsSchema = z.object({
|
||||
partial_observation: z.strictObject({
|
||||
kind: z.literal("retained-evidence"),
|
||||
evidence_id: z.string(),
|
||||
}),
|
||||
});
|
||||
|
||||
cliTest(
|
||||
"refuses ambiguous descriptor field identities through CLI/MCP while retaining source Evidence",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-protobuf-field-identity-");
|
||||
const capture = join(root, "capture.har");
|
||||
const text = protobufHar(body, "application/x-protobuf");
|
||||
await writeFile(capture, text);
|
||||
const call = await cli.sharedMcpCall!();
|
||||
for (const variant of ["forward", "reverse", "nested", "oneof"]) {
|
||||
const definition = descriptorDefinition();
|
||||
const message = definition.file[0]!.messageType[0]!;
|
||||
if (variant === "nested") {
|
||||
message.nestedType[0]!.field.push(
|
||||
create(FieldDescriptorProtoSchema, {
|
||||
name: "alias",
|
||||
number: 1,
|
||||
type: 5,
|
||||
label: 1,
|
||||
}),
|
||||
);
|
||||
} else if (variant === "oneof") {
|
||||
message.oneofDecl.push(
|
||||
create(OneofDescriptorProtoSchema, { name: "choice" }),
|
||||
);
|
||||
message.field.push(
|
||||
create(FieldDescriptorProtoSchema, {
|
||||
name: "alias",
|
||||
number: 1,
|
||||
type: 5,
|
||||
label: 1,
|
||||
oneofIndex: 0,
|
||||
}),
|
||||
);
|
||||
} else {
|
||||
message.field[1]!.number = 1;
|
||||
if (variant === "reverse") message.field.reverse();
|
||||
}
|
||||
const descriptor = join(root, `${variant}.pb`);
|
||||
const bytes = toBinary(FileDescriptorSetSchema, definition);
|
||||
await writeFile(descriptor, bytes);
|
||||
const output = await cli.sharedCliRun!({
|
||||
arguments: [
|
||||
"inspect-web-network-capture",
|
||||
capture,
|
||||
"har",
|
||||
"--record",
|
||||
"0",
|
||||
"--protobuf-schema",
|
||||
descriptor,
|
||||
"--protobuf-message",
|
||||
"capture.Message",
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
const expected = {
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
issues: [
|
||||
{
|
||||
path: ["protobuf", "schema", "descriptor_set_path"],
|
||||
reason: "invalid_format",
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
expect(output.exitCode, variant).toBe(1);
|
||||
expect(output.json, variant).toMatchObject(expected);
|
||||
const response = await call("inspect_web_network_capture", {
|
||||
capture_path: capture,
|
||||
format: "har",
|
||||
record_ordinals: [0],
|
||||
protobuf: {
|
||||
schema: {
|
||||
descriptor_set_path: descriptor,
|
||||
message_type: "capture.Message",
|
||||
},
|
||||
},
|
||||
});
|
||||
const error = parseMcpToolError(response).error;
|
||||
expect(error, variant).toMatchObject(expected);
|
||||
const source = retainedDetailsSchema.parse(
|
||||
error.details,
|
||||
).partial_observation;
|
||||
const retained = await call("inspect_analysis_view", {
|
||||
source,
|
||||
view: { kind: "summary" },
|
||||
});
|
||||
expect(retained.isError, JSON.stringify(retained)).not.toBe(true);
|
||||
expect(
|
||||
analysisViewResultSchema.parse(
|
||||
parseEvidence(retained.structuredContent).normalized_result,
|
||||
),
|
||||
).toMatchObject({
|
||||
summary: {
|
||||
artifact: {
|
||||
path: capture,
|
||||
bytes: Buffer.byteLength(text),
|
||||
sha256: sha256(text),
|
||||
},
|
||||
retained_records: 1,
|
||||
protobuf_coverage: { status: "partial", inspected_records: 0 },
|
||||
},
|
||||
});
|
||||
expect(await readFile(descriptor)).toEqual(Buffer.from(bytes));
|
||||
}
|
||||
expect(await readFile(capture, "utf8")).toBe(text);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"keeps repeated wire occurrences and message-local number reuse through CLI/MCP",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-protobuf-field-control-");
|
||||
const capture = join(root, "capture.har");
|
||||
const descriptor = join(root, "valid.pb");
|
||||
const definition = toBinary(
|
||||
FileDescriptorSetSchema,
|
||||
descriptorDefinition(),
|
||||
);
|
||||
await writeFile(capture, protobufHar(body, "application/x-protobuf"));
|
||||
await writeFile(descriptor, definition);
|
||||
const output = await cli.sharedCliRun!({
|
||||
arguments: [
|
||||
"inspect-web-network-capture",
|
||||
capture,
|
||||
"har",
|
||||
"--record",
|
||||
"0",
|
||||
"--protobuf-schema",
|
||||
descriptor,
|
||||
"--protobuf-message",
|
||||
"capture.Message",
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
expect(output.exitCode, output.stdout + output.stderr).toBe(0);
|
||||
const evidence = parseEvidence(output.json);
|
||||
const report = webNetworkCaptureSchema.parse(evidence.normalized_result);
|
||||
expect(report.protobuf_coverage?.status).toBe("complete");
|
||||
const decoded = report.records[0]?.protobuf?.[0];
|
||||
expect(decoded?.schema).toMatchObject({
|
||||
descriptor_set_path: descriptor,
|
||||
descriptor_set_sha256: sha256(definition),
|
||||
descriptor_set_bytes: definition.length,
|
||||
message_type: "capture.Message",
|
||||
});
|
||||
expect(decoded?.fields).toMatchObject([
|
||||
{
|
||||
field_id: "offset:0",
|
||||
number: 1,
|
||||
schema_name: "first",
|
||||
schema_value: "1",
|
||||
},
|
||||
{
|
||||
field_id: "offset:2",
|
||||
number: 1,
|
||||
schema_name: "first",
|
||||
schema_value: "2",
|
||||
},
|
||||
{
|
||||
field_id: "offset:4",
|
||||
number: 2,
|
||||
schema_name: "second",
|
||||
schema_value: "3",
|
||||
},
|
||||
{
|
||||
field_id: "offset:6",
|
||||
number: 3,
|
||||
schema_name: "child",
|
||||
schema_type: "capture.Message.Child",
|
||||
},
|
||||
{
|
||||
field_id: "offset:8",
|
||||
number: 1,
|
||||
schema_name: "value",
|
||||
schema_value: "4",
|
||||
},
|
||||
]);
|
||||
expect(decoded?.fields).toHaveLength(5);
|
||||
const call = await cli.sharedMcpCall!();
|
||||
const response = await call("inspect_web_network_capture", {
|
||||
capture_path: capture,
|
||||
format: "har",
|
||||
record_ordinals: [0],
|
||||
protobuf: {
|
||||
schema: {
|
||||
descriptor_set_path: descriptor,
|
||||
message_type: "capture.Message",
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
expect(parseEvidence(response.structuredContent).evidence_id).toBe(
|
||||
evidence.evidence_id,
|
||||
);
|
||||
expect(await readFile(descriptor)).toEqual(Buffer.from(definition));
|
||||
},
|
||||
);
|
||||
@@ -0,0 +1,221 @@
|
||||
import { create, toBinary } from "@bufbuild/protobuf";
|
||||
import {
|
||||
DescriptorProtoSchema,
|
||||
EnumDescriptorProtoSchema,
|
||||
FileDescriptorSetSchema,
|
||||
ServiceDescriptorProtoSchema,
|
||||
} from "@bufbuild/protobuf/wkt";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect } from "vitest";
|
||||
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { webNetworkCaptureSchema } from "../../../src/domain/webNetworkCapture.js";
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import { protobufHar } from "../../fixtures/protobufCapture.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
|
||||
const scalarMessage = (name: string, fieldName: string) =>
|
||||
create(DescriptorProtoSchema, {
|
||||
name,
|
||||
field: [{ name: fieldName, number: 1, type: 5, label: 1 }],
|
||||
});
|
||||
const descriptorDefinition = () =>
|
||||
create(FileDescriptorSetSchema, {
|
||||
file: [
|
||||
{
|
||||
name: "identity.proto",
|
||||
package: "capture",
|
||||
syntax: "proto3",
|
||||
messageType: [scalarMessage("Message", "alpha")],
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
const collidingDescriptor = (scope: string, reverse: boolean) => {
|
||||
const definition = descriptorDefinition();
|
||||
const file = definition.file[0]!;
|
||||
let messageType = "capture.Message";
|
||||
if (scope === "same-file") {
|
||||
file.messageType.push(scalarMessage("Message", "beta"));
|
||||
if (reverse) file.messageType.reverse();
|
||||
} else if (["across-files", "enum", "service"].includes(scope)) {
|
||||
const other = structuredClone(file);
|
||||
other.name = "other.proto";
|
||||
if (scope === "enum") {
|
||||
other.messageType = [];
|
||||
other.enumType.push(
|
||||
create(EnumDescriptorProtoSchema, {
|
||||
name: "Message",
|
||||
value: [{ name: "ZERO", number: 0 }],
|
||||
}),
|
||||
);
|
||||
} else if (scope === "service") {
|
||||
other.messageType = [];
|
||||
other.service.push(
|
||||
create(ServiceDescriptorProtoSchema, { name: "Message" }),
|
||||
);
|
||||
} else other.messageType[0]!.field[0]!.name = "beta";
|
||||
definition.file.push(other);
|
||||
if (reverse) definition.file.reverse();
|
||||
} else {
|
||||
const alternative =
|
||||
scope === "map-entry"
|
||||
? create(DescriptorProtoSchema, {
|
||||
name: "Message",
|
||||
options: { mapEntry: true },
|
||||
field: [
|
||||
{ name: "key", number: 1, type: 9, label: 1 },
|
||||
{ name: "value", number: 2, type: 5, label: 1 },
|
||||
],
|
||||
})
|
||||
: scalarMessage("Message", "beta");
|
||||
const children = [file.messageType[0]!, alternative];
|
||||
if (reverse) children.reverse();
|
||||
file.messageType = [
|
||||
create(DescriptorProtoSchema, { name: "Outer", nestedType: children }),
|
||||
];
|
||||
messageType = "capture.Outer.Message";
|
||||
}
|
||||
return { bytes: toBinary(FileDescriptorSetSchema, definition), messageType };
|
||||
};
|
||||
|
||||
cliTest(
|
||||
"refuses ambiguous registry identities independently of declaration order through CLI/MCP",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory(
|
||||
"rea-protobuf-registry-identity-",
|
||||
);
|
||||
const capture = join(root, "capture.har");
|
||||
const text = protobufHar(Buffer.from([8, 1]), "application/x-protobuf");
|
||||
await writeFile(capture, text);
|
||||
const call = await cli.sharedMcpCall!();
|
||||
for (const scope of [
|
||||
"same-file",
|
||||
"across-files",
|
||||
"nested",
|
||||
"enum",
|
||||
"service",
|
||||
"map-entry",
|
||||
])
|
||||
for (const reverse of [false, true]) {
|
||||
const { bytes, messageType } = collidingDescriptor(scope, reverse);
|
||||
const descriptor = join(root, `${scope}-${reverse}.pb`);
|
||||
await writeFile(descriptor, bytes);
|
||||
const expected = {
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
issues: [
|
||||
{
|
||||
path: ["protobuf", "schema", "descriptor_set_path"],
|
||||
reason: "invalid_format",
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
// Both orders cross the actual CLI adapter; other geometries reuse
|
||||
// the production MCP session and the same application workflow.
|
||||
if (scope === "same-file") {
|
||||
const output = await cli.sharedCliRun!({
|
||||
arguments: [
|
||||
"inspect-web-network-capture",
|
||||
capture,
|
||||
"har",
|
||||
"--record",
|
||||
"0",
|
||||
"--protobuf-schema",
|
||||
descriptor,
|
||||
"--protobuf-message",
|
||||
messageType,
|
||||
"--json",
|
||||
],
|
||||
});
|
||||
expect(output.exitCode).toBe(1);
|
||||
expect(output.json).toMatchObject(expected);
|
||||
}
|
||||
const response = await call("inspect_web_network_capture", {
|
||||
capture_path: capture,
|
||||
format: "har",
|
||||
record_ordinals: [0],
|
||||
protobuf: {
|
||||
schema: {
|
||||
descriptor_set_path: descriptor,
|
||||
message_type: messageType,
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(
|
||||
parseMcpToolError(response).error,
|
||||
`${scope}-${reverse}`,
|
||||
).toMatchObject(expected);
|
||||
expect(await readFile(descriptor)).toEqual(Buffer.from(bytes));
|
||||
}
|
||||
expect(await readFile(capture, "utf8")).toBe(text);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"keeps empty-package, package, parent and case-sensitive registry identities distinct",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-protobuf-registry-scopes-");
|
||||
const capture = join(root, "capture.har");
|
||||
const descriptor = join(root, "scopes.pb");
|
||||
const definition = create(FileDescriptorSetSchema, {
|
||||
file: [
|
||||
{
|
||||
name: "root.proto",
|
||||
syntax: "proto3",
|
||||
messageType: [
|
||||
scalarMessage("Message", "root_value"),
|
||||
scalarMessage("message", "case_value"),
|
||||
create(DescriptorProtoSchema, {
|
||||
name: "Outer",
|
||||
nestedType: [scalarMessage("Message", "nested_value")],
|
||||
}),
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "package.proto",
|
||||
package: "capture",
|
||||
syntax: "proto3",
|
||||
messageType: [scalarMessage("Message", "package_value")],
|
||||
},
|
||||
],
|
||||
});
|
||||
const bytes = toBinary(FileDescriptorSetSchema, definition);
|
||||
await writeFile(
|
||||
capture,
|
||||
protobufHar(Buffer.from([8, 1]), "application/x-protobuf"),
|
||||
);
|
||||
await writeFile(descriptor, bytes);
|
||||
const call = await cli.sharedMcpCall!();
|
||||
for (const [messageType, fieldName] of [
|
||||
["Message", "root_value"],
|
||||
["message", "case_value"],
|
||||
["Outer.Message", "nested_value"],
|
||||
["capture.Message", "package_value"],
|
||||
] as const) {
|
||||
const response = await call("inspect_web_network_capture", {
|
||||
capture_path: capture,
|
||||
format: "har",
|
||||
record_ordinals: [0],
|
||||
protobuf: {
|
||||
schema: {
|
||||
descriptor_set_path: descriptor,
|
||||
message_type: messageType,
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
const report = webNetworkCaptureSchema.parse(
|
||||
parseEvidence(response.structuredContent).normalized_result,
|
||||
);
|
||||
expect(report.protobuf_coverage?.status).toBe("complete");
|
||||
expect(report.records[0]?.protobuf?.[0]?.fields).toMatchObject([
|
||||
{ number: 1, schema_name: fieldName, schema_value: "1" },
|
||||
]);
|
||||
}
|
||||
expect(await readFile(descriptor)).toEqual(Buffer.from(bytes));
|
||||
},
|
||||
);
|
||||
@@ -0,0 +1,154 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js";
|
||||
import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { reconstructionVerificationResultSchema } from "../../../src/domain/reconstructionVerificationSchemas.js";
|
||||
import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js";
|
||||
|
||||
const unchangedRight = () => {
|
||||
const left = FUNCTION_COMPARISON_EXAMPLE.left;
|
||||
return createEvidence(
|
||||
{ path: "/fixture/rebuilt", sha256: "2".repeat(64), format: "mach-o" },
|
||||
left.provider,
|
||||
{
|
||||
operation: left.operation,
|
||||
parameters: left.parameters,
|
||||
result: left.normalized_result,
|
||||
confidence: left.confidence,
|
||||
authority: left.authority,
|
||||
relationshipCoverage: { status: "complete" },
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
const verify = async (
|
||||
call: Awaited<ReturnType<typeof connectLocalToolsMcp>>["call"],
|
||||
comparisonId: string,
|
||||
dimensions: readonly ("overall" | "pseudocode" | "assembly")[],
|
||||
) => {
|
||||
const response = await call("verify_reconstruction", {
|
||||
specification: {
|
||||
name: "Scoped function uncertainty",
|
||||
claims: dimensions.map((dimension) => ({
|
||||
kind: "structural-function",
|
||||
claim_id: dimension,
|
||||
title: `Check ${dimension}`,
|
||||
comparison_evidence_id: comparisonId,
|
||||
dimension,
|
||||
})),
|
||||
},
|
||||
});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
return reconstructionVerificationResultSchema.parse(
|
||||
parseEvidence(response.structuredContent).normalized_result,
|
||||
);
|
||||
};
|
||||
|
||||
it.each(["pseudocode", "overall"] as const)(
|
||||
"keeps an overall claim unknown for an active %s question through stdio MCP",
|
||||
async (dimension) => {
|
||||
const { call } = await connectLocalToolsMcp();
|
||||
const left = FUNCTION_COMPARISON_EXAMPLE.left;
|
||||
const right = unchangedRight();
|
||||
const compared = await call("compare_functions", { left, right });
|
||||
expect(compared.isError, JSON.stringify(compared)).not.toBe(true);
|
||||
const comparison = parseEvidence(compared.structuredContent);
|
||||
expect(comparison.normalized_result).toMatchObject({ status: "unchanged" });
|
||||
const selectors = ["overall", "pseudocode", "assembly"] as const;
|
||||
expect(await verify(call, comparison.evidence_id, selectors)).toMatchObject(
|
||||
{
|
||||
status: "pass",
|
||||
summary: { total: 3, passed: 3, unknown: 0 },
|
||||
},
|
||||
);
|
||||
|
||||
const probe = {
|
||||
operation: "analyze_function",
|
||||
rationale: "Check the unresolved pseudocode condition.",
|
||||
};
|
||||
const recorded = await call("record_unknown", {
|
||||
question: "Does the compared implementation preserve the condition?",
|
||||
severity: "high",
|
||||
domain: "function-comparison",
|
||||
supporting_evidence_ids: [comparison.evidence_id],
|
||||
contradicting_evidence_ids: [],
|
||||
required_authority: null,
|
||||
required_confidence: "derived",
|
||||
required_environment: null,
|
||||
recommended_probes: [probe],
|
||||
relationships: [],
|
||||
comparison_conclusion_scope: {
|
||||
comparison_evidence_id: comparison.evidence_id,
|
||||
dimensions: [dimension],
|
||||
},
|
||||
});
|
||||
expect(recorded.isError, JSON.stringify(recorded)).not.toBe(true);
|
||||
const unknown = toolContract("record_unknown").outputSchema.parse(
|
||||
recorded.structuredContent,
|
||||
).result;
|
||||
const aggregate = await verify(call, comparison.evidence_id, ["overall"]);
|
||||
expect(aggregate).toMatchObject({
|
||||
status: "unknown",
|
||||
claims: {
|
||||
items: [
|
||||
{ observed_status: "unchanged", unknown_ids: [unknown.unknown_id] },
|
||||
],
|
||||
},
|
||||
recommended_probes: [
|
||||
{ ...probe, claim_ids: ["overall"], unknown_ids: [unknown.unknown_id] },
|
||||
],
|
||||
});
|
||||
const selected = await verify(call, comparison.evidence_id, selectors);
|
||||
expect(selected.claims.items).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
claim_id: "assembly",
|
||||
status: "pass",
|
||||
unknown_ids: [],
|
||||
}),
|
||||
expect.objectContaining({
|
||||
claim_id: "pseudocode",
|
||||
status: dimension === "pseudocode" ? "unknown" : "pass",
|
||||
unknown_ids: dimension === "pseudocode" ? [unknown.unknown_id] : [],
|
||||
}),
|
||||
]),
|
||||
);
|
||||
|
||||
const resolved = await call("update_unknown", {
|
||||
unknown_id: unknown.unknown_id,
|
||||
expected_revision: unknown.revision,
|
||||
status: "resolved",
|
||||
severity: unknown.severity,
|
||||
supporting_evidence_ids: unknown.supporting_evidence_ids,
|
||||
contradicting_evidence_ids: unknown.contradicting_evidence_ids,
|
||||
required_authority: unknown.required_authority,
|
||||
required_confidence: unknown.required_confidence,
|
||||
required_environment: unknown.required_environment,
|
||||
recommended_probes: unknown.recommended_probes,
|
||||
relationships: unknown.relationships,
|
||||
resolution: {
|
||||
disposition: "withdrawn",
|
||||
rationale: "Question explicitly withdrawn for this control.",
|
||||
evidence_ids: [],
|
||||
},
|
||||
});
|
||||
expect(resolved.isError, JSON.stringify(resolved)).not.toBe(true);
|
||||
expect(
|
||||
await verify(call, comparison.evidence_id, ["overall"]),
|
||||
).toMatchObject({
|
||||
status: "pass",
|
||||
claims: { items: [{ unknown_ids: [] }] },
|
||||
});
|
||||
const exported = await call("get_evidence_bundle", { detail: "complete" });
|
||||
const bundle = toolContract("get_evidence_bundle").outputSchema.parse(
|
||||
exported.structuredContent,
|
||||
).result;
|
||||
for (const source of [left, right, comparison])
|
||||
expect(
|
||||
bundle.records.find(
|
||||
({ evidence_id }) => evidence_id === source.evidence_id,
|
||||
),
|
||||
).toEqual(source);
|
||||
},
|
||||
);
|
||||
@@ -10,8 +10,103 @@ import { validateRecordedCrashSources } from "../../../src/native/pwntools/Recor
|
||||
import {
|
||||
recordedCrashSchema,
|
||||
RECORDED_LINUX_CORE_PROFILES,
|
||||
type RecordedCrashArchitecture,
|
||||
} from "../../../src/domain/native/recordedCrash.js";
|
||||
|
||||
/** Header-only metadata carrier; no thread, unwind, or MTE payload is decoded. */
|
||||
const auxiliaryHeader = (
|
||||
architecture: RecordedCrashArchitecture,
|
||||
rawType: number,
|
||||
reportedType: string | number,
|
||||
) => {
|
||||
const report = recordedCrashFixture(undefined, architecture);
|
||||
report.segments = report.segments.map((segment) => ({
|
||||
...segment,
|
||||
type: reportedType,
|
||||
type_id: rawType,
|
||||
file_size: "0x4",
|
||||
memory_size: "0x4",
|
||||
}));
|
||||
report.notes = [];
|
||||
report.threads = [];
|
||||
const snapshot = recordedCrashFixtureBytes(report);
|
||||
snapshot.writeUInt32LE(rawType, 64);
|
||||
return { report: recordedCrashSchema.parse(report), snapshot };
|
||||
};
|
||||
|
||||
it.each([
|
||||
["aarch64-little-endian", 0x70000000, "PT_AARCH64_ARCHEXT"],
|
||||
["aarch64-little-endian", 0x70000001, "PT_AARCH64_UNWIND"],
|
||||
// The ABI names MTE, but pinned pyelftools still reports its numeric type.
|
||||
["aarch64-little-endian", 0x70000002, 0x70000002],
|
||||
["aarch64-little-endian", 0x70000003, 0x70000003],
|
||||
["x86_64-little-endian", 0x70000000, 0x70000000],
|
||||
["x86_64-little-endian", 0x70000001, 0x70000001],
|
||||
["aarch64-little-endian", 0x6474e551, "PT_GNU_STACK"],
|
||||
["x86_64-little-endian", 0x6474e551, "PT_GNU_STACK"],
|
||||
] as const)(
|
||||
"binds %s program type %s to its exact parser projection %s",
|
||||
(architecture, rawType, reportedType) => {
|
||||
const { report, snapshot } = auxiliaryHeader(
|
||||
architecture,
|
||||
rawType,
|
||||
reportedType,
|
||||
);
|
||||
const original = Buffer.from(snapshot);
|
||||
expect(() =>
|
||||
validateRecordedCrashStructure(
|
||||
report,
|
||||
snapshot,
|
||||
report.decoder_diagnostics,
|
||||
),
|
||||
).not.toThrow();
|
||||
expect(snapshot).toEqual(original);
|
||||
expect(report.note_interpretation_completeness).toBe("unknown");
|
||||
expect(report.live_process_identity).toBe("unknown");
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
["aarch64-little-endian", 0x70000000, "PT_AARCH64_UNWIND"],
|
||||
["aarch64-little-endian", 0x70000001, "PT_AARCH64_ARCHEXT"],
|
||||
["x86_64-little-endian", 0x70000000, "PT_AARCH64_ARCHEXT"],
|
||||
["x86_64-little-endian", 0x70000001, "PT_AARCH64_UNWIND"],
|
||||
["aarch64-little-endian", 0x70000001, 0x70000001],
|
||||
["aarch64-little-endian", 0x70000002, "PT_AARCH64_MEMTAG_MTE"],
|
||||
] as const)(
|
||||
"rejects %s program type %s reported as %s outside its pinned projection",
|
||||
(architecture, rawType, reportedType) => {
|
||||
const { report, snapshot } = auxiliaryHeader(
|
||||
architecture,
|
||||
rawType,
|
||||
reportedType,
|
||||
);
|
||||
expect(() =>
|
||||
validateRecordedCrashStructure(
|
||||
report,
|
||||
snapshot,
|
||||
report.decoder_diagnostics,
|
||||
),
|
||||
).toThrow(/Reported core structure differs/);
|
||||
},
|
||||
);
|
||||
|
||||
it("still binds the remaining AArch64 auxiliary header fields to their source", () => {
|
||||
const { report, snapshot } = auxiliaryHeader(
|
||||
"aarch64-little-endian",
|
||||
0x70000001,
|
||||
"PT_AARCH64_UNWIND",
|
||||
);
|
||||
snapshot.writeUInt32LE(4, 64 + 4);
|
||||
expect(() =>
|
||||
validateRecordedCrashStructure(
|
||||
report,
|
||||
snapshot,
|
||||
report.decoder_diagnostics,
|
||||
),
|
||||
).toThrow(/Reported core structure differs/);
|
||||
});
|
||||
|
||||
it.each(["x86_64-little-endian", "aarch64-little-endian"] as const)(
|
||||
"admits a %s core whose header carries that profile's e_machine",
|
||||
(architecture) => {
|
||||
|
||||
@@ -8,7 +8,7 @@ import { trackBrowser } from "./cdpBrowserProvider.support.js";
|
||||
it.each([
|
||||
["related", { openerId: "allowed-page" }, 1, null],
|
||||
["foreign", { openerId: "other-page" }, 0, "out_of_target_scope"],
|
||||
["unknown", {}, 0, "unknown_target_relationship"],
|
||||
["unknown", {}, 0, "unattributed_target"],
|
||||
] as const)(
|
||||
"keeps %s worker ownership separate from attachment state",
|
||||
async (_relation, ownership, count, exclusion) => {
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
import { rm, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
import { afterEach, expect, it } from "vitest";
|
||||
|
||||
import { webPageInspectionSchema } from "../../../src/domain/browserObservationSchemas.js";
|
||||
import { electronPageInspectionSchema } from "../../../src/domain/javascript/electronObservation.js";
|
||||
import { runCliJson } from "../../fixtures/cliJsonProcess.js";
|
||||
import {
|
||||
startFakeCdpBrowser,
|
||||
type FakeCdpBrowser,
|
||||
} from "../../fixtures/fakeCdpBrowser.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
const browsers: FakeCdpBrowser[] = [];
|
||||
const temporary: string[] = [];
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(browsers.splice(0).map(async (browser) => browser.close()));
|
||||
await Promise.all(
|
||||
temporary
|
||||
.splice(0)
|
||||
.map(async (root) => rm(root, { recursive: true, force: true })),
|
||||
);
|
||||
});
|
||||
|
||||
const normalizedResult = (value: unknown): unknown => {
|
||||
if (
|
||||
typeof value !== "object" ||
|
||||
value === null ||
|
||||
!("normalized_result" in value)
|
||||
)
|
||||
throw new Error("Expected CLI Evidence");
|
||||
return value.normalized_result;
|
||||
};
|
||||
|
||||
it("keeps missing worker ownership unknown through inspect-web-page", async () => {
|
||||
const browser = await startFakeCdpBrowser({
|
||||
commandResult: ({ method }, origin) => {
|
||||
if (method !== "Target.getTargets") return undefined;
|
||||
const worker = {
|
||||
type: "worker",
|
||||
title: "Fixture",
|
||||
attached: false,
|
||||
canAccessOpener: false,
|
||||
url: `${origin}/worker.js`,
|
||||
};
|
||||
return {
|
||||
targetInfos: [
|
||||
{ ...worker, targetId: "related-opener", openerId: "allowed-page" },
|
||||
{
|
||||
...worker,
|
||||
targetId: "related-parent",
|
||||
parentFrameId: "frame-main",
|
||||
},
|
||||
{ ...worker, targetId: "withheld-unknown" },
|
||||
{ ...worker, targetId: "withheld-empty-url", url: "" },
|
||||
{ ...worker, targetId: "withheld-foreign", openerId: "other-page" },
|
||||
{
|
||||
...worker,
|
||||
targetId: "withheld-foreign-parent",
|
||||
parentFrameId: "other-frame",
|
||||
},
|
||||
{ ...worker, targetId: "withheld-invalid", openerId: null },
|
||||
{
|
||||
...worker,
|
||||
targetId: "withheld-disallowed",
|
||||
openerId: "allowed-page",
|
||||
url: "https://foreign.test/worker.js",
|
||||
},
|
||||
],
|
||||
};
|
||||
},
|
||||
});
|
||||
browsers.push(browser);
|
||||
const evidence = await runCliJson([
|
||||
"inspect-web-page",
|
||||
browser.endpoint,
|
||||
"allowed-page",
|
||||
"--observation-ms",
|
||||
"0",
|
||||
"--json",
|
||||
]);
|
||||
expect(evidence).toMatchObject({
|
||||
operation: "inspect_web_page",
|
||||
provider: { id: "rea-cdp-browser" },
|
||||
});
|
||||
const inspection = webPageInspectionSchema.parse(normalizedResult(evidence));
|
||||
expect(inspection.workers).toEqual([
|
||||
expect.objectContaining({
|
||||
target_id: "related-opener",
|
||||
opener_target_id: "allowed-page",
|
||||
parent_frame_id: null,
|
||||
}),
|
||||
expect.objectContaining({
|
||||
target_id: "related-parent",
|
||||
opener_target_id: null,
|
||||
parent_frame_id: "frame-main",
|
||||
}),
|
||||
]);
|
||||
expect(inspection.completeness.unavailable_sections).toContain("workers");
|
||||
expect(
|
||||
inspection.completeness.excluded.filter(
|
||||
({ section }) => section === "workers",
|
||||
),
|
||||
).toEqual([
|
||||
{ section: "workers", reason: "disallowed_origin", count: 1 },
|
||||
{ section: "workers", reason: "invalid_protocol_value", count: 1 },
|
||||
{ section: "workers", reason: "out_of_target_scope", count: 2 },
|
||||
{ section: "workers", reason: "unattributed_target", count: 2 },
|
||||
]);
|
||||
expect(JSON.stringify(evidence)).not.toContain("withheld-");
|
||||
expect(
|
||||
browser.commands.some(({ method }) => method === "Target.detachFromTarget"),
|
||||
).toBe(true);
|
||||
}, 20_000);
|
||||
|
||||
it("uses the same ownership facts for passive Electron worker inspection", async () => {
|
||||
const root = await createTestTempDirectory("rea-worker-ownership-");
|
||||
temporary.push(root);
|
||||
await writeFile(join(root, "index.html"), "<html></html>");
|
||||
await writeFile(join(root, "worker.js"), "void 0;");
|
||||
const browser = await startFakeCdpBrowser({
|
||||
electronFileUrl: pathToFileURL(join(root, "index.html")).href,
|
||||
commandResult: ({ method }) => {
|
||||
if (method !== "Target.getTargets") return undefined;
|
||||
const worker = {
|
||||
type: "worker",
|
||||
title: "Fixture",
|
||||
attached: false,
|
||||
canAccessOpener: false,
|
||||
url: pathToFileURL(join(root, "worker.js")).href,
|
||||
};
|
||||
return {
|
||||
targetInfos: [
|
||||
{ ...worker, targetId: "related", openerId: "electron-page" },
|
||||
{ ...worker, targetId: "withheld-unknown" },
|
||||
{ ...worker, targetId: "withheld-empty-url", url: "" },
|
||||
{ ...worker, targetId: "withheld-foreign", openerId: "other-page" },
|
||||
{ ...worker, targetId: "withheld-invalid", parentFrameId: 42 },
|
||||
],
|
||||
};
|
||||
},
|
||||
});
|
||||
browsers.push(browser);
|
||||
const evidence = await runCliJson([
|
||||
"inspect-electron-page",
|
||||
browser.endpoint,
|
||||
"electron-page",
|
||||
"--observation-ms",
|
||||
"0",
|
||||
"--json",
|
||||
]);
|
||||
expect(evidence).toMatchObject({
|
||||
operation: "inspect_electron_page",
|
||||
provider: { id: "rea-cdp-electron" },
|
||||
});
|
||||
const inspection = electronPageInspectionSchema.parse(
|
||||
normalizedResult(evidence),
|
||||
);
|
||||
expect(inspection.workers).toEqual([
|
||||
expect.objectContaining({
|
||||
target_id: "related",
|
||||
opener_target_id: "electron-page",
|
||||
parent_frame_id: null,
|
||||
}),
|
||||
]);
|
||||
expect(inspection.completeness.unavailable_sections).toContain("workers");
|
||||
expect(
|
||||
inspection.completeness.excluded.filter(
|
||||
({ section }) => section === "workers",
|
||||
),
|
||||
).toEqual([
|
||||
{ section: "workers", reason: "invalid_protocol_value", count: 1 },
|
||||
{ section: "workers", reason: "out_of_target_scope", count: 1 },
|
||||
{ section: "workers", reason: "unattributed_target", count: 2 },
|
||||
]);
|
||||
expect(JSON.stringify(evidence)).not.toContain("withheld-");
|
||||
expect(
|
||||
browser.commands.some(({ method }) => method === "Target.detachFromTarget"),
|
||||
).toBe(true);
|
||||
}, 20_000);
|
||||
Loaded 100 of 114 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user