merge: inherit resolved environment changes

This commit is contained in:
morluto committed 2026-10-28 05:09:10 +08:00
commit bbce4fc2ea
622 files changed
+24509 -196582

No files matched your search

+21
View File
@@ -0,0 +1,21 @@
{
"name": "rea",
"interface": {
"displayName": "REA"
},
"plugins": [
{
"name": "rea",
"source": {
"source": "npm",
"package": "rea-agents",
"version": "6.8.0"
},
"policy": {
"installation": "AVAILABLE",
"authentication": "ON_INSTALL"
},
"category": "Developer Tools"
}
]
}
@@ -17,6 +17,9 @@ readiness or provider commands.
Installing this skill supplies instructions; it does not register the REA MCP
server, install analysis engines, or add tools to an already-running agent.
The exception is a Claude Code or Codex plugin install: the plugin bundles
this skill and registers its own `rea` server, outside the state that
`rea setup` and `rea doctor` manage.
If REA tools are available and their registration is not known to be stale,
proceed directly to the target. Do not run diagnostics or setup before every
@@ -37,37 +40,18 @@ repeat or known not to have started. If it may already have changed the
target, or it fails again, tell the user and ask them to restart the MCP
client. If REA could not start its connection, retry once, then tell the user
and ask them to restart the MCP client. Do not retry a startup failure that
returns the same unchanged result. Run `rea doctor` when REA could not start
returns the same unchanged result. For plugin installs, follow
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin).
For setup-owned registrations, run `rea doctor` when REA could not start
or tools stay missing after that restart. For `cleanup_incomplete`, a retryable
capacity limit, or `configuration_invalid`, follow the recovery guide's ladder
before asking for a restart. Read that guide before a second repair attempt.
1. Diagnose without changing REA or client configuration. `npx` may download
the package and write to npm's cache. For Codex, run
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
`commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
file registration: doctor reports the Grok Bot chat step, and setup does not
write `mcp.json`. If the client is unknown, use `doctor --json` and
inspect its registration results before choosing a setup scope.
2. Distinguish the reason. Missing, malformed, or stale registration needs a
scoped configuration repair. An aligned registration with no tools in the
active session needs a restart/reconnection; doctor cannot prove that the
current agent has connected. A missing provider affects only tasks requiring
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
and Android inspection has separate bring-your-own JADX/Java prerequisites.
3. For a configuration repair, prepare the read-only plan:
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
Use the current client's ID, show its exact proposed paths, backups, and
changes, and obtain approval before setup writes configuration or installs
Hopper. Setup normally installs the matching bundled skill too; include that
replacement in the reviewed plan. After approval, apply the same scope with
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
only if that separate installation was needed and explicitly approved.
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
in the session, then resume the original investigation. If they remain
absent, inspect the client's MCP launch error rather than repeating setup.
Choose the installation owner before repairing registration. Plugin installs use
[plugin recovery](references/connection-and-recovery.md#rea-installed-as-a-client-plugin);
only setup-owned registrations use the guide's
[registration repair plan](references/connection-and-recovery.md#repair-registration-when-needed).
A plugin-only doctor result of `missing` does not call for `rea setup`.
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
JADX, Kuna, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
@@ -86,7 +70,7 @@ returns the Evidence record directly. This fallback does not establish that
MCP is configured. Native CLI tasks still require their selected engine.
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
Configuration repair starts with a read-only setup plan; show paths, backups and
For setup-owned registration, configuration repair starts with a read-only setup plan; show paths, backups and
skill replacement before applying authorized changes. Preserve existing approval
and target-execution boundaries.
@@ -13,6 +13,27 @@ own APK path and explicit selectors; do not open a native session first.
Choose an `overload_index` from the class inventory when methods are ambiguous
instead of guessing.
For JNI implementations, discover `inspect_android_jni_bindings`. Select raw
DEX class/method identities and optional descriptors, ABI or logical library
paths. Prefer `detail: "summary"`; complete Evidence remains retained and
`android-jni` views page declarations, libraries, candidates or binding rows
with exact fields. Global native-pair ambiguity survives class selection.
Preserve slot/relocation provenance and source coverage: matched means observed
candidate presence; partial scans do not establish uniqueness or absence.
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
target code. Runtime registration and loading remain unknown. See the
[JNI guide](https://github.com/morluto/rea/blob/main/docs/android-analysis.md#link-native-declarations-to-jni-candidates).
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
`detail: "summary"` to receive normalized symbols or the selected body without
duplicate raw class metadata. Complete Evidence stays retained under the linked
parent ID and can be exported; CLI summary delivery requires `--evidence_output`.
Omitted or complete detail preserves the full inline response. Keep
selecting by observed names; the separate `mapping` block reports each symbol
as `mapped`, `ambiguous` or `unmapped` after checking DEX types and prototypes.
REA does not verify that the mapping belongs to the APK build.
The provider requires a separately supplied headless JADX MCP JAR 0.7.1 and
Java 17 or newer. `REA_JADX_MCP_JAR` selects the absolute JAR path; `JAVA_HOME`
can select Java. Use `rea doctor --provider jadx --json` to check the JADX
@@ -17,7 +17,7 @@ does not establish which server an approval wildcard covers. See
| Observation | Meaning and next step |
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
| No REA tools in the active session | If REA was installed as a client plugin, follow the plugin section below. Otherwise inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
| MCP connection dropped or REA could not start | If the connection drops, retry that call once only when it is safe to repeat or known not to have started. If it may already have changed the target, or the same failure returns, tell the user REA's connection failed and ask them to restart the MCP client. If REA could not start its connection, retry once, then tell the user and ask them to restart the MCP client. Do not retry a startup failure that returns the same unchanged result. Run `rea doctor` when REA could not start or tools stay missing after restart. Doctor cannot restore a live connection by itself. |
@@ -116,6 +116,32 @@ is not a recommendation to change it. Do not invent a setting. Host memory and
- `invalid_request`: correct the reported fields from the current schema. Do
not repeat the unchanged call.
## REA installed as a client plugin
A Claude Code or Codex plugin owns its `rea` server and this skill. Doctor's
`--client` and `--skill` checks and `rea setup` read and write only
setup-owned state, so for a plugin-only install they report `missing`, and
applying setup would add a second `rea` server. If this skill was loaded from
a plugin, or the client lists a `rea` plugin, do not prepare a setup plan for
missing tools. Instead:
- Check that the plugin is installed and enabled with the client's own
listing: `claude plugin list` and `/mcp` in Claude Code, `codex plugin list`
in Codex.
- Restart or reconnect the client, then read its launch error for the `rea`
server if tools are still absent. The plugin starts its bundled runtime; a missing build or version mismatch
requires a reinstall, or a fresh build when developing from source.
- Check analysis engines through the connected server's
`binary_session.result.tool_availability`. `rea doctor --provider <id> --json`
from a shell reports that shell's environment, which can differ from the
environment the client gave the plugin server. If you still run it through
the package runner, use the version pinned in the plugin manifest's server
entry (`rea-agents@<version>`), not `@latest`, so it matches the server the
plugin starts.
Use the setup repair below only when the user wants a setup-owned registration
instead of the plugin, and remove or disable the plugin first.
## Repair registration when needed
For Codex:
+21
View File
@@ -0,0 +1,21 @@
{
"$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
"name": "rea",
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"owner": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"plugins": [
{
"name": "rea",
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
"source": {
"source": "npm",
"package": "rea-agents",
"version": "6.8.0"
},
"category": "development"
}
]
}
+28
View File
@@ -0,0 +1,28 @@
{
"name": "rea",
"version": "6.8.0",
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
"author": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"homepage": "https://rea.tools/",
"repository": "https://github.com/morluto/rea",
"license": "MIT",
"keywords": [
"reverse-engineering",
"binary-analysis",
"decompiler",
"mcp-server"
],
"skills": "./skills",
"mcpServers": {
"rea": {
"command": "node",
"args": [
"${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs",
"6.8.0"
]
}
}
}
+13
View File
@@ -0,0 +1,13 @@
{
"mcpServers": {
"rea": {
"command": "node",
"args": [
"./scripts/plugin-mcp-launcher.mjs",
"6.8.0"
],
"cwd": ".",
"startup_timeout_sec": 30
}
}
}
+38
View File
@@ -0,0 +1,38 @@
{
"name": "rea",
"version": "6.8.0",
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
"author": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"homepage": "https://rea.tools/",
"repository": "https://github.com/morluto/rea",
"license": "MIT",
"keywords": [
"reverse-engineering",
"binary-analysis",
"decompiler",
"mcp-server"
],
"skills": "./skills",
"mcpServers": "./.codex-plugin/mcp.json",
"interface": {
"displayName": "REA",
"shortDescription": "Reverse engineer anything with agents",
"longDescription": "Use REA to explain shipped behavior from native binaries, managed assemblies, Electron and JavaScript bundles, packages, firmware, and browser targets, with evidence that separates observations from inferences.",
"developerName": "morluto",
"category": "Developer Tools",
"capabilities": [
"Interactive",
"Read",
"Write"
],
"websiteURL": "https://rea.tools/",
"defaultPrompt": [
"Use REA to explain how this binary implements a feature.",
"Use REA to compare two versions of this application.",
"Use REA to recover the sources from this JavaScript bundle."
]
}
}
+18 -16
View File
@@ -855,6 +855,24 @@ jobs:
# The matching native artifact establishes real Job Object and private
# runtime boundaries for these fixtures. Ghidra and Java are not needed;
# real engine claims remain in the separate controlled-provider workflow.
# Run native proofs before the broad curated suite so an unrelated
# fixture failure cannot hide the real control and recovery checks.
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-process-boundary-fixture
path: .cache/windows-process-boundary/
# Passing the fixture adds the breakaway, caller/child token, environment
# semantics and ancestor reparse observations, which the lane skips
# entirely when it is absent.
- run: >-
npm run verify:windows-native -- .
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-native-inspection-fixture
path: .cache/windows-native-inspection-fixture/
- name: Verify native inspection failure ownership and recovery
run: node scripts/verify-windows-native-inspection.mjs
- name: Run curated Windows capability tests
run: >-
npx vitest run --coverage.enabled=false
@@ -907,22 +925,6 @@ jobs:
tests/process-global/runnerCancellation.test.ts
tests/process-global/windowsOwnedCommand.test.ts
tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-process-boundary-fixture
path: .cache/windows-process-boundary/
# Passing the fixture adds the breakaway, caller/child token, environment
# semantics and ancestor reparse observations, which the lane skips
# entirely when it is absent.
- run: >-
npm run verify:windows-native -- .
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: windows-native-inspection-fixture
path: .cache/windows-native-inspection-fixture/
- name: Verify native inspection failure ownership and recovery
run: node scripts/verify-windows-native-inspection.mjs
- name: Verify real Node Inspector discovery and observation
timeout-minutes: 10
env:
+1 -1
View File
@@ -50,7 +50,7 @@ Thumbs.db
# Local reference sources and analysis artifacts
/_reference/
/experiments/analysis-efficiency/results/
/experiments/analysis-efficiency/
# Generated website download
/website/public/examples/notes-example.zip
+4 -1
View File
@@ -10,7 +10,10 @@
"node_modules/**",
"CHANGELOG.md",
"package-lock.json",
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
"_reference/**",
"third_party/**"
"third_party/**",
".codex-plugin/mcp.json"
]
}
+1 -1
View File
@@ -88,7 +88,7 @@ configuration. Restart your agent afterward.
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
[other agents](docs/installation.md#supported-agents). See
[installation and setup](docs/installation.md) for provider configuration and
manual MCP registration.
manual MCP registration and Claude Code/Codex plugin installation.
MCP uses compact input schemas by default to reduce context overhead. Set
`REA_MCP_INPUT_SCHEMA_PROFILE=full` in your MCP registration when you need the
+225 -4
View File
@@ -1,5 +1,9 @@
import com.atxx.jhmcp.*;
import com.google.gson.*;
import com.android.tools.smali.dexlib2.Opcodes;
import com.android.tools.smali.dexlib2.dexbacked.DexBackedDexFile;
import com.android.tools.smali.dexlib2.dexbacked.DexBackedClassDef;
import com.android.tools.smali.dexlib2.dexbacked.DexBackedMethod;
import io.modelcontextprotocol.kotlin.sdk.server.*;
import io.modelcontextprotocol.kotlin.sdk.types.*;
import jadx.api.*;
@@ -9,14 +13,21 @@ import jadx.api.plugins.input.data.ICodeReader;
import jadx.api.plugins.input.data.IMethodRef;
import jadx.api.plugins.input.insns.InsnIndexType;
import jadx.api.plugins.input.insns.Opcode;
import jadx.core.codegen.TypeGen;
import jadx.core.dex.info.MethodInfo;
import jadx.core.dex.nodes.*;
import java.io.PrintStream;
import java.io.StringReader;
import java.io.InputStream;
import java.io.IOException;
import java.nio.file.Path;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.concurrent.CountDownLatch;
import java.util.stream.Collectors;
import java.util.zip.CRC32;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import kotlin.Unit;
import kotlin.ResultKt;
import kotlin.coroutines.Continuation;
@@ -178,6 +189,205 @@ class ReaJadxBridge {
manifestSnapshot = null;
}
// One bounded DEX at a time, beside the already admitted analysis JVM. The
// response allowance accounts for worst-case JSON escaping before retention.
private static final int NATIVE_DEX_BYTES = 64 * 1024 * 1024;
private static final long NATIVE_TOTAL_DEX_BYTES = 256L * 1024 * 1024;
// This JSON is escaped again inside an 8 MiB MCP frame.
private static final long NATIVE_JSON_BYTES = 4L * 1024 * 1024;
/** The bundled decoder allocates its declared UTF-16 length before validating bytes. */
private static final class NativeDex extends DexBackedDexFile {
private OptionalIndexedSection<String> boundedStrings;
NativeDex(byte[] bytes) { super(Opcodes.getDefault(), bytes); }
@Override public OptionalIndexedSection<String> getStringSection() {
if (boundedStrings == null) {
var source = super.getStringSection();
boundedStrings = new OptionalIndexedSection<>() {
@Override public int size() { return source.size(); }
@Override public int getOffset(int index) { return source.getOffset(index); }
@Override public String getOptional(int index) { return index == -1 ? null : get(index); }
@Override public String get(int index) {
int offset = getBuffer().readSmallUint(source.getOffset(index));
int units = getDataBuffer().readerAt(offset).readSmallUleb128();
if (units > NATIVE_JSON_BYTES / 6
|| units > getDataBuffer().getBuf().length - offset)
throw new IllegalArgumentException("DEX string exceeds the native decoding allowance");
return source.get(index);
}
};
}
return boundedStrings;
}
}
private static void checkNativeInterrupted() {
if (Thread.currentThread().isInterrupted())
throw new java.util.concurrent.CancellationException("Native declaration inventory interrupted");
}
private static final class NativeInventory {
final List<Map<String, Object>> methods = new ArrayList<>();
final List<String> unknowns = new ArrayList<>();
long retainedBytes = 1024;
int dexEntries;
int examinedDexEntries;
int examinedClasses;
int examinedMethods;
boolean partial;
boolean diagnosticLimit;
void unknown(String reason) {
partial = true;
long bytes = 64L + reason.length() * 6L;
if (unknowns.size() < 16 && retainedBytes + bytes <= NATIVE_JSON_BYTES - 4096) {
unknowns.add(reason);
retainedBytes += bytes;
} else if (!diagnosticLimit) {
diagnosticLimit = true;
unknowns.add("Additional native-inventory limitations exceed the retained diagnostic allowance.");
}
}
boolean retain(Map<String, Object> method, long bytes) {
if (retainedBytes + bytes > NATIVE_JSON_BYTES - 4096) {
unknown("Native declarations exceed the 4 MiB retained JSON allowance; remaining declarations are unknown.");
return false;
}
retainedBytes += bytes;
methods.add(method);
return true;
}
Map<String, Object> result() {
return Map.of("scope", "apk-root-dex-declarations", "methods", List.copyOf(methods),
"coverage", Map.of("status", partial ? "partial" : "complete",
"dex_entries", dexEntries, "examined_dex_entries", examinedDexEntries,
"examined_classes", examinedClasses, "examined_methods", examinedMethods),
"unknowns", List.copyOf(unknowns));
}
}
/** Match Android's standard root multidex carriers, without a basename guess. */
private static boolean nativeDexEntry(String name) {
return name.equals("classes.dex") || name.matches("classes[0-9]+\\.dex");
}
private static byte[] readNativeDex(ZipFile archive, ZipEntry entry) throws IOException {
long size = entry.getSize();
if (size < 0 || size > NATIVE_DEX_BYTES)
throw new IllegalArgumentException("DEX exceeds the 64 MiB single-member read allowance");
byte[] bytes = new byte[(int) size];
CRC32 crc = new CRC32();
try (InputStream input = archive.getInputStream(entry)) {
int offset = 0;
while (offset < bytes.length) {
checkNativeInterrupted();
int read = input.read(bytes, offset, Math.min(65536, bytes.length - offset));
if (read < 0) throw new IOException("DEX is shorter than its ZIP declaration");
if (read == 0) continue;
crc.update(bytes, offset, read);
offset += read;
}
if (input.read() != -1) throw new IOException("DEX exceeds its ZIP declaration");
}
if (entry.getCrc() < 0 || entry.getCrc() != crc.getValue())
throw new IOException("DEX does not match its ZIP CRC32 declaration");
return bytes;
}
/** Raw DEX ownership precedes JADX's duplicate-class selection and aliases. */
static Map<String, Object> nativeDeclarations(Path apk) {
NativeInventory inventory = new NativeInventory();
try (ZipFile archive = new ZipFile(apk.toFile())) {
Map<String, Integer> names = new HashMap<>();
var directory = archive.entries();
while (directory.hasMoreElements()) {
checkNativeInterrupted();
ZipEntry entry = directory.nextElement();
if (!entry.isDirectory() && nativeDexEntry(entry.getName())) {
inventory.dexEntries++;
names.merge(entry.getName(), 1, Integer::sum);
}
}
long totalBytes = 0;
int entryIndex = -1;
directory = archive.entries();
while (directory.hasMoreElements()) {
checkNativeInterrupted();
ZipEntry entry = directory.nextElement();
entryIndex++;
String path = entry.getName();
if (entry.isDirectory() || !nativeDexEntry(path)) continue;
if (names.get(path) != 1) {
inventory.unknown("Duplicate root DEX entry name cannot be selected uniquely by the ZIP reader: " + path);
continue;
}
if (entry.getSize() < 0 || entry.getSize() > NATIVE_DEX_BYTES
|| entry.getSize() > NATIVE_TOTAL_DEX_BYTES - totalBytes) {
inventory.unknown("Root DEX member exceeds the 64 MiB member or 256 MiB total read allowance: " + path);
continue;
}
totalBytes += entry.getSize();
try {
DexBackedDexFile dex = new NativeDex(readNativeDex(archive, entry));
var classes = dex.getClassSection();
for (int classIndex = 0; classIndex < classes.size(); classIndex++) {
checkNativeInterrupted();
DexBackedClassDef cls = classes.get(classIndex);
String type = cls.getType();
if (!type.startsWith("L") || !type.endsWith(";"))
throw new IllegalArgumentException("Class definition has no reference type");
String className = type.substring(1, type.length() - 1).replace('/', '.');
// Passing false retains duplicate encoded method references too.
for (Iterable<? extends DexBackedMethod> group : List.of(
cls.getDirectMethods(false), cls.getVirtualMethods(false))) {
for (DexBackedMethod method : group) {
checkNativeInterrupted();
inventory.examinedMethods++;
if ((method.getAccessFlags() & 0x100) == 0) continue;
String methodName = method.getName();
String returnType = method.getReturnType();
long characters = 2L + returnType.length();
for (CharSequence parameter : method.getParameterTypes()) {
characters += parameter.length();
if (characters * 6L > NATIVE_JSON_BYTES) break;
}
long bytes = 512L + 6L * (className.length() + methodName.length()
+ path.length() + characters);
if (inventory.retainedBytes + bytes > NATIVE_JSON_BYTES - 4096) {
inventory.unknown("Native declarations exceed the 4 MiB retained JSON allowance; remaining declarations are unknown.");
return inventory.result();
}
StringBuilder descriptor = new StringBuilder((int) characters).append('(');
for (CharSequence parameter : method.getParameterTypes()) descriptor.append(parameter);
descriptor.append(')').append(returnType);
Map<String, Object> row = Map.of("source_index", inventory.methods.size(),
"class_name", className, "method_name", methodName, "descriptor", descriptor.toString(),
"dex_entry", path, "dex_entry_index", entryIndex, "class_index", classIndex,
"method_index", method.getMethodIndex());
if (!inventory.retain(row, bytes)) return inventory.result();
}
}
inventory.examinedClasses++;
}
inventory.examinedDexEntries++;
} catch (IOException | RuntimeException failure) {
if (failure instanceof java.util.concurrent.CancellationException) throw (RuntimeException) failure;
inventory.unknown("Root DEX native declarations could not be fully read from " + path + ": "
+ failure.getClass().getSimpleName()
+ (failure.getMessage() == null ? "" : " (" + failure.getMessage() + ")"));
}
}
} catch (IOException failure) {
// A failed archive close must fail the operation so the provider retires
// its owned JVM; do not cache a result with lost file-resource ownership.
throw new IllegalArgumentException("APK ZIP native declaration inventory could not finish its reader", failure);
}
return inventory.result();
}
/** Original archive identity, never a deobfuscated basename or name suffix. */
private ManifestSnapshot manifest(JadxSession current) {
if (manifestSession != current || manifestSnapshot == null) {
@@ -270,7 +480,7 @@ class ReaJadxBridge {
.map(Object::toString).collect(Collectors.joining(", "))
+ "): " + method.getReturnType().toString();
return new MethodSnapshot(name, rawName, shortId, List.copyOf(bodyAbsenceMarkers), Map.of(
"name", name, "signature", signature,
"name", name, "raw_name", rawName, "signature", signature,
"dex_descriptor", descriptor,
"is_constructor", method.isConstructor(), "def_pos", 0));
}).toList();
@@ -280,8 +490,11 @@ class ReaJadxBridge {
List<Map<String, Object>> valuedFields = new ArrayList<>();
boolean anyValue = false;
for (FieldNode field : orderedFields) {
// Raw DEX identity for mapping lookups; the display type can carry generics.
Map<String, Object> entry = Map.of("name", field.getAlias(),
"type", field.getType().toString(), "def_pos", 0);
"raw_name", field.getFieldInfo().getName(),
"type", field.getType().toString(),
"dex_type", TypeGen.signature(field.getFieldInfo().getType()), "def_pos", 0);
fields.add(entry);
Map<String, Object> value = dexStaticValue(field);
if (value == null) valuedFields.add(entry);
@@ -293,10 +506,11 @@ class ReaJadxBridge {
}
}
List<String> inner = node.getInnerClasses().stream().map(ClassNode::getFullName).toList();
List<String> innerRaw = node.getInnerClasses().stream().map(ClassNode::getRawName).toList();
Map<String, Object> summary = Map.of(
"full_name", cls.getFullName(), "raw_name", cls.getRawName(), "name", cls.getName(),
"method_count", methods.size(), "field_count", fields.size(),
"inner_class_count", inner.size(), "inner_classes", inner,
"inner_class_count", inner.size(), "inner_classes", inner, "inner_class_raw_names", innerRaw,
"methods", methods.stream().map(MethodSnapshot::summary).toList(), "fields", List.copyOf(fields));
// DEX static values are opt-in, so method and reference lookups keep the member-only reply.
Map<String, Object> summaryWithValues = summary;
@@ -470,6 +684,8 @@ class ReaJadxBridge {
"available_processors", Runtime.getRuntime().availableProcessors(),
"java_version", System.getProperty("java.version"),
"metadata_scope", "parsed_members"));
if (name.equals("rea_native_declarations"))
return json(nativeDeclarations(Path.of(string(args, "path"))));
JadxSession current = session();
if (name.equals("get_app_info")) return json(manifest(current).summary());
if (name.equals("get_android_manifest"))
@@ -581,7 +797,7 @@ class ReaJadxBridge {
SessionHolder holder = new SessionHolder(defaults);
ServerCapabilities capabilities = new ServerCapabilities(
new ServerCapabilities.Tools(null), null, null, null, null, null, null, Map.of());
Server server = new Server(new Implementation("rea-jadx-bridge", "7", null, null, null),
Server server = new Server(new Implementation("rea-jadx-bridge", "9", null, null, null),
new ServerOptions(capabilities), "", ignored -> Unit.INSTANCE);
SessionToolsKt.registerSessionTools(server, holder);
ClassToolsKt.registerClassTools(server, holder);
@@ -601,6 +817,11 @@ class ReaJadxBridge {
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of()), List.of(), null),
"Observed worker JVM and metadata scope", null, null, null, null, null, null);
server.addTool(diagnostics, (connection, request, continuation) -> bridge.handle("rea_jvm_status", request.getArguments()));
registerBridgeTool(server, bridge, "rea_native_declarations",
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of("path",
new kotlinx.serialization.json.JsonObject(Map.of("type",
kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))))), List.of("path"), null),
"Original ACC_NATIVE declarations from the admitted APK's standard root DEX members, without loading or decompiling the APK.");
for (String name : List.of("get_app_info", "get_android_manifest")) {
String description = name.equals("get_app_info")
? "Namespace-aware summary of the exact root AndroidManifest.xml resource."
+195
View File
@@ -286,6 +286,162 @@ instructions into field declarations, leaving an empty or incomplete method
body. The selected method's disassembly preserves those instructions and retains
the same source byte limit and explicit truncation reporting.
## Apply an R8/ProGuard mapping
Release builds shrunk by R8 or ProGuard rename classes and members. Select the
build's `mapping.txt` with `mapping_path` on `inspect_android_class` or
`inspect_android_method`, or `--mapping-path` on the matching CLI command:
```sh
rea inspect-android-class /path/Example.apk a.b --mapping-path /path/mapping.txt \
--detail summary --evidence_output /tmp/class-evidence.json
```
For agent workflows, select `detail: "summary"` on class and method inspection.
It preserves the complete normalized member inventory or selected method body,
including mapping annotations, while retaining duplicate raw provider calls in
the complete parent Evidence. The reply links that parent's identity and digest;
export it with `export_evidence_bundle` or read its summary again through
`inspect_analysis_view`. CLI summaries require `--evidence_output` and return
the portable bundle path. Omitted or `complete` detail preserves complete inline
Evidence. Search and reference operations retain their existing contracts.
Symbol summary views admit at most **1 MiB**, including view context. An admitted
method's source can already consume that amount; a larger combined summary
fails explicitly instead of silently truncating facts. MCP refusal includes
the retained parent ID. Select complete detail, or inspect one method rather
than a whole class, when the requested summary does not fit.
The mapping path must be absolute. Selectors still take the observed names, such as
`a.b`, and observed rows are unchanged. The result adds one `mapping` block with
the file's path, SHA-256 and size, the header values the file reports
(`compiler`, `compiler_version`, `min_api`, `map_version`, `pg_map_id`,
`pg_map_hash`), valid record counts and up to 16 malformed line samples. Its
`symbols` list has one entry per observed class, inner class, method and field;
`inspect_android_method` lists the class and the selected method. `coverage`
counts each resolution status. With summary delivery, these unchanged facts are
under the view's `summary` field; the view separately reports source coverage.
Lookups use the bridge's raw DEX names, not JADX display aliases such as a
renamed reserved word. Each symbol resolves as:
- `mapped`: exactly one record matches. A class needs one record for its raw
name (`verified_by: "class"`). A method must also match its DEX prototype
(`"descriptor"`) and a field its DEX type (`"type"`). R8 residual signature
metadata is used when present; otherwise original types are translated
through the file's class records.
- `ambiguous`: `multiple-records`, `descriptor-mismatch`, `type-mismatch`,
`ambiguous-class` or `residual-signature-conflict`, with `candidate_count` and
up to 8 candidates. Conflicting residual metadata never selects a winner. A name match
whose prototype or type does not verify is never reported as mapped.
- `unmapped`: no valid record matched the raw identity.
- `unknown` with reason `projection-budget`: this claim did not fit the combined
reply budget. The observed raw symbol remains present; `coverage.unknown`
counts these omissions. Inspect that specific class or method separately,
or consult the selected local mapping file for complete source records.
A line continues an inline frame stack when it repeats the previous line's
minified range and obfuscated name and gives a single call-site line, such as
`:7`. Only the outermost original of a stack becomes a candidate. Original
line numbers and source files are not projected. Malformed records are counted
and skipped, so an `unmapped` symbol might be described by one of them. A file
that is missing, not a regular file, not UTF-8, larger than the byte limit or
without any class record, such as `usage.txt` or `seeds.txt`, is refused before
JADX is used. Requests read their mappings one at a time, in the same queue as
the JADX work and under the same operation deadline. A mapping failure therefore never starts or retires the JADX
session, and a different mapping reuses the loaded APK.
## Link native declarations to JNI candidates
When advertised, `inspect_android_jni_bindings` links original `ACC_NATIVE`
declarations from standard root `classes*.dex` members to bundled ELF libraries.
The supplied JAR's DEX reader preserves duplicate declarations before decompiler
deduplication. This operation does not load the APK into the decompiler.
MCP defaults to retained summary delivery when `detail` is omitted; select
`complete` explicitly to receive the full report inline.
```json
{
"path": "/absolute/Example.apk",
"class_name": "org.conscrypt.NativeCrypto",
"method_names": ["EVP_PKEY_new_RSA"],
"abis": ["arm64-v8a"],
"detail": "summary"
}
```
Class/method names and optional `descriptors` select raw DEX identities.
`library_paths` selects exact logical APK paths such as
`lib/arm64-v8a/libconscrypt_jni.so`; each library's `source_path` preserves its
original ZIP spelling. Omitted filters include all scoped facts; `[]` matches
none. Class selection never narrows the global inventory used for table-class
attribution.
Independent `declarations`, `libraries`, `candidates` and `bindings` collections
have stable IDs and source ordinals. Candidates retain APK/library digests,
ABI, function addresses and file offsets. Table candidates also retain section,
table and entry identity plus all three name/signature/function pointer slots
and their relocation sources. JNI exports follow the
[UTF-16 escaping and short-before-long rules](https://docs.oracle.com/en/java/javase/21/docs/specs/jni/design.html#resolving-native-method-names);
failed escaping does not acquire an invented export match. Tables require
NUL-terminated [Modified UTF-8 strings](https://docs.oracle.com/en/java/javase/21/docs/specs/jni/types.html#modified-utf-8-strings),
valid descriptors and executable load mappings. REL, RELA and RELR retain
separate provenance; unsupported packed relocations and unresolved pointer
spans stay explicit unknowns.
A static `JNINativeMethod` entry names no class. Attribute it uniquely only
when exactly one declaration in the complete recorded native inventory has
that `(name, descriptor)` pair. Multiple classes remain ambiguous after a
class filter; a partial inventory cannot establish unique attribution.
Each selected declaration has a row per selected/observed ABI. `matched` means
candidate presence was observed; `ambiguous` means multiple observed targets
or class associations remain. Binding `coverage` is separate: a matched
partial row does not establish complete uniqueness, and an empty partial scan
is `unknown`, never `unmatched`. ELF/ABI label mismatches preserve facts without
asserting a binding. No result establishes library loading, runtime registration
or method invocation.
Summary returns the first 64 selected binding rows, their declaration/candidate
closure and every selected library's identity, coverage and failure reason;
counts and page coverage describe the full selected binding set. Selected
declaration IDs have separate preview coverage, so native declarations remain
visible even when the APK has no bundled ABI libraries. Complete
Evidence stays retained. Read exact fields or further pages without reopening
files or rerunning analysis:
```json
{
"source": { "kind": "retained-evidence", "evidence_id": "ev_<digest>" },
"view": {
"kind": "android-jni",
"collection": "bindings",
"offset": 64,
"limit": 64,
"fields": ["declaration_id", "abi", "status", "coverage", "candidate_ids"]
}
}
```
Collections also include `declarations`, `candidates` and `libraries`, with
exact filters and batch lookup by declaration/candidate/library IDs. `row_index`
is the original parent collection index; DEX and
ZIP source ordinals stay separate. Complete detail returns the full report.
CLI uses the same selection and delivery contracts. Its common one-shot
default remains complete; summary delivery requires an explicit Evidence path:
```sh
rea inspect-android-jni-bindings /absolute/Example.apk \
--query '{"class_name":"org.conscrypt.NativeCrypto","method_names":["EVP_PKEY_new_RSA"],"abis":["arm64-v8a"]}' \
--detail summary --evidence_output /absolute/fresh-jni-evidence.json
```
Optional `--view` accepts the same JSON page and requires summary delivery.
Evidence bundles never overwrite files. Complete declarations and one library
selection are cached within the immutable APK session. Changing ABI/library
selection replaces the scan; class/method selection reuses it. Fresh analysis
still validates current caller APK bytes. Idle/close releases the caches;
retained Evidence remains historical and independently readable.
## Interpret the result
- Package inspection retains decoded manifest XML, summary declarations,
@@ -322,6 +478,11 @@ the same source byte limit and explicit truncation reporting.
`dex_descriptor`, for example `(I)V`. Display names can be JADX aliases;
the descriptor alone is not a complete raw DEX method ID. Overload indices
are tied to this artifact and engine, not stable identities across builds.
- A `mapping` block appears only when the caller selected `mapping_path`. Its
names are claims from that file, not observations of the APK. REA does not
check that the mapping belongs to this build; `pg_map_id` is reported, not
compared. Evidence keeps the selected path in its parameters and the file's
SHA-256 in the result.
- Method inspection reports Java or smali, fallback markers and complete/partial
returned text. Decompiled source is a derived representation. A complete text
result does not establish complete semantics or a byte-identical reconstruction.
@@ -411,6 +572,19 @@ target/configuration changes, failure, timeout, cancellation, MCP disconnect or
server shutdown. One-shot CLI commands always join cleanup before returning.
REA never writes to the original APK or launches target code.
JNI scans reuse the analyzer process without loading the APK into the
decompiler. They admit at most 64 MiB per DEX member and 256 MiB of DEX bytes,
with a 4 MiB native-declaration JSON allowance before MCP string escaping.
Native libraries are processed sequentially: one 64 MiB buffer, 256 MiB total
selected read work, and a 16 MiB parsed-library-facts allowance. ZIP inventory
and library counts also have explicit resource bounds. Unexamined occurrences
remain unknown. A complete relationship report has a separate 16 MiB serialized
allowance; summaries/pages allow 1 MiB including their provenance envelope.
These are resource admissions, not claims about maximum valid Android formats.
Select fewer ABI/library occurrences or smaller retained pages when needed.
Completed parsed library facts survive failed reader finalization in typed
partial observations; the exact reader owner remains available for cleanup retry.
On Windows, stdin writes preserve stream backpressure and run outside the Node
event loop. Cancellation terminates the retained job, including descendants,
before workspace cleanup. On Windows, a detached REA helper owns the private
@@ -496,6 +670,27 @@ complete, case-sensitive search contract. A protocol frame above **8 MiB**, or c
**32 MiB per operation**, fails with a diagnostic instead of returning an incomplete inventory
as complete. These are memory-safety boundaries, not undocumented result caps.
A selected R8/ProGuard mapping is re-read and SHA-256 checked on every call, up
to **64 MiB**. The serialized provider keeps one successful parsed index, keyed
by byte length and digest with fixed parser options. Identical bytes reuse that
index; changed bytes are reparsed, missing files still fail, and each response
preserves the currently selected path. The index is released on provider idle
retirement or close. Omitting `mapping_path` never applies the cached mapping.
During replacement, the old index can coexist with the new parse until it
succeeds; these transient allocations count toward process memory.
Before retaining records, the parser admits an estimated **128 MiB** index
representation and **8 MiB** of expanded mapped claims per residual class,
including shared header facts. The index allowance accounts for strings and
containers; it is not a measured V8 heap cap. Parameter and metadata expansion
is admitted before splitting or JSON decoding. These fixed resource bounds
have no runtime override; refusal reports `resource_constraint`, the source
line and mapping SHA-256. Ambiguous lookups count all candidates while keeping
only the first eight in stable file order. A separate **8 MiB** combined claim
budget applies to the actual query set, including outer and inner classes,
before cloning the report. Claims that do not fit become explicit `unknown`
entries; other admitted claims and all observed identities remain available.
## Real verification
Download the explicit public fixtures once; this installs no tools:
+1 -1
View File
@@ -75,7 +75,7 @@ graph TD
end
subgraph "Domain Layer"
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>format declarations, Evidence, comparisons, application graphs"]
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>format declarations, provider selection, Evidence, comparisons, application graphs"]
end
subgraph "Configuration"
+49 -17
View File
@@ -497,17 +497,51 @@ comments and unrelated settings. It resolves that file from `HERMES_HOME`, else
from the platform default Hermes itself uses — `%LOCALAPPDATA%\hermes` on
Windows, `~/.hermes` elsewhere — including any `HERMES_DATA_DIR_SUFFIX`. Hermes
connects an entry unless `enabled` is false, so setup states `enabled: true`.
Aliases whose anchor is defined in the `rea` entry retain their original resolved
values when setup refreshes or uninstall removes that entry. Setup does not
redirect another server's alias to REA's new launcher or policy.
This also covers an anchor on the removed registration key. References to
definitions outside the entry keep sharing the original value, including when
another definition later reuses the same anchor name.
Hermes personal skills install under the active Hermes home's `skills` directory.
Setup, doctor and uninstall follow the root's `active_profile` selection, as
plain Hermes CLI commands do. An explicit `HERMES_HOME` pointing to
`profiles/<name>` selects that named profile directly. An invalid, missing or
deleted sticky profile is reported instead of writing into the default home;
it also remains an unresolved skill selection, so setup cannot report a
successful empty installation. Overrides nested beneath the native Hermes home
use that native home's sticky selection and profile root together, matching
[Hermes's root rules](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_constants.py#L83).
Repair the profile or run `hermes profile use default` before retrying.
Setup, doctor and uninstall share REA's sticky-profile resolver.
A custom `HERMES_HOME` outside the platform default home
reads its own `active_profile`. An override physically inside the default home
uses that default home's sticky selection and profile root together, including
when the final override directory does not yet exist.
For example, with `HERMES_HOME=~/.hermes/sub` and `~/.hermes/active_profile` set
to `coder`, setup selects `~/.hermes/profiles/coder/config.yaml` and that
profile's skills directory. It does not read `~/.hermes/sub/active_profile`.
Without an outer named selection, this override uses `~/.hermes/sub/config.yaml`.
Use a home outside the platform default tree for an independent sticky selection.
This describes REA's current routing, not full Hermes CLI equivalence.
At Hermes commit `748c79a1`, the
[sticky lookup](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_cli/main.py#L612-L632)
also reads the outer selection, but
[`resolve_profile_env`](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_cli/profiles.py#L2431-L2461)
resolves it beneath the configured `HERMES_HOME`. For nested overrides, confirm
the profile Hermes actually uses and point `HERMES_HOME` directly at that
`profiles/<name>` directory before setup. Compatibility is tracked in
[#3939](https://github.com/morluto/rea/issues/3939).
REA resolves existing ancestor directories before checking physical containment,
including symlinks and junctions, even when the override directory does not exist.
An override beneath a symlinked default home therefore uses the default home's
sticky selection. If `~/.hermes/link` points outside the default home,
`HERMES_HOME=~/.hermes/link/new` instead uses that override root, whether or not
`new` exists. Previewed destinations retain the configured path spelling.
An unreadable path or dangling alias leaves the home unresolved; repair that path
before configuring Hermes. Inspect setup's previewed destinations before applying
changes.
An explicit `HERMES_HOME` pointing to `profiles/<name>` selects that named
profile directly, bypassing sticky selection. An invalid, missing or deleted
sticky profile is reported instead of writing into the default home; repair the
profile or run `hermes profile use default` before retrying. It also remains an
unresolved skill selection, so setup cannot report a successful empty installation.
REA follows `HERMES_HOME` whitespace trimming, environment-variable expansion,
and home expansion before resolving that profile. Setup previews the resolved
configuration and skill destinations; doctor and uninstall use those same paths.
@@ -524,10 +558,12 @@ installs its skill copy there.
## Factory Droid
For [Factory Droid](https://docs.factory.com/harness/mcp), setup writes
`mcpServers.rea` into the user-level `~/.factory/mcp.json`. Entries carry
`command`, `args`, and `env`. Droid discovers personal skills from
`~/.factory/skills` ([skills](https://docs.factory.com/harness/skills)), so
setup installs its skill copy there.
`mcpServers.rea` into the user-level `~/.factory/mcp.json` (the
`FACTORY_HOME_OVERRIDE` override replaces the home that contains `.factory`).
Entries carry `command`, `args`, and `env`. Droid discovers personal skills
from `~/.factory/skills` ([skills](https://docs.factory.com/harness/skills),
following `FACTORY_HOME_OVERRIDE` when set), so setup installs its skill copy
there.
## Trae
@@ -703,10 +739,6 @@ can override an inherited disable. Lower files and other server entries remain
unchanged; a legacy server named `servers` that conflicts with the native table
requires manual repair before setup can write.
When Hermes aliases refer to an anchored REA entry, refreshing or removing REA
retains the aliases' original resolved values. Setup does not redirect another
server's alias to REA's new launcher or policy.
With no client selected, the interactive skill choice targets the shared
`~/.agents/skills` copy. Selecting a client targets its personal destination
when supported. Use `rea setup --skill-client shared --skill` for the shared
+4 -1
View File
@@ -36,7 +36,7 @@ The MCP tool `project_harmony_application_graph` takes the same
- exact component paths, artifact IDs, and hashes (manifests, Ark bytecode,
resources, native libraries, JavaScript, signing)
- a packaging-model hint: Stage is claimed only from a `module.json` entry
- a packaging-model hint: Stage is claimed only from an exactly cased `module.json` entry
outside a classified resource path, FA only from `config.json` outside a
classified resource path, otherwise `unknown`
- FA resources under `assets/<module>/resources/` as well as Stage `resources/`
@@ -46,6 +46,9 @@ The MCP tool `project_harmony_application_graph` takes the same
- path-based N-API bridge hypotheses between bytecode and `libs/<abi>` native
libraries
An unavailable manifest can still supply a path-based model hint; its content
identity remains unknown. Manifest names follow case-sensitive ZIP member names.
File and slice occurrences supply component and child-package paths; directory
trees and symlinks retain their identities in the source inventory without
being classified as code files. Ark and JavaScript runtime hints summarize the
+1 -1
View File
@@ -240,7 +240,7 @@ The verifier requires the resolved procedure in both inventory and procedure sea
The initial real workflows use stdio registrations for the legacy Windows GUI profile and a Windows x64 headless supervisor with IDA 9.3. The headless workflow has also passed on macOS arm64 with IDA Professional 9.1 (build 9.1.250226) against the same upstream commit `c133c3853faa111a9b00ee615c013b720d0c4acd`, using a source-built arm64 Mach-O fixture; Mach-O C symbols carry a leading underscore, so the procedure name is `_rea_fixture_add`. Streamable HTTP is tested against a loopback MCP fixture with the pinned client SDK; a real IDA HTTP-supervisor workflow remains unverified. Linux headless engine operation, other IDA releases, architectures, and modern attached GUI tools require their corresponding real workflows before support is claimed.
Native Windows verification uses IDA Pro 9.3 with the public two-function fixture and Windows `whoami.exe`, using upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c` with a loopback-listener change. It covers production CLI/MCP parity, distinct-function switching, identity and connection cleanup. See the [correctness and performance measurements](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). This follow-up did not run Windows IDA Free or native macOS.
Native Windows verification uses IDA Pro 9.3 with the public two-function fixture and Windows `whoami.exe`, using upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c` with a loopback-listener change. It covers production CLI/MCP parity, distinct-function switching, identity and connection cleanup. See the [correctness and performance measurements](https://github.com/morluto/rea/blob/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency/ida-native-windows.md). This follow-up did not run Windows IDA Free or native macOS.
Headless readiness distinguishes an explicit `auto_analysis_ready: false` from
unknown readiness (`null`). The former returns a retryable capability refusal
+107 -7
View File
@@ -3,13 +3,14 @@
Run REA, connect your selected agents, and check the tools needed for your task.
Setup previews configuration changes and asks for approval before applying them.
| Task | Section |
| --------------------------------- | --------------------------------------------------------------------------------------------------- |
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](ida-provider.md), or [Kuna](kuna-provider.md) |
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
| Task | Section |
| ---------------------------------- | --------------------------------------------------------------------------------------------------- |
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
| Install a Claude Code/Codex plugin | [Agent plugins and marketplaces](#agent-plugins-and-marketplaces) |
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](ida-provider.md), or [Kuna](kuna-provider.md) |
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
## Start setup
@@ -444,6 +445,9 @@ and other MCP servers. Purging removes REA's legacy cache and state under
archives under `${XDG_CACHE_HOME:-~/.cache}/rea/hopper-releases`. Other cache
namespaces remain intact. A caller-selected `REA_KUNA_CACHE_DIR` is retained
and reported because REA cannot establish ownership of its entire contents.
Lexically equivalent spellings of that default directory, including a trailing
separator, a `.` segment, or a repeated separator, share its purge result.
The comparison preserves POSIX `..` traversal and literal backslashes.
A client configuration that is malformed, unreadable, or at an unsafe
path stops the operation before anything is removed, and a client that fails
while being updated stops the remaining removals. Uninstall reports each
@@ -1433,6 +1437,21 @@ current setting. For Windows PE resource analysis, the exact name is
`WindowsResourceReference`. Disabling an analyzer skips the facts it would have
produced.
Ghidra's Objective-C Message Analyzer can log
`Required Objective-C data type not found in data type manager` /
`Try adding libobjc.dylib` when a Mach-O links Objective-C runtime symbols but
defines no class metadata (thin Cocoa glue in a C++ binary is a common case).
That text is non-fatal Ghidra analyzer output: auto-analysis continues, and the
analyzer simply skips `objc_msgSend` call-target recovery. REA's headless import
does not accept a "add this library" option, and on modern macOS
`/usr/lib/libobjc.A.dylib` exists only in the dyld shared cache, so there is no
on-disk library to add. Treat the line as a coverage limitation, not an REA
failure; check the session's `failure_kind` (and related provider errors) for
any real abort. To silence that analyzer, set
`REA_GHIDRA_DISABLED_ANALYZERS="Objective-C Message Analyzer"`. For large
executables that miss the startup deadline instead, raise
`REA_GHIDRA_STARTUP_TIMEOUT_MS` and the [Ghidra heap](#ghidra-heap-and-cpu-controls).
### Ghidra heap and CPU controls
Set resource controls in the environment that launches the CLI or MCP server,
@@ -1534,6 +1553,87 @@ read-only operations, digest identity, and runtime cleanup. The separate
ordinary user. It requires the built Windows native bundle and an existing
Ghidra/JDK installation; neither is inferred from startup alone.
## Agent plugins and marketplaces
REA's npm package is the shared Claude Code and Codex plugin artifact. It
contains `.claude-plugin/plugin.json`, `.codex-plugin/plugin.json`, and the
canonical generated `skills/` tree. Both manifests select `./skills` and run a plain Node bootstrap pinned to
that exact package version. The client must have a supported standalone `node`
on its PATH. Claude expands its documented `CLAUDE_PLUGIN_ROOT` in the script
argument; Codex resolves `cwd: "."` from the installed plugin root before
launching the relative script. The bootstrap verifies the package version and runs its bundled runtime in
the same process, preserving stdio and REA's provider/worker cleanup. It does
not start another package runner or download a different runtime. The existing release skill
generator also pins CLI recovery commands to the same version.
The repository's single-plugin marketplaces are declared in
`.claude-plugin/marketplace.json` and `.agents/plugins/marketplace.json`.
Both select an exact `rea-agents` npm version instead of mutable source-checkout
contents. Release Please updates their selectors alongside the plugin manifests.
No plugin-only publication or generated skill files committed to Git are needed.
The qualified release archive contains a runtime-only manifest and
`npm-shrinkwrap.json` plus bundled runtime dependencies, including PTY packages
for each supported OS and CPU; development dependencies and development-only overrides
stay in the source manifest and lockfile. Both clients receive this same archive.
This distribution becomes installable after the first qualified npm release
containing these plugin manifests. Earlier published packages are not changed
by this source proposal. For local source development, run `npm ci && npm run build:cached` to generate
the runtime and packaged skill before loading the repository as a plugin.
A source plugin uses the locally built runtime and generated skill; rebuild after
source changes. Missing builds and mismatched version pins fail with an actionable
error. The authored `.agents/skills/` tree is not the plugin's shipped skill layout.
Windows source workflows that need native controls still require the
[Windows source-checkout build](https://github.com/morluto/rea/blob/main/native/windows/README.md#build).
In Claude Code:
```text
/plugin marketplace add morluto/rea
/plugin install rea@rea
```
In Codex CLI, add the marketplace and install REA:
```bash
codex plugin marketplace add morluto/rea
codex plugin add rea@rea
```
Each release updates both manifest versions and their pinned server version
together, which is when installed copies refresh the bundled skill and move to
the new server. A plugin does not run `rea setup`, so provider
configuration still follows the sections above. Use the plugin or a
`rea setup` registration for a client, not both, to avoid two `rea` servers.
`rea doctor` does not see plugin-owned state. Its client and skill checks read
only the registrations and skill locations that `rea setup` manages, so for a
plugin-only install `rea doctor --client <id>` and `rea doctor --skill` report
`missing` and recommend `rea setup`. Do not follow that remediation. Check the
plugin with the client instead (`claude plugin list` and `/mcp` in Claude
Code, `codex plugin list` in Codex).
For analysis engines, ask the connected server: have the agent call
`binary_session` and read `result.tool_availability`, which reports what the
server the plugin started can use, with reasons and remediation. See
[MCP contracts](mcp-contracts.md).
A plugin does not put `rea` on your `PATH`. To check an engine from a
terminal, use the package runner at the version the plugin pins:
<!-- x-release-please-start-version -->
```bash
npx -y [email protected] doctor --provider ghidra --json
```
<!-- x-release-please-end -->
If your installed plugin is older, use the version in its manifest instead.
That report describes the terminal's environment, not the plugin server's.
The server inherits the client's environment, so a setting such as
`GHIDRA_INSTALL_DIR` that exists only in your terminal does not reach it.
## MCP Registry
REA is published in the official MCP Registry as `io.github.morluto/rea`. Registry
+13 -2
View File
@@ -1248,11 +1248,22 @@ failures for such entries name this cause and the operation to rerun with
Advertised input and output schemas contain no reachable recursive references.
Input schemas inline repeated definitions before applying the selected
presentation profile. Both `full` and `compact` start from this self-contained
projection; output schemas can share schema-local definitions. Full inputs
retain nested fields, descriptions and valid literal examples. The compatibility
projection; output schemas can share schema-local definitions. Analyst-authored
full inputs retain nested fields, descriptions and valid literal examples. The compatibility
projections below can relax advertised constraints; canonical runtime validation
still enforces the complete input contract.
`compare_web_captures` declares its complete producer observations as object
operands instead of repeating their full output schemas in the input catalog.
Pass the original `inspect_web_page` normalized result in `before.inspection`
and `after.inspection`, optional complete `discover_webmcp_tools` results in
`webmcp`, or complete `capture_browser_scenario` results in the scenario fields.
Do not construct partial observations from the abbreviated declaration: the
server still validates every canonical field before comparison. Input groups
and analyst-authored normalization rules remain explicit. This presentation
changes no capture bytes, identity, completeness, or comparison semantics and
does not reduce `inspect_analysis_view` selectors or other tools' declarations.
When a host forwards input schemas to the model, their examples consume context.
REA's advertised examples therefore never copy complete producer Evidence.
An embedded Evidence record is shown as its exact `{"kind":"retained-evidence","evidence_id":"ev_..."}`
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
-112
View File
@@ -1,112 +0,0 @@
{
"schemaVersion": 1,
"repository": "morluto/rea",
"version": "6.7.0",
"base": "2c4338db1fdb7773e05376301c558cff2a0b4b80",
"target": "b83b9caf4a5f0241a5fd370792f1aed1bffbb2cf",
"commits": [
{
"sha": "8cbaf6c7021b97dd4de6b5771c35fbed06b240e9",
"subject": "fix(tests): isolate command finalization module mocks",
"body": "fix(tests): isolate command finalization module mocks\n\nMove the five registered WASM finalization cases into the existing per-file isolated project. Preserve all result, exit-code, observation and cleanup assertions. A preloaded shared CLI graph reproduces three failures before the move; the isolated covered packet passes.\n\nFixes #4959\n",
"parents": ["2c4338db1fdb7773e05376301c558cff2a0b4b80"],
"authors": [
{
"name": "morluto",
"email": "[email protected]",
"user": {
"login": "morluto",
"__typename": "User"
}
}
],
"pullRequests": []
},
{
"sha": "6b180783ed04ebc87548c8385c80e05725804c47",
"subject": "fix(tests): budget CLI coverage from parsed options",
"body": "fix(tests): budget CLI coverage from parsed options\n\nUse the pinned Vitest parser for coverage and shard flags. Keep ordinary CLI workers at 384 MiB and covered workers at the tested 512 MiB with unchanged concurrency. Retain complete coverage and add actual-config regressions for false, provider-only and spaced-shard options.\n\nValidation: 27 focused cases; 16 covered cases with CLI preloaded; 5 docs checks; typecheck/lint; exact discovery of 1578 files. Measured covered control: 22.18s elapsed, 46.52s CPU, about 2.33 GiB sampled summed process-family RSS on macOS arm64. Full Linux CI pending. No runtime/device verification claimed.\n\nFixes #4960\n",
"parents": ["8cbaf6c7021b97dd4de6b5771c35fbed06b240e9"],
"authors": [
{
"name": "morluto",
"email": "[email protected]",
"user": {
"login": "morluto",
"__typename": "User"
}
}
],
"pullRequests": []
},
{
"sha": "bb212ffecb47e129cc569b9867b2b6e7c9b65685",
"subject": "fix(tests): inherit server identity deadline policy",
"body": "fix(tests): inherit server identity deadline policy\n\nRemove per-case ten-second overrides from MCP catalog and identity checks. Keep the suite-owned bounded ordinary and coverage deadlines, all assertions, and session cleanup.\n\nFixes #4967\n",
"parents": ["6b180783ed04ebc87548c8385c80e05725804c47"],
"authors": [
{
"name": "morluto",
"email": "[email protected]",
"user": {
"login": "morluto",
"__typename": "User"
}
}
],
"pullRequests": []
},
{
"sha": "b83b9caf4a5f0241a5fd370792f1aed1bffbb2cf",
"subject": "fix(setup): edit the duplicate JSON member that clients read (#4957)",
"body": "fix(setup): edit the duplicate JSON member that clients read (#4957)\n\nJSON.parse, JSONC parse, and native Claude Code keep the last duplicate\nobject member, and parseClientConfiguration therefore validates the last\none. The shared setup/uninstall serializer instead edited text through\njsonc-parser's modify and findNodeAtLocation, which select the first\nmatching member, so setup and uninstall on configurations with repeated\nserver tables or rea members wrote into a shadowed member and failed\nreadback.\n\nLocate every JSON text edit by the effective (last) member in one place,\nreplacing the first-member modify path. Removing an effective member\nthat shadows an earlier duplicate would activate that duplicate, whatever\nits ownership, so the serializer now returns a tagged shadowed_member\nfailure and setup and uninstall refuse before writing a backup or the\nconfiguration.\n\nFixes #4952\n\nCo-authored-by: Claude Opus 5.5 <[email protected]>",
"parents": ["bb212ffecb47e129cc569b9867b2b6e7c9b65685"],
"authors": [
{
"name": "Kaoru0822",
"email": "[email protected]",
"user": {
"login": "kaoru0822-kitauji",
"__typename": "User"
}
},
{
"name": "Claude Opus 5.5",
"email": "[email protected]",
"user": {
"login": "claude",
"__typename": "User"
}
}
],
"pullRequests": [4957]
}
],
"pullRequests": [
{
"number": 4957,
"title": "fix(setup): edit the duplicate JSON member that clients read",
"url": "https://github.com/morluto/rea/pull/4957",
"author": {
"login": "kaoru0822-kitauji",
"__typename": "User"
},
"mergeCommit": {
"oid": "b83b9caf4a5f0241a5fd370792f1aed1bffbb2cf"
},
"issues": [
{
"number": 4952,
"url": "https://github.com/morluto/rea/issues/4952",
"author": {
"login": "xu91102",
"__typename": "User"
},
"repository": {
"nameWithOwner": "morluto/rea"
}
}
]
}
]
}
+19 -1
View File
@@ -349,7 +349,10 @@ tag remains in main's ancestry. Keep the released tag unchanged.
The synchronization must include `.release-please-manifest.json`,
`package.json`, both root versions in `package-lock.json`, `CHANGELOG.md`,
`server.json`, and the versioned documentation examples. Run
`server.json`, both plugin manifests (`.claude-plugin/plugin.json` and
`.codex-plugin/plugin.json`), the Codex server declaration (`.codex-plugin/mcp.json`), both
marketplaces (`.claude-plugin/marketplace.json` and
`.agents/plugins/marketplace.json`), and the versioned documentation examples. Run
`npm run docs:generate`, `npm run docs:check`, and the release configuration
tests against the combined tree. After the merge, verify that the released
tag is an ancestor of main:
@@ -383,3 +386,18 @@ tag. Do not dispatch a fresh publish phase to repair an already-created
release: Release Please will not create the same release again. Do not move
the tag, delete the release, or unpublish npm as a retry. A defective public
package requires a reviewed correction and a new version.
### Plugin artifact metadata
Package verification selects files with npm, then packs an owned staging directory.
The release manifest omits development dependencies and development-only overrides;
its generated `npm-shrinkwrap.json` preserves the runtime dependency resolutions
and integrity values from `package-lock.json`. Runtime or unresolved overrides
fail packaging rather than being dropped. Source metadata is never rewritten.
The archive bundles production dependencies because Codex fetches npm plugins
without installing dependencies. The producer includes all locked PTY platform
archives after checking their integrity, without re-resolving the runtime tree.
Optional Frida stays outside the bundle so ordinary npm installation can retain
its host-specific installer. This also lets Claude install locked dependencies under its plugin policy while
retaining development security overrides. Both plugin marketplaces select the
same qualified npm artifact, which the release workflow publishes unchanged.
+20 -5
View File
@@ -5,6 +5,12 @@ lane. Start with the commands below; use [real-toolchain lanes](#real-toolchain-
for engine prerequisites and [client verification](client-verification.md) for
installed-client and agent runs.
Historical verification records may refer to `experiments/analysis-efficiency`.
Those experiment scripts are available in the [archived source tree](https://github.com/morluto/rea/tree/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency);
they are not included in current checkouts. The retained [test support](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/README.md)
contains deterministic checkers and fixtures. The [allocation report](verification/analysis-efficiency/mcp-integer-allocation.md)
and its original receipt JSON remain documentation, checked by the report producer.
## Developer commands
Use source feedback while editing, explicit boundary checks for the changed
@@ -204,7 +210,11 @@ changing a callback solely to satisfy its declared `void` return type.
Ordinary Knip remains. `check`, `check:pr`, and `check:ci:deep` run reachability
then typed lint sequentially after cached work. Typed lint builds the project
type graph and stays separate from fast feedback. Its default Go settings are
`GOMAXPROCS=2` and `GOMEMLIMIT=1024MiB`; caller-selected settings take precedence.
`GOMAXPROCS=1` and `GOMEMLIMIT=1024MiB`; caller-selected settings take precedence.
One Go worker reuses checker caches across the file queue without changing the
file inventory or TypeScript project scope. The Go memory target is soft and
does not cap the complete process family's memory; measure the full invocation
when enforcing a host budget.
Routine iterations and rebases need focused regressions and relevant checks.
Before handing off a PR, run `npm run check` and generated-document checks when
@@ -1164,8 +1174,8 @@ its metadata regressions against the same checksum-pinned engine and public APK.
It runs on branch pushes affecting its bridge, fixture or engine inputs, and can
also be dispatched manually or called by another workflow. It uses the runner's
JDK with two active processors and a 512 MiB heap; it needs no npm dependency
installation. This lane checks exact raw/display class selectors and the existing
cold-metadata regressions. It does not establish CLI/MCP parity or application
installation. This lane checks exact raw/display class selectors, raw member names and field
DEX types used by mapping lookups, and the existing cold-metadata regressions. It does not establish CLI/MCP parity or application
execution behavior. The broader `verify:android` lane checks CLI/MCP parity;
neither lane runs the APK.
@@ -1211,7 +1221,7 @@ HarmonyOS `.har` libraries are deliberately not suffix-classified because
Synthetic producer regressions run independently:
```sh
npm run test:focused -- tests/boundary/process/jadxIntegration.test.ts tests/boundary/process/androidAnalysisMcp.test.ts
npm run test:focused -- tests/boundary/process/jadxIntegration.test.ts tests/boundary/process/androidAnalysisMcp.test.ts tests/boundary/process/androidMappingMcp.test.ts
```
## Apktool resource decoding
@@ -1374,7 +1384,7 @@ other engine versions and architectures remain unverified; see the
Native IDA MCP verification uses `"protocol": "native"` with an already-running native plugin and requires `--second-procedure other_function --second-marker unique_second_body_text` to exercise GUI function switching and a return to the first function. These options remain optional for other profiles. The literal marker must appear in the second function's pseudocode and be absent from the first. Native cloud/widget failures are reported without hidden retries.
The initial Linux x64 IDA Free 9.4 run used native MCP 1.1.2 with a local navigation correction. Windows x64 verification uses **IDA Pro 9.3**, the two-function public fixture, and the shipped Windows `whoami.exe` application. The native plugin was built from upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c`, including the merged navigation fix, with a local loopback-listener change. Production CLI/MCP parity, switching, identity and cleanup passed. Correctness and performance measurements are recorded in the [Windows native verification notes](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). Stock 1.1.2 switching, native macOS, and Windows IDA Free remain unverified. See the [native bridge prerequisites](ida-provider.md#optional-native-ida-mcp-bridge).
The initial Linux x64 IDA Free 9.4 run used native MCP 1.1.2 with a local navigation correction. Windows x64 verification uses **IDA Pro 9.3**, the two-function public fixture, and the shipped Windows `whoami.exe` application. The native plugin was built from upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c`, including the merged navigation fix, with a local loopback-listener change. Production CLI/MCP parity, switching, identity and cleanup passed. Correctness and performance measurements are recorded in the [Windows native verification notes](https://github.com/morluto/rea/blob/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency/ida-native-windows.md). Stock 1.1.2 switching, native macOS, and Windows IDA Free remain unverified. See the [native bridge prerequisites](ida-provider.md#optional-native-ida-mcp-bridge).
## Kuna lane
@@ -1624,6 +1634,11 @@ results after a failure. Available failed-shard reports remain diagnostic only:
failed, cancelled, skipped, or missing selected shards cannot pass the gate or
publish successful timing history. This deliberately avoids relying on an
import graph for generated files, filesystem inputs, subprocesses, or bridges.
The portable report rejects an entirely skipped module inventory and lists
fully skipped modules in its summary. A mixed passed/skipped inventory is
allowed: Vitest's skip state does not establish whether a skipped module was
required on that runner. Review the listed skips against platform and provider
prerequisites; the inventory alone does not prove every required case ran.
Scheduled qualification is admitted hourly at minute 23. A forced full run at
03:23 UTC refreshes runner and dependency drift every day. Other hourly runs
@@ -22,7 +22,7 @@ Seven dependency-boundary variants match parsed values and complete error issues
## Tasks, artifacts and bounds
Reuse the shared SDK task driver and independent ELF/section/zlib/JSON oracles from [the catalog study](mcp-input-catalog.md): official Alpine v3.21 x86_64 ripgrep 14.1.1-r0 and Zellij 0.40.0-r0. All 42/422 packages and 88/1,063 edges match the independent oracles. Select serde name/version/kind, repeat exact retained-parent reads twice, verify a canonical negative-offset refusal, export/check complete Evidence and confirm the unchanged directory. `kind: null` stays unknown. Targets are not executed; dependency presence does not establish runtime use. Acquisition URLs, archive/executable/oracle hashes and complete answers are in [the receipts](mcp-integer-allocation.json). Previously acquired public bytes are rehashed, not redownloaded.
Reuse the shared SDK task driver and independent ELF/section/zlib/JSON oracles from [the archived catalog study](https://github.com/morluto/rea/blob/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency/mcp-input-catalog.md): official Alpine v3.21 x86_64 ripgrep 14.1.1-r0 and Zellij 0.40.0-r0. All 42/422 packages and 88/1,063 edges match the independent oracles. Select serde name/version/kind, repeat exact retained-parent reads twice, verify a canonical negative-offset refusal, export/check complete Evidence and confirm the unchanged directory. `kind: null` stays unknown. Targets are not executed; dependency presence does not establish runtime use. Acquisition URLs, archive/executable/oracle hashes and complete answers are in [the receipts](https://github.com/morluto/rea/blob/main/docs/verification/analysis-efficiency/mcp-integer-allocation.json). Previously acquired public bytes are rehashed, not redownloaded.
Normal cohort: three alternating baseline/candidate pairs for each task × cold/warm server compile cache × default compact/full inputs: 72 trials, plus four excluded full-task warm priming runs. Separate diagnostic cohort: three alternating pairs per profile, cold Zellij tasks, 12 trials. All pass, with identical complete parent IDs/hashes and answers.
@@ -115,6 +115,12 @@ Question, explicit recipe, server instructions, one complete name/description/in
- Both runtime builds and all six static checks plus architecture guards pass. Before the final main synchronization, 5,972 source/contract/public CLI/MCP tests pass across 323 files; four skip for unavailable/guarded coverage. After synchronization, both builds/static checks repeat successfully, 77 affected producer and CLI/MCP tests pass, and all 974 registered numeric-parameter parser cases match again. Includes real SQLite CLI/MCP record/column selection and JavaScript heap/deadline preservation. A first direct schema-validity invocation lacked generated catalog metadata; the supported focused entry generates it and all 15 cases pass. A later affected-test invocation selected two nonexistent paths; the corrected three-file producer/consumer run is the 77-case result above. Neither setup failure was counted as a passing test, and no production or timeout change was needed.
- All 84 measured tasks and four warm priming tasks pass independent oracles, canonical refusal, parent/directory identity and owned cleanup checks. This does not verify external Hopper/Ghidra/IDA, Android devices, all providers or foreign OS behavior. The earlier macOS numbers in #4360 describe a different revision/platform and are not combined with these results.
The historical cohort commands below use the retired experiment layout. Run them
from the matching recorded producer checkout; the [archived source tree](https://github.com/morluto/rea/tree/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency)
provides the former files for reference. Current checkouts retain the report and
its conformance producer, rather than the full measurement suite. Moving these
files does not requalify the historical measurements against current REA.
Build the pinned baseline/candidate in separate clean worktrees with the locked toolchain. Use the candidate’s producer scripts for both roles. `measurement_inputs` in the JSON records the complete matrix, questions/recipes, runtime bindings and producer hashes; extract it unchanged as the bindings file. Historical receipts require their matching producer revision. One normal task:
```bash
@@ -133,11 +139,12 @@ python experiments/analysis-efficiency/mcp-startup-costs.py "$RECEIPT_ROOT" \
The aggregator requires the recorded tiktoken version, exact catalog/parent/runtime/producer/guard identities, clean source, successful payload cleanup and matching matrix coverage. Default driver behavior continues without an inspector; the heap flag is confined to the experiment.
Regenerate or verify every numerical section directly from the committed JSON:
In a current checkout, regenerate or verify every numerical section directly
from the committed JSON:
```bash
node experiments/analysis-efficiency/mcp-integer-report.mjs
node experiments/analysis-efficiency/mcp-integer-report.mjs --check
node tests/support/analysis-efficiency/mcp-integer-report.mjs
node tests/support/analysis-efficiency/mcp-integer-report.mjs --check
```
The documentation conformance gate checks current receipts, rejects stale normal/heap tables, and refuses missing producer markers without writing.
@@ -0,0 +1,320 @@
# Static Android JNI verification, 2026-10-27
Issue #6403 asks which bundled native functions could implement original DEX
native declarations. The implementation reads immutable APK bytes, preserving
declarations, library occurrences, exported/table candidates and selected
per-ABI bindings as independently queryable facts. No target, library,
`JNI_OnLoad` or `RegisterNatives` code was executed.
## Revisions and independent oracles
The full qualification and alternating delivery measurements below use
implementation `4a1839de6f091ade19e82557874c358ed34b8e36`, based on main
`03f17cf41d3c844f9383d7792d9b1a14330d0bb2`. Subsequent merge
`e8a8a50725cd384ae38ce66a94bafdaba1138108` incorporates main
`d6f88becd531cabafca1d968b89cf7fc77a80041`; its browser input-presentation and
SQLite test changes are separate from JNI analysis. Post-merge confirmation is
recorded below. The native parser's built SHA-256 is
`3bf0a2c342f1c404f5861b7c913f7f1b931f740a8c808b518474fc8de20821d5`.
Linux x86-64, Node 24.22.1/npm 11.16.0 and JDK 21.0.12.1 qualified the workflows.
JADX MCP 0.7.1 remains the supplied unmodified JAR, SHA-256
`6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75`.
The REA metadata bridge is version 9. JNI declarations use the JAR's existing
dexlib2 reader before decompiler deduplication, with no `load_apk` call.
The genuine Android NDK r27d package is revision 27.3.13750724, clang/LLD 18.0.4.
Its official Linux ZIP is 663,956,036 bytes, publisher SHA-1
`22105e410cf29afcf163760cc95522b9fb981121`; observed SHA-256 is
`601246087a682d1944e1e16dd85bc6e49560fe8b6d61255be2829178c8ed15d9`.
Pins come from the [official repository manifest](https://dl.google.com/android/repository/repository2-3.xml).
The package was streamed, hashed and selectively extracted into a private task
cache, without installing or changing global tools. The selected compiler,
sysroot and four additional official libunwind archives total 319,933,537 bytes.
An initial link failure exposed those omitted archives; the corrected fixture
build passed in 4.606 seconds, CPU 6.026 seconds, peak family RSS 110.4 MiB.
[Committed source fixtures and reproduction](https://github.com/morluto/rea/blob/main/tests/fixtures/android/jni-bindings/README.md)
build native declarations, javac headers and two libraries for arm64-v8a,
armeabi-v7a, x86 and x86_64. Ordinary REL/RELA, RELR and Android packed APS2
variants include overloads, BMP/supplementary names, explicit Modified UTF-8,
`.data` and `.data.rel.ro` tables, repeated exports in two libraries, an
unmatched declaration and two classes sharing one registered pair. Source,
`javac -h`, independent DEX flag/index reads, readelf and a separate byte/relocation
oracle establish expected facts. Oracle SHA-256 is
`fa634493bc8559a80eda3b70f01d44550c2f06e176cfc2a7cc7e10794b1eb793`.
| Static APK | SHA-256 |
| -------------------- | ------------------------------------------------------------------ |
| Ordinary | `c4eb50ef8bdecc6f56c454294c0c8f198fa51a4174d101dc720a849976546649` |
| RELR | `5a2ce56191f633575e87bd239702f988d0d1ad82795a203e13a4f907f8624658` |
| APS2 | `d2f0d613bc8f22ed1a6435f13b5ae1f088e904216aa8a04dd70c27c5cb606778` |
| Conscrypt derivative | `5985c40d5a6b14b64c202c80335d851adb7e8a987e6154a1488a194253856385` |
The Conscrypt derivative packages the public Android 2.8.0 AAR's original
library bytes with its class files transformed by D8 8.0.40. It is a static
test container, not a vendor APK or a claimed launchable application. The AAR
SHA-256 is `752044c289bc98e2dcabc00a1f88ca158b3aa3f505b38f124552bf4b2a0902b1`.
The original-name DEX is used, not an R8-renamed DEX paired with unchanged JNI
strings. D8's missing platform-adapter dependency warning remains a qualification
limit; no full application linkage or signature verification is claimed.
## Executed correctness and Evidence checks
The built pure parser passed independent checks over 25 real libraries:
24 genuine NDK outputs plus Conscrypt arm64. All 33 golden table entries match
their strings, slot addresses/file offsets, relocated values, encodings,
function addresses and code offsets. Conscrypt yields 367 structural entries;
the requested `EVP_PKEY_new_RSA ([B[B[B[B[B[B[B[B)J` candidate is `0xfa11c`.
Its name/signature/function slots are `0x2ada68`, `0x2ada70`, `0x2ada78`,
with relocated values `0x749a8`, `0x7affa`, `0xfa11c`. This built-artifact
qualification took 0.684 seconds, CPU 0.736 seconds, peak family RSS 99.5 MiB.
Real SDK/stdio MCP and CLI checks independently validate:
- All 12 source native declarations across four ABIs, exact library digests,
overloads, UTF-16 names and per-library alternatives. Ordinary/RELR return
32 matched, 12 ambiguous and four unknown rows. The source-known unmatched
method remains unknown because symbolic writes/NOBITS leave scanning partial.
- APS2 preserves 20 observed export matches, four ambiguous export rows and
24 unknown rows. It explicitly reports unsupported table relocation encoding,
retaining each selected library's reason even without a positive candidate.
- Class selection never hides the other class sharing a registered pair.
All selected native declarations survive independently of available ABI rows;
the no-library selection case is covered by a domain regression, not claimed
as a separate real SDK artifact qualification.
- All four retained collections support exact fields, empty filters and batch
ID retrieval. Parent-array `row_index` remains separate from raw DEX/ZIP
source ordinals. Advertised converted schemas validate with the real SDK/Ajv.
- Complete portable parents preserve the same normalized facts through summary
and CLI delivery. Ordinary and each Conscrypt cohort copy the source to an
exclusively created owned file, inspect it, remove only that copy, then read
retained summary, exact table slots, binding fields and a full export.
Original fixture bytes remain unchanged. Post-removal success is observed;
hidden internal engine activity is not directly instrumented by this harness.
The first three four-ABI cohorts execute 107 SDK calls and three cold CLI parity
checks. Each Conscrypt cohort adds 28 investigation calls, five separately
accounted retained-removal calls and one CLI parity check. Four alternating
Conscrypt cohorts preserve the same normalized fact digest:
`7e4738d2dd9e518277d5268ce5c334d9d25728198f66b2102cdc3411865f2d46`.
Raw parent transcripts show one native inventory RPC on the first investigation,
then only JVM-status RPCs on later main queries; normalized native facts remain
present. No JNI parent contains `load_apk`. Library reuse is owned by the same
immutable APK session; ABI/library selection replaces one scan, while class
and method selection reuses it. Fresh analysis admission still hashes current
caller APK bytes. Historical views never reopen them.
Type/lint/build and 396 tests across ten exact files pass together: native/domain
projection, producer parsing, real ZIP occurrence/cleanup boundaries, unchanged
JADX session/idle/cancellation behavior, public contracts and existing IL2CPP
decoding/ownership. The gate took 215.198 seconds, CPU 257.470 seconds, peak
family RSS 4396.1 MiB. A full real JDK/JADX metadata run additionally passed
native reads before any APK load, malformed/duplicate DEX and CRC negatives,
raw aliases, constants, absence facts, manifest identity and downstream CLI/MCP
bodies/direct-invoke parity from both DEX entries: 154.890 seconds, CPU 233.275
seconds, peak family RSS 984.8 MiB.
Finalization regressions retain already parsed library facts/digests in typed
partial observations before ZIP close. Caller/deadline-first cancellation,
failed close, blocked admission and retry preserve their first cause and exact
resource lease. Interrupted later libraries preserve earlier completed facts.
Stable table candidate identity uses physical pointer-slot address/file offset,
not inferred run grouping. Known ZIP size claims stay separate from verified
read size/hash.
## Completed-task costs
Earlier main cannot decode/register JNI tables with equivalent coverage, so
manual export lookup is not an equivalent before workflow. The controlled
comparison is complete versus summary delivery of the same new investigation,
same bytes, query, selected pages and proof export. Conscrypt trials alternate
summary/complete/summary/complete, each with one cold and one warm task.
Each fixture task has four investigations, 12 selected collection pages and a
proof export (17 calls). Each Conscrypt task has one investigation, the same
12 page checks and proof export (14 calls). Task time includes public calls and
local oracle checks; one-time startup/catalog, separate retained-removal
qualification, final cleanup and a cold CLI parity run are included in the
whole-process columns, not hidden in task timings. Ordinary includes removal
qualification; RELR/APS2 do not, so their whole-process costs are not a paired
performance comparison. All four Conscrypt cohorts use the same phase flags.
| Cohort | Cold task s | Warm task s | Whole s | CPU s | Peak family MiB | Warm text tokens | Warm SDK tokens |
| -------------------- | ----------: | ----------: | ------: | -----: | --------------: | ---------------: | --------------: |
| Ordinary | 11.058 | 0.606 | 32.658 | 45.124 | 1215.7 | 58,319 | 401,681 |
| RELR | 12.140 | 0.797 | 29.122 | 38.375 | 1266.2 | 58,231 | 401,502 |
| APS2 | 12.578 | 0.917 | 30.892 | 41.071 | 1268.0 | 50,900 | 386,634 |
| Conscrypt summary | 11.237 | 0.444 | 33.999 | 46.953 | 1282.0 | 32,265 | 349,128 |
| Conscrypt complete | 10.846 | 0.456 | 32.907 | 45.948 | 1398.3 | 224,954 | 737,723 |
| Conscrypt summary 2 | 19.205 | 1.200 | 57.545 | 77.644 | 1456.2 | 32,223 | 349,055 |
| Conscrypt complete 2 | 20.978 | 0.925 | 64.513 | 81.218 | 1502.0 | 224,526 | 736,950 |
Warm text estimates are about 85.7% lower for summary delivery. Cold text
estimates are 32,087/32,202 versus 247,593/247,912. SDK-inclusive warm estimates
are about 52.6% lower. Latency/CPU/peak-memory samples vary substantially and do
not establish a general improvement. Cold costs include the first generic
view-schema compilation in the client; subsequent tasks reuse it. Startup/
catalog alone ranges 2.692–4.955 seconds across these Conscrypt cohorts.
Token estimates use tiktoken 0.14.0/o200k_base. Every task counts its analyst
question, discovered input catalog once, arguments and all text replies,
including pages and export receipts. SDK totals additionally count output
schemas and structured-content duplication. Qualifications and CLI response
costs are reported separately by the cost script. Exported file contents are
verified locally and are not counted as model context unless actually returned
inline. Actual billing, hidden prompts, client rendering/pruning and OS cache
state are unmeasured.
### Latest-main confirmation
After merging main `d6f88becd` into `e8a8a5072`, type/lint/build and the eight
input-presentation/public-contract cases pass. The first direct schema-suite
invocation could not load its generated test catalog; `test:focused` supplied
the required artifacts and all 16 schema tests then passed (63.310 seconds,
CPU 72.830 seconds, peak family RSS 1943.5 MiB). No schema assertion was relaxed.
Fresh full Conscrypt SDK/CLI cohorts use the same proof, 12 pages and owned-copy
removal flags. Both retain the identical fact digest above:
| Delivery | Cold task s | Warm task s | Whole s | CPU s | Peak family MiB | Warm text tokens | Warm SDK tokens |
| -------- | ----------: | ----------: | ------: | -----: | --------------: | ---------------: | --------------: |
| Summary | 11.739 | 0.680 | 36.712 | 51.209 | 1395.1 | 31,932 | 348,522 |
| Complete | 11.161 | 0.459 | 34.692 | 48.319 | 1385.2 | 224,954 | 737,723 |
Summary text estimates remain about 85.8% lower. This pair does not show a
latency or peak-memory improvement. Each adds 33 SDK calls and one cold CLI
parity check, bringing the executed public qualification to 305 SDK calls and
nine CLI checks across the pinned cohorts.
`docs:check`, `format:check` and `docs:build` pass together: 44.365 seconds,
CPU 56.935 seconds, peak sampled family RSS 1300.6 MiB. Generated catalogs,
schemas, packaged skills and provider binaries remain uncommitted outputs.
All expensive checks use CPU affinity 0,1, bounded heaps and one active process
family, with an 8 GiB RSS guard and 4 GiB host-availability guard. Peak memory is
the sum of family RSS sampled every 100 ms: shared pages can be double-counted
and short peaks can be missed. No guard stop or OOM occurred.
### Default-summary delivery confirmation
The new MCP operation now defaults to retained `summary` when `detail` is omitted.
Explicit `complete` remains supported; the CLI retains its common explicit-detail
policy. The actual SDK regression verifies both modes, caller selectors and
reading an unselected declaration from the complete parent after source removal.
Type/lint/build and all five cases in that MCP boundary file pass: 80.012 seconds,
CPU 110.240 seconds, peak sampled family RSS 4747.6 MiB. An earlier attempt stopped
at the test-function length lint rule; the fixture helper was extracted before
this successful gate.
A fresh genuine Conscrypt workflow omits `detail` on JNI investigation calls and
passes 33 SDK calls, all 12 retained pages, proof export, one cold CLI parity
check and the separate owned-copy removal qualification. The same investigation
fact digest above is retained. Cold/warm task times are 15.608/0.481 seconds;
whole process-family time is 45.239 seconds, CPU 61.038 seconds, peak sampled RSS
1431.2 MiB. Warm text/SDK token estimates are 31,928/348,516 using the same cost
method. This adds a tenth CLI check and brings the successful public qualification
total to 338 SDK calls. Explicit-complete removal qualification remains separate
from the investigation task; default summaries do not discard the full parent.
Final `docs:check`, `format:check` and `docs:build` also pass: 77.269 seconds,
CPU 70.506 seconds, peak sampled family RSS 1191.1 MiB.
The updated harness adds `--omit-detail` (with `--detail summary`), SHA-256
`c2a5d090f32429c852e3c974083bb9c21a0dc6a608407687031f155a722b53d7`.
The earlier cohort tables use the original frozen harness below. Default-run
summary/catalog/call receipt SHA-256 values are respectively
`c10e4d702b9694b0ccc69926a4e1f688fa8dea33a3de2e2837ea15190a706b37`,
`0c36244f5ff788fabd5491144b0711e99ad211ce3c09e1812d1d4da5753624ac`, and
`e1f021f0a7a92f101a037a1467a3428b4c620482eb556d81501a5ea64226ab82`.
### Review corrections and final qualification
Four review findings were reproduced or confirmed at their producing boundaries:
- The shared relocation reader had charged every file-backed write against a
sparse retention limit. Its single immutable decoder now streams writes;
IL2CPP applies them to its existing private copy and retains only bounded
unresolved/zero-fill state and a file-offset bitmap. JNI keeps its bounded
sparse provenance index. The two actual IL2CPP registration-read regressions
each include 65,537 extra file-backed relative writes (AArch64 and x86-64)
and verify the original bytes remain unchanged.
- GNU version headers could replay shared auxiliary chains and names. The shared
export owner now bounds total record visits from section bytes and the existing
export-row allowance, caches decoded/failed string offsets within 16 MiB, and
yields checkpoints through the same generator used by synchronous callers.
Refusal retains exports with partial coverage and unknown version names.
Both GNU table variants exercise 10,000 headers replaying 65,535 auxiliary
records and a 65,535-byte name, verifying one decode and cooperative yields.
- A named mapped-string read-limit failure now distinguishes budget exhaustion
from invalid text. JNI preserves later candidates, marks tables partial and
projects absent candidates as unknown. Both long names and descriptors are
covered. An additional actual JNI-consumer regression queues cancellation
after GNU name decoding begins and stops before export-symbol decoding.
- The static verifier now requires the Conscrypt oracle only when expectations
advertise it. Executed controls pass with 25 libraries/33 entries when present,
and 24 libraries/32 entries for NDK-only expectations; an advertised missing
oracle rejects with ENOENT and writes no success receipt.
Type/lint/build and 330 cases in 13 related domain, parser, archive, IL2CPP and
actual CLI/stdio MCP files pass together: 192.437 seconds, CPU 224.376 seconds,
peak sampled family RSS 4190.2 MiB. After adding the during-version cancellation
case, type/lint and all 38 JNI parser cases pass: 39.842 seconds, CPU 67.581
seconds, peak RSS 3928.0 MiB. These overlap earlier gates; counts are not summed
as unique tests.
The final built parser again passes the independent 25-library/33-entry oracle:
0.975 seconds, CPU 1.009 seconds, peak RSS 103.9 MiB. Built parser SHA-256 is
`7c0638751427b2b282b8e55bb21c95f6c3f05c246b0ea66a31d0280694f9ffeb`;
its receipt also fingerprints every shared parsing dependency. The fixed oracle
SHA-256 above is unchanged. Large-capacity IL2CPP verification uses valid producer
fixtures, not a newly qualified real IL2CPP application.
A fresh Conscrypt investigation qualification again passes 33 SDK calls,
12 pages, proof, a cold CLI check and owned-copy removal. Investigation facts
retain the same digest above. Cold/warm task times are 23.529/1.015 seconds;
whole time 66.284 seconds, CPU 77.940 seconds, peak RSS 1402.6 MiB. Warm text/SDK
estimates are 31,955/348,569. Host/time variation does not establish a general
latency improvement. Cumulative successful public qualification is now 371 SDK
calls and eleven CLI parity checks, across the explicitly separated revisions.
The final cohort summary/calls SHA-256 values are
`84df4f4ed97f6d1a931010db8e16420e5f06df371940ed9225eb99a7a7c01094` and
`2c22b89e5ec4b26f15f0e9c5f8fdfd4af8583ad0926ad32edb359ecd991cd7fd`;
its catalog hash matches the default-summary cohort. No resource guard or OOM
occurred. Final `docs:check`, `format:check` and `docs:build` pass in
38.602 seconds, CPU 49.448 seconds, peak sampled family RSS 1286.2 MiB.
## Reproduction and limitations
Build the runtime, generate the source fixtures using selected existing JDK,
NDK and D8 paths, then run the maintained
[pure parser verifier](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-jni-library.mjs)
against the built parser. The
[public workflow harness](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-jni.mjs)
accepts `--variant ordinary|relr|aps2|conscrypt`, `--detail complete|summary`,
`--proof --pages --cli` and optional `--retained-removal`, explicit selected
repository/artifact/JDK/JAR/oracle paths and a fresh output directory. Frozen
harness SHA-256:
`14836895659de0f161c479b2b013ec39d754d172fd9935898872ab9cb5937035`.
The [cost script](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-jni-costs.py)
consumes every case's catalog, calls and summary receipts, fingerprints them
and keeps qualification costs separate.
Initial source integration type/lint errors were corrected before qualification.
A first CLI verifier attempt incorrectly treated its summary wrapper as flat
Evidence; the CLI had written its valid bundle. The corrected harness saves raw
stdout/stderr before assertions and validates the unwrapped named contract.
One driver used the wrong Conscrypt derivative filename and failed before
analysis; corrected cohorts use the pinned file above. A direct schema-suite
invocation lacked generated test catalog metadata; artifact-aware verification
is recorded separately. These attempts are not counted as successful runs.
Only Linux/JDK21/JADX0.7.1 and the fixed genuine NDK variants/public Conscrypt
arm64 sample are qualified. Cross-ABI ELF fixtures do not establish real
Windows/macOS provider lifecycle, device/emulator operation or runtime JNI
registration. Existing IL2CPP regressions cover the shared relocation refactor;
no new real IL2CPP application qualification is claimed. Runtime-built tables,
obfuscated strings, unsupported encodings and missing split/runtime-loaded code
remain unknown. Byte digests identify artifacts, not authenticity or build
ownership.
@@ -0,0 +1,366 @@
# Caller-selected Android mapping investigation — 2026-10-27
## Question, revisions and oracle
Apply a caller-selected local R8 mapping while preserving observed DEX identities,
original mapping-derived names/types, overloads, uncertainty, both artifact
identities and complete source Evidence. Inspect the requested inventory and
method bodies, then export complete Evidence. APKs and native code are never
executed, installed or modified.
Both controls use main `4e67bc399951768737eeaf539f7bc9fa8373b244`:
- Before: `3a588662d916f801a8c444dd86f1c65c81c76bf9`, the integrated
#7137 mapping implementation, without parsed-index reuse or retained symbol
summaries. This is a feature-equivalent optimization control, not a claim
that unmodified main already supported mappings.
- After: `6b65d3a4a1a8307417365b9a1e9060e607538aa3`, bounded/cached mapping
and explicit retained summary delivery. Later documentation changes do not
alter the measured implementation. Following main’s experiment archive, the
maintained harness/fixtures now live under tests and this report under
docs/verification; the original measured scripts remain available at that pinned revision.
- The original #7137 author is 555aaditya; integration retains their co-author
credit and preserves #7379's shared manifest contracts/default-package tests.
The [source fixture](https://github.com/morluto/rea/blob/main/tests/fixtures/android/r8-mapping/README.md) has four classes,
eight fields and fifteen methods, including constructors, overloads, Unicode
names, nested/default-package classes, arrays and renamed types in prototypes.
The same actual R8 invocation emits DEX and mapping. Handwritten source
expectations are checked against pre-minification `javap -s -p`; an independently
compiled dexlib2 reader inventories raw DEX identities without REA's parser,
bridge or JADX aliases. All four classes and fifteen selected bodies are checked
on each investigation. Seventeen valid method mapping records include three
line ranges for one constructor; they are not seventeen distinct DEX methods.
The public-library task transforms the shipped `classes.jar` from Conscrypt
Android 2.8.0 with actual R8. Its 949,641-byte mapping has 530 class, 1,263 field
and 10,342 method records. A separate D8 conversion of the original class files
provides the unminified descriptor oracle. Every selected `NativeCrypto` field
(28) and method (390), plus inner-class identity, is verified; bodies are selected
for `EVP_PKEY_new_RSA`, `EVP_DigestSignInit`, `SSL_new`, `SSL_free` and
`SSL_get_error`. These are static DEX-only ZIP containers, not runnable APKs.
D8 warned that a platform adapter's `com.android.org.conscrypt.OpenSSLSocketImpl`
was missing; complete dependency/runtime linkage is unverified. R8 additionally
used `-dontwarn **`, with shrinking/optimization disabled and name minification
allowed. Stable R8 did not consume an existing APK's DEX.
## Pinned artifacts
| Artifact | Observed SHA-256 |
| ----------------------------- | ------------------------------------------------------------------ |
| R8 8.0.40 Maven JAR | `ab1379835c7d3e5f21f80347c3c81e2f762e0b9b02748ae5232c3afa14adf702` |
| JADX MCP 0.7.1 JAR | `6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75` |
| Android 36 platform JAR | `d9eb9da824d9e247a352f570f01e1169e725b2954bca9e283a71786c59b59f9a` |
| Source fixture input JAR | `88f3d46018ac2effb166ceed79fdd6e8b58b6bb08d2649cd0aa4f90dc260260c` |
| Source fixture DEX | `0b4c49cf3d706f5c2c4c15a10ff27dfbabb9b07f463d508f6f348eafb3245909` |
| Source fixture static APK | `f8c0849978ed7eceeeb293223cf2139c1749f672148dce1e16d018fa727445c9` |
| Source fixture mapping | `5ca162587794af1bd1da00f79ee64fd4ecde6f7c91048ea5596260104df6e483` |
| Conscrypt 2.8.0 AAR | `752044c289bc98e2dcabc00a1f88ca158b3aa3f505b38f124552bf4b2a0902b1` |
| Conscrypt shipped classes JAR | `2cd9b6bc444cfca69048e8327383b7f5a13fc7978465e6e0fd5111a2000822f6` |
| Conscrypt R8 DEX | `cd4d7eb9b05e1a84d064f7f9e4b33d68a8f92a9f58a6a62e4041972964ef6d27` |
| Conscrypt static APK | `9a0224400b91b030213e0f6ba63d899a4ad6333f930414c768ce88bbef423e91` |
| Conscrypt mapping | `67cd69e706afee50a1cc4e13bd9fd26056932c569d0ea7706dfe6fe8215a2e3c` |
R8 publisher and independent official Bazel checksum pin are linked from the
fixture README. Conscrypt was downloaded from
[Maven Central](https://repo.maven.apache.org/maven2/org/conscrypt/conscrypt-android/2.8.0/conscrypt-android-2.8.0.aar)
and verified against its publisher SHA-256 sidecar. No generated binary or
mapping file is committed. JAR/ZIP timestamps can change a source rebuild's
artifact hash; the table identifies these measured bytes.
## Owning changes and safe bounds
The provider keeps one successful parsed index. Each call still performs a
stable regular-file read and SHA-256 check. Only identical digest/length with
fixed parser options reuses parsed facts; current path provenance comes from
the new read. Changed content, including same-size/same-mtime rewrites, is
reparsed; missing or cancelled input never falls back to stale facts. Idle
retirement and close release the index. During replacement, old and new parsed
indices can briefly coexist. APK/query results are not cached by this mechanism.
The domain parser scans lines without a whole-file split, accounts retained
strings/containers before admitting records, and admits parameter/metadata
expansion before split/JSON parsing. The fixed 64 MiB input, estimated 128 MiB
index and 8 MiB per-residual-class claim allowances are resource constraints,
not format claims or measured V8 heap caps. Replaced metadata, deduplicated
range splits and overwritten inline frames are not cumulatively billed.
Ambiguity scans count all alternatives but retain only eight stable candidates;
conflicting residual metadata stays ambiguous. A separate combined 8 MiB claim
budget covers actual queries including all inner classes; omitted claims become
explicit `unknown`, preserving every raw identity and source SHA. It is not a
whole-report/wire-byte cap.
Existing shared retained-Evidence projection now supports class/method summaries
through both CLI and MCP. They preserve complete normalized facts and selected
bodies, omitting duplicate raw RPC metadata from the inline response. Complete
parent Evidence remains exportable. Default/complete delivery is compatible;
summary is explicit, and CLI requires a non-overwriting portable output bundle.
Whole summary views admit 1 MiB including context: an admitted one-MiB body may
therefore require complete detail. Refusal preserves the MCP parent ID and is
nonretryable for the same selection; no facts are silently truncated.
## Completed-task measurements
Linux x86-64, Node 24.22.1/npm 11.16.0, Temurin JDK 21.0.12.1, actual JADX
0.7.1, one analysis worker, 1 GiB JVM heap and two active processors. Harness,
server and children were restricted to CPUs 0/1; Node heap was 3 GiB. Expensive
checks ran serially under an 8 GiB sampled family-RSS stop and a 4 GiB host-free
memory guard. No guard stopped a completed task and no OOM occurred.
The [public stdio MCP harness](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-mapping.mjs) uses named live tool
contracts and Ajv 2020 validation of SDK-converted advertised outputs. All
correctness comparisons, hash reads, discovery and cleanup are included in the
whole-harness cost. First-provider calls use fresh processes/JVMs; OS file caches
were warm, and the host was shared. Conscrypt order was before/after/after/before.
Each small task makes 19 inspections and one proof export; each library task
makes six inspections and one proof export. Every exported parent is checked
against the exact inline normalized facts, source parameters and APK SHA, and
must retain raw calls. The complete/summary lanes use the same requested facts,
provider, options, source bytes, discovery families (`android,session`) and proof.
Peak memory is summed process-family RSS sampled every 100 ms, including client,
server, JVM and children. Shared pages can be counted twice and short peaks can
be missed; this is not PSS, kernel peak accounting or an enforced production
heap measurement. CPU is completed-family user+system time.
| Cohort | First task s | Warm task s | Whole harness s | CPU s | Peak family MiB |
| ----------------- | -----------: | -----------: | --------------: | -----: | --------------: |
| small-before | 7.342 | 1.611 | 12.060 | 16.680 | 781.7 |
| small-after | 7.960 | 1.842 | 12.945 | 17.413 | 791.5 |
| conscrypt-before | 8.057 | 1.015, 0.893 | 13.264 | 17.774 | 920.0 |
| conscrypt-after | 6.975 | 0.833, 0.817 | 11.743 | 17.242 | 855.0 |
| conscrypt-after2 | 6.987 | 0.725, 0.748 | 11.561 | 16.553 | 872.6 |
| conscrypt-before2 | 7.046 | 0.917, 0.886 | 11.855 | 17.093 | 949.7 |
Warm Conscrypt task medians were 0.905 s before and 0.783 s after (about 14%
lower across four warm samples per implementation). Cold/whole-task and memory
samples vary; no universal startup or memory improvement is established. The
small fixture pays extra summary/retention overhead: 12.060 to 12.945 seconds
for the whole harness, with CPU 16.680 to 17.413 seconds. Explicit complete
delivery remains available.
An earlier separate **cache-only** cohort used complete responses without proof
export, on the same base. Conscrypt warm six-call medians fell from about
0.981 to 0.765 s; before ranges were 0.950–0.996 s and after 0.743–0.850 s.
Whole harness samples were 12.861–13.876 to 11.641–11.647 s, CPU
16.916–18.297 to 16.454–16.893 s, and peak family 910.1–938.1 to
853.1–876.8 MiB. Small before/after whole costs were 18.493/16.110 s,
17.495/16.589 CPU s and 763.4/776.3 MiB. These samples motivated reuse but
are not substituted for the completed proof-export task above.
## Whole-task token estimates
[android-mapping-costs.py](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-mapping-costs.py) executed with tiktoken
0.14.0/o200k_base. Text projection counts a fixed task question, all discovered
name/description/input schemas once per investigation, every RPC argument and
all text replies, including proof export. SDK transport additionally counts
output schemas and structured-content duplicates. Exported file contents are
checked offline rather than injected into model context; export IO/CPU/memory
is included above. Prompt replay, host pruning/rendering, hidden prompts,
protocol handshake/framing and actual model billing are unmeasured. Estimates
are not billed usage. Each table row is one warm completed investigation.
| Cohort | Estimated text tokens | Estimated SDK transport tokens |
| ----------------- | --------------------: | -----------------------------: |
| small-before | 43,526 | 167,094 |
| small-after | 41,755 | 159,507 |
| conscrypt-before | 180,213 | 510,351 |
| conscrypt-after | 69,836 | 217,646 |
| conscrypt-after2 | 69,908 | 217,789 |
| conscrypt-before2 | 177,902 | 505,723 |
Conscrypt text estimates fall about 61%, from approximately 178–180k to 70k,
while preserving all requested facts and complete proof. Small tasks improve
only about 4%; summary context has its own cost. The cache-only cohort kept
similar token counts (~172–174k library text tokens without proof export),
revealing that parse caching alone did not address repeated raw class metadata.
Full inventories still cost more context than a direct known-method summary.
Normalized investigation digests match exactly across all before/after lanes:
- Source fixture: `ec76d1e086c8561370fc1ed5dfef67bb88b185e8fb57c7e30afe9f1f153634eb`.
- Conscrypt: `7ebf70795dfc23bf2b0d0e813837325fa221f15560ca2384fdb06f26d69d78a5`.
## Post-review ownership and final workflow confirmation
Greptile found three concrete boundaries subsequently fixed in
`160573cb36f478a088b8a5284fa571cd97dc9a86`: translated prototypes could expand
before comparison; a warm request without a mapping could bypass a pending
mapping descriptor's cleanup; and timeout classification inspected the outer
cleanup wrapper. Type matching now compares incrementally within the observed
descriptor, retained file cleanup gates shared request admission and idle
retirement independently of engine cleanup, and the first interruption is
classified before cleanup composition. All 126 related cases pass serially:
55 parser, five file-reader, four first-cause/close outcomes, three cleanup,
52 session/idle/cleanup and seven MCP mapping cases.
A separate bounded diagnostic found and corrected source-string retention.
On Node 24.22.1/V8 13.6.233.17-node.58, four small selected class/method results
from four distinct mappings containing 10 MiB of ignored comments each retained
41,944,028 extra external string bytes through the actual immutable Evidence
ledger. Clearing it released that backing; detached-string controls retained
zero extra external bytes. The parser now owns admitted UTF-16 strings once,
sharing section names and copying only retained outermost/deduplicated facts.
It preserves lone surrogate code units and never clones the entire index or
copies names per lookup. The identical post-fix ledger experiment retained
zero extra external bytes; its RSS delta was 151,552 bytes, versus 62,898,176
before. These are engine-specific observations, not portable heap guarantees.
The unchanged Evidence/ledger modules and changed parser are fingerprinted by
[the diagnostic](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-mapping-retention.mjs).
Before/after whole diagnostic costs were 2.995/2.582 seconds, CPU 4.976/4.382
seconds and peak sampled family RSS 145,903,616/113,287,168 bytes, with explicit
256 MiB old-space selection and a 1 GiB family guard.
Final equivalent Conscrypt investigations compare the integrated unoptimized
control `5ecc6ccc662678c00a45bd171f40798634774155` with fixed `160573cb3`, both
on main `e99b6c2c3bcc5f4f050b51b432145144d0d47dc0`, using the same artifacts,
facts, seven calls including full proof export, and before/after/after/before
order. Every normal-fact digest remains
`7ebf70795dfc23bf2b0d0e813837325fa221f15560ca2384fdb06f26d69d78a5`.
The final source-fixture summary/proof/negative qualification also passes all
43 calls from the maintained harness path.
| Final cohort | First s | Warm s | Whole s | CPU s | Peak family MiB | Warm text estimate |
| ----------------- | ------: | -----------: | ------: | -----: | --------------: | -----------------: |
| conscrypt-before | 7.187 | 1.027, 1.006 | 12.370 | 17.307 | 917.7 | 177,855 |
| conscrypt-after | 7.376 | 0.813, 0.866 | 12.402 | 17.522 | 868.7 | 69,819 |
| conscrypt-after2 | 6.878 | 0.707, 0.726 | 11.439 | 16.383 | 883.4 | 69,902 |
| conscrypt-before2 | 7.111 | 0.989, 1.186 | 12.566 | 17.479 | 930.1 | 177,719 |
The final warm ranges were 0.989–1.186 seconds before and 0.707–0.866 after;
text estimates remain about 61% lower, ~177.7–177.9k to ~69.8–69.9k. Whole-task
and cold samples overlap and are retained alongside the original cohort above;
these measurements do not establish a general startup or memory advantage.
### Follow-up complexity review
Two additional review findings exposed repeated linear work despite bounded
retention. The parsed owner now records one agreed residual alias (or explicit
conflict) per original type, while preserving every class record and its count.
Method identities have a direct residual-signature index; missing signatures
still merge into the first record, including the initial unsigned-to-signed
transition. Superseded alias arrays and candidate searches were removed.
The representation allowance includes the new index containers/entries.
The reproducible
[complexity diagnostic](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-mapping-complexity.mjs)
compares the built parser from `160573cb3` with the follow-up parser SHA-256
`9c4c24f7a10a17b8ec08e74b2588b9ba87bc30f3e3d16690c3e07c03fbce233b`.
The copied before module only rewrites its three unchanged dependency imports
to absolute URLs. Each scenario verifies late matches, counts and answer digest;
answers match across revisions. Node 24.22.1, two CPUs, 512 MiB old space,
8 GiB sampled family guard, one before/after trial per size:
| Scenario | Revision | Parse ms | Lookup ms | Whole s | CPU s | Peak family MiB |
| ----------------------------------------------------- | -------- | -------: | --------: | ------: | ----: | --------------: |
| 20k identical aliases, 1k candidates × 255 parameters | before | 212.60 | 5169.61 | 5.635 | 5.690 | 149.3 |
| same | after | 207.31 | 22.84 | 0.449 | 0.559 | 117.1 |
| 40k distinct signatures of one original method | before | 8032.93 | 5.70 | 8.300 | 8.371 | 117.5 |
| same | after | 321.58 | 5.43 | 0.562 | 0.532 | 117.8 |
The smaller alias cohort (1k aliases, otherwise identical) looked up in
289.18/24.64 ms before/after. The smaller signature cohort (10k) parsed in
785.65/89.68 ms. These synthetic cases establish removal of the reported
growth, not universal latency or memory gains; they involve no agent tokens.
All 60 parser, two report and seven public MCP mapping cases pass serially,
along with type/lint/build (79.084 s, CPU 110.273 s, peak family 4393.9 MiB).
The final indexed implementation also passes the real source fixture's
43 summary/proof/negative calls (13.838 s, CPU 18.212 s, peak 797.5 MiB),
and the Conscrypt 21-call investigation including proof export. The latter's
normal-fact digest remains the value above: first task 7.670 s, warm tasks
0.860/0.981 s, whole 12.706 s, CPU 17.870 s, peak 868.8 MiB. Warm whole-task
text estimates are 69,940, SDK-inclusive 217,975; tokenizer/scope remain the
same. Complete class/method facts and provenance remain accessible.
For CI head `160573cb3`, all failed/cancelled shard logs and source-test logs
were audited. Its executed merge `ee2398458bd8c56666ecaf120c1c3c21f350eea4`
uses main parent `0676ad254c206871e0f4b120ba300cbbf9383b62`. The 90 observed
failed test identities were selected on a pristine worktree at that exact
parent: 88 reproduced in the primary run (128.320 s, CPU 147.775 s, peak
2593.3 MiB), then the two newline-bearing ZCode titles reproduced in an exact
follow-up (2.388 s, CPU 2.788 s, peak 416.9 MiB). No feature code was present
in either control. These are baseline failures; cancelled cases without a
completed result remain unverified by CI, with local verification above.
## Executed verification and limits
- Parser/reader/protocol/provider mapping regressions cover raw names, exact
descriptors/types, overloads, inline outermost frames, malformed/partial
records, stable candidate samples, preallocation refusal, repeated metadata,
cache freshness/path aliases/deletion, cancellation and descriptor ownership.
- Real R8 source and public-library stdio MCP tasks verify all selected symbols
and advertised schemas. Actual CLI complete and summary inspections verify
raw/mapped identity, exact normalized parity and complete portable parents.
- Actual Java metadata regression and downstream CLI/MCP body/direct-invoke
parity pass, including reserved raw names, frozen aliases, default packages,
encoded constants, native absence and classes.dex/classes2.dex identity.
- Summary regressions preserve partial/unknown facts and optional historical
omissions, validate SDK output contracts, read retained data after APK/mapping
deletion, reject missing portable CLI output and exercise the one-MiB boundary.
- Type/lint/build, named schema/profile checks and relevant source/boundary checks
are executed separately. Final PR validation records the current command set.
The final combined 12-file Android run passed 219 of 220 cases, including all
52 session/idle/cleanup integration cases. One existing package-summary SDK
case hit its 30-second limit under combined load. The unchanged four-case file
then passed with one worker: its SDK case took 11.479 seconds; whole isolated
harness 31.867 seconds, CPU 35.064 seconds and peak family 1,551,081,472 bytes.
No assertion or timeout was relaxed. The combined timeout remains a recorded
resource/concurrency limitation rather than being reported as an all-green run.
Earlier failed attempts are retained rather than counted as passes: a resolver
refactor used `only.mapped` after `only` already held the mapped fact; its
regressions failed and were fixed. Tagged-error migration exposed one stale
string-only test. New SDK schema test typing and a redundant JSON preadmission
were corrected at their owning boundaries. A first summary MCP case timed out
while enumerating unrelated full catalog contracts; selecting the actual
Android/application/session families preserves every assertion and passes
without raising timeouts. An initial library harness requested absent
`SSL_shutdown` and skipped it; the harness now rejects absent requests, and
reported final cohorts explicitly verify all five requested methods.
The mapping cancellation regression distinguishes caller-first from deadline-
first finalization; the original implementation relabelled caller cancellation
as timeout (one failing caller-first case and one passing deadline-first case
on the unchanged optimization control). An initial test's public deadline code was corrected to the actual
`provider_timeout` contract. Fixed code preserves the first cause and cleanup.
Only Linux x86-64/JDK21/JADX0.7.1 and R8 8.0.40 mapping-format 2.1 were qualified.
Residual-signature conflicts use derived negative mappings, not a real mapping-
format 2.2 producer. No Windows/macOS, emulator/device, app execution, full APK
rebuild/linkage, source/line recovery, APK signature verification or model-usage
trial is claimed. Mapping/build association remains explicitly caller-selected.
## Reproduction
Generate source/DEX/oracle using the fixture README with selected existing tools.
Keep separate before/after worktrees, pin their revisions above, build both and
use identical target/mapping/oracle bytes. Then run the same harness for each:
```sh
node tests/support/analysis-efficiency/android-mapping.mjs \
--repository "$SELECTED_REPOSITORY" --target "$STATIC_APK" \
--mapping "$MAPPING_FILE" --output "$FRESH_RESULTS" \
--java-home "$SELECTED_JDK" --jar "$JADX_MCP_JAR" \
--oracle "$RAW_DEX_ORACLE" --expectations "$ORIGINAL_EXPECTATIONS" \
--proof
```
Add `--detail summary` only on the after implementation. The small fixture
selects all classes/bodies. For Conscrypt add `--class-name U0 --repeats 3` and
`--methods EVP_PKEY_new_RSA,EVP_DigestSignInit,SSL_new,SSL_free,SSL_get_error`;
`U0` is the measured artifact's raw class identity, not a universal residual name.
Use the original NativeCrypto D8 descriptor inventory for that expectation file.
Run `--variants` separately on the source fixture to exercise derived malformed,
partial, duplicate and conflicting mappings; those extra qualification calls
are not part of the timing/token table's repeated task.
```sh
python tests/support/analysis-efficiency/android-mapping-costs.py \
--output "$TOKEN_RESULTS" "$BEFORE_RESULTS" "$AFTER_RESULTS"
```
Capture elapsed/CPU/family RSS with the bounded host runner or an equivalent
process-family sampler; alternate trials and keep each label's raw receipts and
failed attempts. Complete Evidence stays in the exported per-pass bundles.
File diff suppressed because it is too large. Load diff
@@ -4,10 +4,14 @@ Issue [#4772](https://github.com/morluto/rea/issues/4772). Qualified runtime
`b49ee40bf330b7bf1c5180b14dcc104c1c613a8e`, tree `0e5f8fe2feae1796c340c07c26e0e28e5d4328d8`,
integrates main `c8ed32853881b08aa83459d4da55a28a4e09b0c8`. Recorded at
`2026-10-22T08:30:58.269Z`. Subsequent verification-only
commits change these reports. The [JSON record](javascript-route-inventory-2026-10-21.json)
commits updated these reports. The [archived JSON record](https://github.com/morluto/rea/blob/7d0efef62d56d0a10134bea62d709302daa1565f/docs/verification/javascript-route-inventory-2026-10-21.json)
pins original inputs, exact commands, profiles, failed controls and historical revisions.
Current-head GitHub CI and automated review must be checked separately.
The full JSON archive is 2,681,223 bytes, with SHA-256
`9d7ea2582a6f33ff0741578aec9f01b9327b3256ce50c582437164868e544d51`. It remains available in Git
at the linked immutable revision; this page retains the historical summary.
The latest main contracts, selected environment and ASAR admission remain shared
by CLI and MCP. The staged-code hook runs lint fixes before formatting, resolving the
producing order behind the earlier CI format failure.
-680
View File
@@ -1,680 +0,0 @@
# Analysis efficiency measurements
The [Android component checkpoint](android-components.md) records exact selected
declaration views, real JADX verification and completed-task token estimates,
including increased schema cost and no demonstrated speed or memory gain.
The [2026-10-21 coverage reconciliation](coverage-review-20261021.md) maps all
46 source directories to concrete workflow evidence, including original feature
PRs, historical no-benefit results and remaining verification boundaries.
The [GGUF validation checkpoint](model-utf8-validation.md) removes decoding of
omitted strings while retaining full UTF-8 validation. It records real-model
phase gains, unchanged task token costs and the higher main-task memory peak.
The [WAT framing checkpoint](wasm-string-framing.md) measures structural scanning
on real SQLite and VS Code components, with unchanged Evidence and token costs,
small whole-task timing gains and no memory improvement.
The [native inventory view checkpoint](native-inventory-views.md) compares
complete delivery with summaries and selected rows on real `true` and jq
inventories, including oversized-response recovery, portable old Evidence,
CLI/MCP parity, tokenizer estimates and independent-engine variation.
The [retained-capture validation checkpoint](retained-capture-validation.md)
reconciles the earlier audit queue and measures immutable historical-capture
validation reuse on public compact and large qlogs, including first-view costs,
small-input limits and unchanged Evidence delivery.
The [Ghidra Evidence projection checkpoint](ghidra-evidence-projection.md)
measures removal of redundant raw copies on real `true` and jq workflows,
with complete provenance, unchanged response budgets, native CLI/MCP checks
and explicit cold-analysis variance.
For Hopper literal string queries, see the [decoded-record checkpoint](hopper-string-records.md).
It measures record reuse through real Linux CLI/MCP workflows, preserving
native recognition gaps, guard-stopped attempts and negative whole-task costs.
The [Linux embedding-runtime follow-up](hopper-linux-python-runtime.md) checks
actual clean setup, dependency repair and normal-user analysis, retaining
failed attempts and distinguishing prerequisite correctness from speedups.
For JEB on Linux, see the [CE native audit](jeb-linux-native.md). It qualifies
the direct headless launch route and measures selected function reads through
real CLI and production stdio MCP, preserving timeout attempts and the limits
of wrapper-cache and token claims.
For Apktool locale queries, see the
[shared-decode and table-only checkpoint](apktool-string-tables.md). It separates
the existing decode reuse from duplicate string delivery, with three real APKs,
CLI/MCP parity, whole-task estimates and negative timing results.
For immutable imported native/JavaScript parents, see the
[validation-reuse checkpoint](immutable-parent-validation.md). It measures the
owning format change with the existing retained-query driver, including first
view costs, small cases, unchanged token estimates, and noisy whole-task results.
For authentic old Evidence and mixed investigations, see the
[retained-query checkpoint](retained-investigation-workflows.md).
`retained-investigation.mjs` compares existing complete delivery with selected
pages, including capacity-refusal recovery and original-record round trips.
It records negative model results and separates imported-format validation work
from the query contract.
For genuine Fiddler captures, see the [SAZ selection checkpoint](saz-selection-workflows.md).
`measure.mjs --saz-selection baseline|focused|retained` compares default complete
delivery (including retained-reference recovery), caller-selected ordinals, and
summary/preview/complete/repeated pages. Each failed RPC remains in task costs.
The body question uses stored bytes, not decoded HTTP application payloads.
Tracks the experiments for [#2016](https://github.com/morluto/rea/issues/2016).
The production CLI/MCP contracts and defaults remain the compatibility baseline.
Downloaded artifacts, runtime copies, exported Evidence, transcripts and detailed
JSON measurements belong outside the checkout. This document keeps the measured
findings, verification scope and revision references; the scripts and `corpus.json`
keep the reproduction inputs. Generated `results/` is ignored.
Follow-up: [latest-main capability audit](latest-main-audit.md) records the
Flutter, SQLite, TShark and sigrok priorities and the first shipped Dart AOT
baseline, preserving existing controls before introducing another selector.
Follow-up: [Dart AOT summary and retained string pools](flutter-aot-summary.md)
records real LocalSend delivery pairs and old Evidence continuation, preserving
ABI selection, snapshot identity, original lower bounds and partial coverage.
Follow-up: [SQLite existing-control measurements](sqlite-existing-controls.md)
tests QGIS's shipped GeoPackage through the existing column selection, preserving
full selected-table schema and typed cells before considering another API.
Follow-up: [small real PCAP recordings](pcap-small-captures.md) compares existing
complete delivery with summary/pages. Preserve the negative SDK/model input
results: compact captures do not justify another production mechanism.
Follow-up: [native PE/DOS entry views and real model verification](native-entry-views.md)
records the post-merge native continuation, including a small-function case where
four selected replies cost more bytes than the complete dossier.
Follow-up: [EVM descriptor and real-template controls](evm-descriptor.md) extends
the offline corpus with digest-pinned Uniswap runtime templates and an independent
published ABI selector oracle, preserving unknown deployment/immutable metadata.
Follow-up: [EVM interface summary delivery](evm-interface-summary.md) records
the opt-in retained view, old-client/Evidence migration checks and repeated SDK
and real-model pairs, including mixed token/time results and strict-answer limits.
Follow-up: [task-scoped preset lifecycle verification](preset-lifecycle.md) checks
real Ghidra profile/snapshot boundaries, startup cancellation and retained cleanup
retry while preserving defaults and distinguishing verification from performance.
Follow-up: [EVM descriptor lifecycle acceptance](evm-lifecycle.md) verifies actual
worker replies, late SDK cancellation, permission-induced cleanup failures,
healthy WASM peers and old CLI parity with explicit runtime and platform scope.
Follow-up: [managed artifact delivery and retained views](managed-artifact-summary.md)
records two shipped assembly checks, previous-runtime portable Evidence, and four
actual Codex trials. Selected replies shrink substantially; observed model input
tokens increase and latency is mixed, so model-token savings are not claimed.
Follow-up: [model tool context and per-response usage](model-context.md) separates
raw MCP replies from observed client execution/discovery and usage. Optional
trial capture retains tool/usage items while excluding account and ambient state;
existing ephemeral trials keep their default behavior.
Follow-up: [Android package delivery and retained manifest](android-package-summary.md)
records real NewPipe SDK pairs, unchanged old requests/Evidence, exact retained
text, and actual Codex outcomes including client errors and a failed discovery.
Reply bytes fall; model-token/time gains are not established.
Follow-up: [resource counters at payload completion](resource-counters.md) adds
optional owned counter observations while preserving unknown whole-scope final
counters, default receipt shape and the unchanged CPU/memory safeguards.
The 12 detailed checkpoint receipts were archived byte-for-byte before removal
from the final PR tree. The archive `review-receipts-00d51e0fc-20261012` is in the
private experiment lab outside the checkout; its `archive-index.json` records
source revision, filenames, sizes and SHA-256.
Raw producer replies, model transcripts and exported Evidence remain in the same
private lab. These local files are not required to build or use REA. Historical
checkpoint commits retain their original receipts; no history was rewritten.
## Optimization priorities and gradual migration
The [larger/non-HTTP packet checkpoint](packet-field-workflows.md) records
same-question SDK and actual Codex comparisons using existing input limits,
field/filter selection, discovery and output windows. Lower delivered JSON did
not establish stable total-token or speed gains; failed plans and negative
controls remain visible.
The [sigrok selection checkpoint](signal-selection-workflows.md) records two
real hardware recordings, independently verified bytes, repeated decoder/pages,
and an eight-task Codex cohort with mixed token/time results. Existing annotation
controls reduce replies; these measurements introduce no production cache,
default or API changes.
Agent token consumption and end-to-end task time are the primary objectives.
Memory and CPU consumption are also measured and controlled. Correctness,
Evidence provenance, caller choices and existing-user compatibility are
acceptance conditions for every optimization.
- Tokens: minimize unnecessary model-visible data while returning the requested
facts and useful provenance inline. Count discovery, repeated calls, errors,
retries and follow-up turns; report total/cached/uncached input and output
separately. Compact JSON bytes are a delivery measurement, not a token count.
- Time: measure completion of the same analyst question, including startup,
provider work, transport, client processing and model corrections. Separate
measured stages and cold/warm state; use repeated comparable trials before
generalizing an observed latency improvement.
- Memory: record aggregate process-tree peaks and limit/cleanup events, account
for retained Evidence and representation expansion, and keep host safeguards.
- CPU: record aggregate CPU time for the completed task. A one-CPU quota is an
experiment control; it does not itself establish reduced CPU consumption.
Migrate domain by domain with focused real-artifact checks and relevant shared
contract checks. The native follow-up verifies selected PE and DOS MZ entry facts;
DOS COM and complete native application behavior remain unverified. The domain
matrix is a growing roadmap. A passed lane does not imply all
advertised platforms/providers have passed. Newly discovered REA bugs remain
in the separate local handoff document for later team repair.
Old clients using existing CLI/MCP requests against the candidate retain the
complete default path. New summary/view options require explicit selection.
Actual old Evidence import/view/export checks preserve original authenticated
records and identities. These checks establish the recorded baseline/workflow
scope, not every historical release or third-party client. A new client targeting
an older server must discover the actual schema and use a supported equivalent
where available; this study does not implement a universal automatic fallback.
The catalog intentionally gains more precise/additive schemas, so its bytes and
digest are not frozen: measured runtime `259e99dd9` has about 2.6% larger complete
catalog JSON than the frozen baseline. The separate post-integration receipt
records the newer runtime, including upstream changes. Optional views can reduce
model-visible result data; old complete requests do not silently switch delivery
or acquire that reduction automatically.
## Reproduce a measurement
Use the repository's pinned Node/npm and locked dependencies. Build each selected
revision, then copy its `dist/`, `bridge/`, `scripts/`, `skills/`, `package.json`
and `package-lock.json` to a separate runtime directory before editing production
sources. That copy needs the matching locked dependencies. The initial frozen
baseline is `42b0ed126df7d5b908ac369715378d5e519051b7`; this study does not
silently substitute a newer `main` or the previous Code Mode experiment.
The public artifacts and acquisition digests are in `corpus.json`. Supply their
actual local paths and an optional JSON object of explicit provider environment
settings. Do not add credentials to that file. Provider settings are inherited
by the child, but the report persists only the selected settings' digest. Runs
reject undeclared inherited REA/Hopper and Ghidra/Java configuration keys before
starting a measured server or creating a model profile; include those settings
explicitly in `--providers`. Both drivers use `provider-environment.json` and
record its digest. The system/runtime environment still needs comparable
controlled conditions; its contents are not serialized. Both drivers also
fingerprint the effective provider `PATH` without recording its value, and
the SDK comparator rejects different or unrecorded search paths. This detects different
search configuration, not replacement of executable bytes at the same path.
Historical receipts
predate this guard, so their selected-settings digest alone does not certify
that no undeclared provider setting was inherited.
```sh
export NODE_OPTIONS=--max-old-space-size=512
export UV_THREADPOOL_SIZE=1
export GOMAXPROCS=1
export GOMEMLIMIT=2GiB
python3 experiments/analysis-efficiency/resource-run.py -- \
node experiments/analysis-efficiency/measure.mjs \
--runtime-root /absolute/frozen-runtime \
--revision 42b0ed126df7d5b908ac369715378d5e519051b7 \
--artifact obsidian.asar --path /absolute/obsidian-1.14.4.asar \
--providers /absolute/selected-providers.json \
--output /absolute/new-measurement-directory
```
The wrapper requires Linux cgroup v2 and a working user systemd manager. It owns
a unique transient scope, restricts the whole process tree to one CPU, caps
memory at 4 GiB with no swap and 192 tasks, and samples memory/tasks/CPU every
100 ms. It stops its own scope above 3 GiB or below 4 GiB host available memory;
startup requires 8 GiB available. Cleanup failures retain the owned scope's
identity. Run heavy builds, providers and model trials serially. Sampled peaks
can miss short spikes; cgroup limit events are separately reported.
An actual Codex + Chromium trial exceeded the initial 192-task allowance and
was stopped automatically below 1 GiB sampled memory, with no OOM. The isolated
browser model retry uses `resource-run.py --task-ceiling 256 -- COMMAND...` to
account for both process trees; memory, swap and CPU controls are unchanged.
Keep the original aborted run and cleanup receipt. This is experiment capacity,
not a REA defect or a successful browser analysis.
## What the numbers mean
`summary.json` records the selected artifact's before/after SHA-256, runtime
file-table digest, dispatcher and lockfile digests, runner source digest,
Node/SDK versions, catalog sizes, caller arguments and producer Evidence IDs.
`runtime-files.json` identifies the exact compiled files measured. Actual
provider settings and runtime files must stay fixed during a comparison.
Every tool call measures three intervals at the public SDK Transport boundary:
- Client work before `send`, including first-use schema preparation.
- Transport/server roundtrip, including framing, I/O, server work and client
JSON parsing. This interval is **not** a Ghidra-only or provider-only timer.
- Client work after the parsed response reaches `onmessage`.
The SDK call's total excludes the runner's subsequent encoding and disk writes.
Result byte counts measure compact JSON of the parsed client result, not raw
wire bytes or tokens. Progress notifications retain their actual arrival times;
they do not imply more precise internal engine timings than the producer emits.
Scope CPU time and aggregate process-tree memory come from the wrapper's final
JSON line. Fresh processes have cold application caches; host page caches are
not globally flushed.
`--delivery summary` is an explicit candidate workflow, initially for artifact
inventory. It must be supported by the selected server's advertised schema.
The default `complete` runs the existing calls unchanged. Compare the same
analyst facts and complete source Evidence before crediting a smaller response.
Projection can reduce delivery bytes without reducing upstream analysis time.
For WASM, the runner validates a summary request against the actual advertised
2020-12 input schema, including SDK-generated references. Supporting servers
return the first-call summary; older servers retain their complete path and use
summary/export views after an observed transport constraint. Original errors
remain in the receipts, and the complete source Evidence ID is a compared fact.
`--javascript-question selected` analyzes only the exact Obsidian sources
`package.json` and `main.js`, keeps inventory identity, and reads each observed
`main.js` module by its exact node ID. Package metadata is not a module selector.
Compare paired runs with the same runner source and provider settings:
```sh
node experiments/analysis-efficiency/compare.mjs \
--left /absolute/complete-run --right /absolute/summary-run \
--output /absolute/new-comparison.json
```
This checks all selected analyst facts, including inventory source IDs, exact
path records, manifest, counts, completeness, contradictions, coverage,
unexplored branches and limitations. It excludes only delivery observation IDs
and the named recovery-path description. Different settings, changed facts or
failed workflows make the comparator fail. Binwalk's per-invocation region UUIDs
are separately preserved with both original values and JSON paths; only those
opaque lookup values are excluded from physical-region fact equality. Original
producer replies are never rewritten. Selected JavaScript comparisons likewise
preserve both values of the two named parent-heap-dependent byte budgets before
excluding their values from fact equality; fixed limits and coverage rows remain
checked. Counts include extra view calls and
observed tool errors. Use repeated runs with alternating order before claiming
a distribution or a speedup; one pair is only one observation.
For a completed WASM result exceeding the MCP response budget, the runner keeps
the original error, exports retained Evidence through the public tool, and
authenticates the exact reported record using the selected runtime's parser.
The report names this recovery; it does not relabel the oversized inline reply
as successful delivery.
## Initial observations and exclusions
The frozen baseline has 2,326,609 bytes of complete catalog JSON, of which
494,207 bytes describe names, descriptions and input schemas. One Obsidian
trial spent about 21.9 s before the first request was sent, then about 0.77 s
in that request's transport/server roundtrip. Other cold runs differed
substantially. These single observations identify a measurement boundary;
they do not establish a speedup or a distribution.
The initial matrix exercised shipped Obsidian, NewPipe, ShareX, OpenWrt, SQLite
WASM and jq artifacts. SQLite produced a complete result larger than the
transport frame and requires an explicit recovery run. A first jq attempt hit
the SDK's default 60 s timeout because the experimental runner used a superseded
three-argument call signature. SDK 2.3.1 accepts request options as argument two;
the corrected jq run completed real Ghidra analysis in about 265 s. That earlier
driver timeout is excluded from REA regression claims. Provisional runs with
different runner source digests are not one paired statistical cohort.
No cross-session engine database cache, default analyzer changes, external
plugin loading, or broad plugin SDK is enabled by this experiment.
## Obsidian delivery checkpoint
The two recorded pairs compare frozen baseline
`42b0ed12` and candidate `f1106181` in complete → summary and summary → complete
order using the same runner source, bytes and provider settings. Both preserve
every selected fact and the atomic inventory Evidence ID.
| Measurement | Complete | Summary + exact path view |
| --------------------------------------------------- | --------------: | ------------------------: |
| Tool-result compact JSON, including lifecycle calls | 937,404 bytes | 16,322 bytes |
| Tool calls per question | 3 | 4 |
| Complete catalog compact JSON | 2,326,609 bytes | 2,370,505 bytes |
| Workflow elapsed, first pair | 11.04 s | 10.23 s |
| Workflow elapsed, second pair | 10.57 s | 10.18 s |
Reply JSON falls by about 98.3%; catalog JSON grows by about 1.9%. The two elapsed
observations do not establish a statistical speedup. Inventory coverage and
upstream byte acquisition are unchanged. The candidate remains opt-in; existing
requests continue to deliver the complete inspection. Each run verifies the
selected artifact's before/after digest and reports no observed OOM or task
limit events. Source typing and projection improve selected delivery without
introducing an engine database cache or an external plugin protocol.
## Task-scoped analyzer experiment
The recorded jq pair explicitly disables
`Decompiler Switch Analysis` through the existing `REA_GHIDRA_DISABLED_ANALYZERS`
setting and compares it with the unchanged Ghidra default. Both use the same
frozen runtime and real jq bytes, one CPU, a fresh ephemeral database and the
same entry/main question. Completed main dossiers exceed the MCP frame in both
runs; existing retained native views recover the selected facts, preserving the
observed transport errors in the reports.
One disabled/default pair takes 174.77/276.23 seconds. Entry and main pseudocode,
main strings and its block count agree. However, callee `0x40daa0` has 604
recovered body bytes in eight ranges by default, and 164 bytes in four ranges
with the same 646-byte span after disabling switch analysis. Equal top-level
counts do not establish call-graph equivalence. **Keep the production default.**
Use the existing control only for a caller-selected task whose required facts
have been checked; no general fast preset is introduced. A separate real
coreutils/Ghidra probe rejects a nonexistent analyzer before bridge readiness,
preserves the named failure and closes the session. Source bytes remain unchanged.
`--schemas selected` is a separate SDK 2.3.1 client experiment. It passes the exact
live named `tools/list` definition to the SDK's public `callTool` option, retaining
output validation. It does not change REA or the default runner mode. Selected
definitions compile per call rather than preparing the entire cached catalog;
report this client effect separately from engine analysis or model behavior.
## Real Codex CLI trials
`agent-trial.py` accepts an explicit frozen runtime and revision, the same
question templates in `questions/`, and optional explicit provider settings.
It fingerprints the actual runtime, runs pinned local Codex CLI with a private
temporary profile, copies only the authentication file with mode 0600, disables
shell/web/multi-agent tools, and removes copied profile state afterward. Raw
events and answers remain outside Git. A zero exit code is not a factual grade.
Review the original tool calls, failed requests, result limitations and answer.
The optional `--enabled-tools FILE` supplies a list of exact names to the
temporary Codex MCP configuration's `enabled_tools`, as described in the
[official MCP configuration](https://learn.chatgpt.com/docs/extend/mcp?surface=cli).
It leaves the REA catalog and other clients intact. Initial Obsidian trials
showed parameter-correction attempts and different delivery choices even with
the same question; filtering alone has not established a cost or quality win.
Requested model aliases are recorded without inventing a resolved model identity.
The browser question uses `--authority browser` to permit only its explicitly
selected public scenario in a fresh, owned headless profile. Browser observations
and cleanup are graded separately from static artifact analysis.
## Real artifact compatibility checkpoint
Six frozen baseline/pilot pairs exercise
Obsidian, NewPipe, ShareX, OpenWrt, SQLite WASM and published historical WETH code
through public MCP tools. All selected facts agree under the stated comparison
rules. Obsidian complete delivery and atomic inventory Evidence identity remain
unchanged. ShareX native-boundary claims remain scoped to the selected DLL;
NewPipe retains a null DEX descriptor rather than inventing one.
SQLite's complete 15,382,128-code-unit WAT exceeds the MCP response budget in both
revisions. The control recovers complete Evidence through file export; the pilot
reads summary and four exact export forms inline, without running WABT again.
Inline byte totals exclude file exports, so they do not establish a size win.
OpenWrt comparison preserves both sets of upstream-generated region UUIDs before
comparing offsets, extents, signatures, coverage and limitations. WETH is a
published historical example; a live RPC request returned HTTP 403, so this is
not live-chain verification. Every pair records unchanged target bytes and zero
observed OOM/task-limit events. The Linux x64 results do not establish behavior
for IDA, Hopper, macOS, Windows or Harmony.
## Same-question model and selected-source follow-up
Two actual SQLite model trials preserve all
12 requested facts. The question, developer instruction, provider settings and
tool allowlist hashes match. A description-only follow-up supplies the exact
summary kind and its fields. Calls change from 42 to 3, workflow time from
292.64 to 63.79 seconds, and reported input tokens from 304,698 to 74,387. The
follow-up reads no unnecessary WAT pages. Both keep the original completed
analysis's transport error and authenticate the retained source/view identities.
This is one stochastic model pair, not a statistical or general causal claim.
The selected JavaScript pair preserves
all 24 observed `main.js` module identities and the exact two-source selection.
The strict comparison initially differs on parent budget values derived from
observed heap availability; both values and paths remain recorded separately.
Other coverage facts, per-family rows, fixed worker limits, omitted/unknown
counts, statistics and Electron observations agree. Coverage remains partial,
with 36 unselected relevant files and 384 semantic unknowns. These static facts
do not establish runtime behavior. The comparator does not rewrite Evidence.
Ten real CLI trial reviews cover native,
managed, Android, firmware, Electron/JavaScript, WASM, EVM and browser tasks.
The selected JavaScript model preserves core facts and partial coverage but
omits full budget/family provenance. The browser producer captures complete
DOM/accessibility/URL evidence and terminates its owned process; direct model
answers lose two accessibility metadata fields to client context clipping and
report them unknown. A requested under-development Codex code-mode configuration
still produces a direct MCP call in the actual trace, so no executor/composition
win is credited. Profile cleanup, original errors, actual calls, usage, facts and
scope limits remain independently reviewed. Enabling a configuration flag does
not prove that a capability was used.
Discovered REA defects are recorded separately for team repair at the user's
request. This follow-up performs no additional REA bug fixes.
## Nonempty managed boundaries and final verification
A second shipped managed library checks
11 ModuleRefs, 147 PInvoke declarations and 225 native implementations from
ShareX HelpersLib 19.0.2. All complete normalized artifact and native-boundary
fields match between baseline and pilot, including source Evidence identities.
This covers nonempty declarations beyond the main ShareX DLL's empty import
tables. It does not verify native exports, execute the assembly or resolve CLR
dependencies. The curated receipt keeps identities, coverage, all ModuleRefs,
the first three PInvoke records and digests of the complete boundary payloads.
Full producer tables and comparison inputs stay in the private lab outputs.
The earlier verification includes 42 real
WABT CLI/MCP cases and the complete real SQLite CLI JSON: the unchanged default
returns the same original source Evidence ID, normalized result and raw result
as the baseline exported MCP record. Source bytes are unchanged. The seven-file
contract/composition run passes 51/52 assertions and hits the existing 20-second
presentation-test deadline while sharing one CPU. That same unmodified assertion
passes alone in 10.835 seconds. The original combined failure and isolated pass
remain recorded; no test deadline or REA bug was patched. Build, type/lint/module
checks and generated-document checks pass. No full `check:pr` or unverified
provider/platform coverage is implied.
## Actual old Evidence migration
The migration experiment imports the original
baseline SQLite Evidence bundle into the pilot runtime, reads a typed summary
and four export forms, and exports the retained bundle again. The complete old
WASM record is deeply equal after import/export, with its original authenticated
Evidence ID. The original baseline Obsidian complete inspection also works as
inline input to the new inventory summary, retaining the old parent and atomic
inventory identities and manifest. Original input files remain byte-identical.
The runtime explicitly selects an absent WABT directory and requests no producer
operation. These views consume existing Evidence without another provider run;
this is portable-result reuse through existing import/export, not an engine
database cache. The five-call workflow completes in 5.458 seconds with a sampled
aggregate peak of 700,542,976 bytes and no observed limit or OOM events.
To repeat it with genuine old public outputs:
```sh
python3 experiments/analysis-efficiency/resource-run.py -- \
node experiments/analysis-efficiency/verify-migration.mjs \
--runtime-root /absolute/frozen-pilot-runtime \
--revision 259e99dd906290189217cf0fb1a19769bcf87101 \
--bundle /absolute/baseline-exported-evidence.json \
--inspection-reply /absolute/baseline-complete-inspection-reply.json \
--output /absolute/new-migration-directory
```
## PR integration verification
The separate post-integration verification pins runtime
`0eb42a1eb8ab17aed04240174cb77612e733a1d8`, integrated with main
`c22111eca2c4bc84a084f12711f405b3e7422a40`. Conflict resolution preserves main's
HAR API-surface views, portable capture admission, exact layout/JavaScript
summary carriers and existing CLI tests alongside inventory/WASM views.
Earlier real-model observations keep their original runtime fingerprints.
Build, the complete `check` static gate and generated documentation checks pass.
Twelve focused files pass 123 assertions, including the presentation test run
separately with its original deadline. Actual package creation, private-prefix
installation and 42 real WABT CLI/MCP cases pass. No package is published.
The complete generic package verifier and full local `check:pr` were not run.
Genuine old SQLite/Obsidian Evidence migration passes again with WABT unavailable.
A fresh public SDK cohort preserves the original Obsidian default result size
(937,404 compact JSON bytes), atomic inventory identity and every selected fact;
explicit summary plus exact paths still returns 16,322 bytes. Complete catalog
JSON is 2,359,082 bytes versus the original baseline's 2,326,609 (about 1.4%
larger). That difference includes intervening main changes and does not isolate
this PR's catalog contribution. One cohort is not a latency distribution.
Recorded integration scopes have no OOM/OOM-kill or hard-limit abort. The full
source-check scope reports three `memory.high` events near its 3 GiB soft
threshold. The subsequent cold pre-push type/lint scope reports 37 such events,
with 3,221,155,840 sampled peak bytes and no hard-limit/OOM event. Heavy lanes stay
serial with the existing CPU, memory and no-swap controls. No further REA bug repairs
are included; separate local findings remain for team follow-up.
## CI attribution
Review of the actual main and PR job logs separates two PR-related
description/test mismatches from pre-existing main failures. The complete-graph
wording check now accepts capitalization changes; the existing effects audit
records why returning WASM execution/network metadata is not foreign execution.
Those two files pass 11 focused assertions without production changes. Main's
own CI already fails the same live prompt-completion assertion and Darwin
packaged-completion checks; the affected implementation and fixture files are
byte-identical here. Those failures remain unchanged for team review at the
maintainer's request. CI failures are not all infrastructure faults, and the
failed CI gate is not presented as passed.
| Failure | Main `c22111eca` | Reviewed PR `8d6f7eb1c` |
| ---------------------------------------- | ---------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| Empty prompt completion after open | [main shard](https://github.com/morluto/rea/actions/runs/38224038323/job/114727940506) | [PR shard](https://github.com/morluto/rea/actions/runs/38228724531/job/114742955092) |
| Packaged prompt completion, Darwin arm64 | [main package](https://github.com/morluto/rea/actions/runs/38224038323/job/114728030045) | [PR package](https://github.com/morluto/rea/actions/runs/38228724531/job/114743092314) |
| Packaged prompt completion, Darwin x64 | [main package](https://github.com/morluto/rea/actions/runs/38224038323/job/114728029923) | [PR package](https://github.com/morluto/rea/actions/runs/38228724531/job/114743092079) |
`src/server/promptCompletion.ts`, `src/application/binary/BinarySession.ts`,
`tests/boundary/mcp/promptCompletion.test.ts` and
`scripts/verify-package-prompts.mjs` are byte-identical to that main revision.
Main commit `07b36c148946f46b99a4ce04e55726e97d0ee03a` introduces ready-provider-only
completion; the failing fixtures expect populated completion immediately after
opening. Whether those expectations or the readiness behavior should change is
left for team review. Commit `00d51e0fcd1d1bba60bf56281725e82d525a27ae`
synchronizes only the two PR-related description checks.
## Frozen runtime references
### Review follow-up
Two substantive bot findings were reproduced and addressed before merge. Atomic
inventory projections now reuse the owning subject/digest/format validation.
Actual public SDK calls with original Obsidian data demonstrated the old
acceptance of null subjects and inconsistent formats; the corrected runtime
rejects both with `invalid_request`. Six boundary assertions also cover missing
subject, wrong format and wrong digest across summary and node views.
The related three-file run passes 37 assertions, and full static/docs gates
pass. Genuine baseline SQLite/Obsidian import/view/export still preserves whole
original records and source/view identities with WABT unavailable. A new real
Obsidian summary workflow preserves every selected fact, atomic source identity
and 16,322 delivered JSON bytes; catalog bytes remain 2,359,082.
Both experimental drivers reject undeclared inherited provider controls before
server/profile creation and log no values. Explicit settings still override
the corresponding inherited values. This prevents claiming equal declared
settings while silently inheriting a different analysis option; it does not
certify every aspect of the inherited system/runtime environment. These are
research-driver controls, with no production configuration changes.
The source/static scope records 98 `memory.high` events, no hard limit/OOM event,
and a sampled peak of 3,221,209,088 bytes. The real migration/Obsidian scope peaks
at 677,367,808 bytes with no limit/OOM events. All raw receipts remain outside Git.
A later WASM finding was also reproduced: portable sources with absent subject
or a different subject path could report another artifact path. The producer and
view now share an exact path/digest identity rule. Actual WABT inspection of the
published VS Code Tree-sitter INI grammar (0.3.1, 4,716 bytes, SHA-256
`1e8cdf3027209100fa54a43d0faed69721ae1116428164c8ad6dcc496a9aa6af`)
supplied the unchanged report for public SDK before/after checks. The old runtime
accepts both inconsistencies; the corrected runtime rejects both with
`invalid_request` and WABT unavailable. Its 26 focused assertions and type/lint
checks pass, and genuine old records still round-trip unchanged. The verification
scope peaks at 3,204,509,696 bytes with no observed limit/OOM event. The initial
CSS producer exceeded the MCP reply budget as expected; that failed probe is
preserved separately and is not credited as a completed identity reproduction.
These revisions identify different experiment stages; model measurements were
not rerun or relabeled as post-integration measurements.
| Stage | Source revision |
| ---------------------------------------------- | ------------------------------------------ |
| Original baseline | `42b0ed126df7d5b908ac369715378d5e519051b7` |
| Inventory delivery pairs | `f110618184e99f89fa776bc5e59703a61baa7c57` |
| Initial WASM pilot | `0002caa07ffd7319672a21ce901dbfbc55e21aac` |
| Exact WASM summary guidance | `259e99dd906290189217cf0fb1a19769bcf87101` |
| Main-integrated package/migration verification | `0eb42a1eb8ab17aed04240174cb77612e733a1d8` |
The baseline runtime file-table SHA-256 is
`3a1765a6fec0031db0b288112d4db9bc0dd503c9c3aa0946f07ce57f7673ab4b`;
the integrated runtime file-table SHA-256 is
`e99d3bdc851822b77e7d5bfaa9b60c57e7b99bd66e27a739bdea2e9300560d5f`.
The integrated lockfile SHA-256 is
`0a55042141899a442a39dce3d49755c65a861f86d1930bd4f9c6d89cd53f5eba`.
Actual old migration inputs have SHA-256
`e1ff3d28801436bcc11693f79cc847a4f31234f49d88449f9e0d77215c3765fc`
(SQLite bundle) and
`7cecd07012bb79f3afad2ea9e70357e056e9eab48c1aa13bbbd310cdcb43eb9f`
(Obsidian complete inspection reply). The source WASM identity
`ev_16b9f6e0fcefcab9603f14fd5e63ea00c4749accd94fef4a4e9fbb260d2e0140`
and atomic inventory identity
`ev_22c17e52489f3ec0e7298a62da37bd985dbbec202d7208a0bb1c20d863790aed`
remain unchanged after the recorded import/view/export workflows.
### Complete carrier identity and provider search configuration
The outer `inspect_artifact` carrier now uses the same owning subject/digest/format
validator as its atomic inventory. Invalid imported/inline carriers are rejected
before selected-view construction; the generic historical Evidence parser and
valid complete records are unchanged. Six additional boundary assertions cover
missing subject, wrong format and wrong digest across summary/node projections.
The negative checks use the small keyed-archive workflow so the original large
ZIP retention/capacity workflow keeps its existing deadline and distinct coverage.
An initial large-workflow placement timed out and is retained as a failed attempt.
The three public MCP workflows, build, fast static gate and generated docs pass.
The final static/test scope observed one soft `memory.high` event, with no hard
memory, task-limit, OOM or OOM-kill event; it is not a performance trial.
Both research drivers fingerprint the effective provider `PATH`, including an
explicit caller override, without serializing its value or the ambient environment.
The SDK comparator rejects absent or unequal search-path fingerprints. This
records search configuration, not executable replacement at an unchanged path;
provider/runtime identities and controlled host conditions still matter.
Actual public SDK calls using the original shipped Obsidian inspection reproduce
accepted null/wrong-format outer subjects on frozen `2ef0467bb` and `invalid_request`
on the corrected compiled runtime. Two fresh Obsidian replays with different
inherited search paths preserve selected facts and source identities, but the
comparator rejects matching settings with only `provider_search_path_sha256`
different. The original records and failed/accepted replies stay in private lab
receipts; no environment value is copied into them.
### Resource-counter closeout qualification
The experimental wrapper now attempts a final cgroup event read before waiting
or cleanup. Disappeared/incomplete counters or still-owned descendants make
`resource_counters_verified` false; last observations remain available separately.
The model grader treats missing final counters as unverified, never as zero,
and any recorded soft/hard/task-limit event prevents an unrestricted resource
pass. Lightweight parser probes cover missing/incomplete and nonzero counters;
an actual short command confirms disappearance is reported as unknown. Regrading
one already-correct SQLite model answer preserves its facts but rejects the old
receipt's absent final resource counters. Earlier receipts record last-sampled
observations, not verified final counters. Their bytes, facts and token reports
remain original observations; resource-qualified performance conclusions need
this additional closeout evidence. No analyzer/build/schema/package behavior is
changed, so unrelated verification is not repeated for this research-driver fix.
The next bounded WASM delivery cohort is documented in [first-call summary measurements](wasm-first-call.md).
@@ -1,65 +0,0 @@
"""Count completed Android MCP tasks; estimates do not establish model billing."""
import argparse
import hashlib
import json
from pathlib import Path
import tiktoken
def compact(value):
return json.dumps(value, ensure_ascii=False, separators=(",", ":"), allow_nan=False)
def inspect(path, encoding):
report = json.loads((path / "report.json").read_text())
catalog = json.loads((path / "catalog.json").read_text())
calls = json.loads((path / "calls.json").read_text())
assert len(report["runs"]) == 2 and len(calls) == 4
assert all(run["answer_verified"] and run["calls"] == 2 for run in report["runs"])
count = lambda text: len(encoding.encode(text, disallowed_special=()))
inputs = [{key: tool[key] for key in ["name", "description", "inputSchema"]}
for tool in catalog["tools"]]
question = (f"For {report['target']}, find the receiver declaring "
"android.provider.Telephony.SMS_RECEIVED and report its identity, permission, "
"enabled/process declarations, per-filter priority and source position. "
"Preserve source coverage and export complete parent Evidence. "
"Static declarations do not establish runtime behavior.")
rows = []
for index, run in enumerate(report["runs"]):
selected = calls[index * 2:index * 2 + 2]
assert [call["name"] for call in selected] == ["inspect_android_package", "export_evidence_bundle"]
assert not any(call["result"].get("isError") for call in selected)
arguments = sum(count(compact({"name": call["name"], "arguments": call["arguments"]}))
for call in selected)
text = sum(count(block["text"]) for call in selected
for block in call["result"]["content"] if block["type"] == "text")
wire_calls = [{key: call[key] for key in ["name", "arguments", "result"]} for call in selected]
rows.append({**run, "estimated_tokens": {
"question": count(question), "input_catalog_once": count(compact(inputs)),
"arguments": arguments, "text_replies": text,
"whole_task_text_projection": count(question) + count(compact(inputs)) + arguments + text,
"sdk_catalog_and_calls": count(compact({"catalog": catalog, "calls": wire_calls})),
}, "inspection_sdk_reply_bytes": len(compact(selected[0]["result"]).encode()),
"inspection_sdk_reply_tokens": count(compact(selected[0]["result"]))})
return {"case": str(path), "mode": report["mode"], "sha256": report["sha256"],
"runs": rows, "actual_model_usage": None,
"receipts_sha256": {name: hashlib.sha256((path / name).read_bytes()).hexdigest()
for name in ["report.json", "catalog.json", "calls.json"]}}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("cases", type=Path, nargs="+")
args = parser.parse_args()
encoding = tiktoken.get_encoding("o200k_base")
records = [inspect(path, encoding) for path in args.cases]
assert len({record["sha256"] for record in records}) == 1
args.output.write_text(json.dumps({"tokenizer": f"tiktoken {tiktoken.__version__}",
"encoding": encoding.name, "scope": "Text projection counts a fixed question, discovered name/description/input schemas once, all RPC arguments and text replies including proof export. Full SDK transport also counts output schemas and structuredContent. Host context injection and model billing are unmeasured.",
"records": records}, indent=2) + "\n")
if __name__ == "__main__":
main()
@@ -1,222 +0,0 @@
# Android component declarations and selected retained facts
Addresses [#4448](https://github.com/morluto/rea/issues/4448), extending the inventory
already on main. Builds on Harsh's partial [#7264](https://github.com/morluto/rea/pull/7264)
revision `987b5aceb66c7526e47cf6d064c22029c2f48c04`, including its process budget.
The historical-field correction preserves omission and marks unknown coverage;
new fields are never invented as absent in old Evidence.
## Owner and compatibility
The Android manifest parser owns declaration interpretation. A shared domain
contract replaces duplicated provider/domain declarations; a single namespace
reader and one nested-declaration pass produce structured filters and existing
flattened action/category fields. Source positions address the exact retained
decoded text. Unsupported data constraints and filter constructs mark partial
coverage, including scheme-specific-part attributes outside the projected fields.
The producer emits all current fields; historical readers accept omitted fields.
Existing complete requests and the existing default summary remain available.
An explicit components view selects exact names/kinds/actions/exported declarations,
fields and pages. Component summaries count explicit/default exports separately.
CLI and MCP share the retained projection and retain complete parent Evidence.
Repeated views reuse validated sealed records under the existing session lifetime;
no new persistent provider cache or live-state freshness claim is introduced.
Rows always retain the parent array ordinal and declaration coverage. The APK
subject, engine identity, source status, limitations and parent linkage remain
available. Nothing here establishes runtime enabling, resource resolution, Android
URI matching, signature authenticity or execution of an app.
## Reproduction inputs
The old runtime is main `15a8f6847ccac581c7ff717b42062a54b1ba8084`.
The timing/token qualification below was captured on that pinned base before the
feature was rebased onto main `6207843814e5267e3a1d9796d5800346f69dc7da`.
The Android producer, bridge and view sources had no upstream changes in that
range; latest-main integration is checked separately rather than relabeling the
older runtime receipts.
Actual verification uses Linux x86-64, Node 24.22.1, Temurin 21.0.12.1,
JADX headless MCP 0.7.1 and the existing REA Java bridge protocol 7.
JAR SHA-256: `6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75`.
The public [Appium ApiDemos 6.0.18 APK](https://github.com/appium/android-apidemos/releases/download/v6.0.18/ApiDemos-debug.apk)
is 6,521,217 bytes, SHA-256
`a9eecf37b26cd084855c530db81c2bb1b91f4c1b095a04f47aa7c20e2791f686`.
It has 355 declared components and 152,541 decoded manifest UTF-16 units.
The recorded sorted relative-path/SHA-256 tables cover compiled `dist/*.js` and
JSON recursively, the dispatcher/environment helper and Android bridge. Table
SHA-256: old runtime `9581afe8ba48c15d803dd7ed3ffc4d10e4e4b99857a0b90206c8012e266daffe`
(2,435 files), final candidate
`a78698c12ae1dc514ffd3f09ca1b881747c9dde7d1a8677ceed7de3233334c40`
(2,439 files). Raw tables remain with the private receipts.
`android-components.mjs` exercises actual public stdio MCP tools and validates
results against the selected named SDK tool definition. Supply `--repository`,
`--target`, a fresh `--output` directory, `--mode before|after`, `--java-home` and
`--jar`. It captures the selected catalog, every RPC, complete exported parents,
first-capture and warm-session results, and checks APK identity and an independent
stdlib ElementTree oracle. No production manifest parser is reused by that oracle.
Keep raw results, APKs and resource receipts outside Git. The selected tool families
are android/application/session, with compact input schemas, two allowed CPUs,
one analysis worker, 1 GiB JVM heap and a 3 GiB Node heap. A separate supervisor
samples process-family memory every 100 ms, with an 8 GiB early stop and a 4 GiB
host-available-memory floor. No guard stop or OOM occurred in these measurements.
## Equivalent completed question
Find the receiver declaring `android.provider.Telephony.SMS_RECEIVED` and report
its identity, permission, enabled/process declarations, per-filter priority and
source position. The old runtime needs complete manifest delivery for the new
facets. The selected candidate returns the matching row directly. Both treatments
make one inspection plus one complete Evidence export per capture; the oracle
verifies one positive match and the total count of 355 source declarations. An additional capture
uses the same server/engine session. First capture is a fresh process/session,
not a flushed OS page cache. Export costs and the extra call are included.
Initial candidate qualification passed the independent oracle. It measured
first/warm inspection RPCs of 11.395/0.329 seconds versus the old runtime's
10.534/0.314 seconds. The complete two-capture process family took 19.065 seconds,
27.687 CPU seconds and 1,193,615,360 peak sampled RSS bytes versus
16.534 seconds, 24.675 CPU seconds and 1,107,955,712 bytes. This run predates the
additional unsupported-data-attribute coverage correction. It establishes no
latency, CPU or memory improvement; retained data has more declaration facts.
The final coverage-corrected candidate and a repeated immutable-main control
also passed the identical oracle. Preserve all four receipts rather than choosing
a favorable timing:
| Runtime, execution order | First/warm inspection RPC seconds | Whole two-capture seconds | CPU seconds | Peak sampled family RSS bytes |
| ------------------------ | --------------------------------: | ------------------------: | ----------: | ----------------------------: |
| Main control 1 | 10.534 / 0.314 | 16.534 | 24.675 | 1,107,955,712 |
| Initial candidate | 11.395 / 0.329 | 19.065 | 27.687 | 1,193,615,360 |
| Final candidate | 14.558 / 0.571 | 23.369 | 36.031 | 1,145,073,664 |
| Main control 2 | 26.348 / 0.511 | 36.537 | 36.381 | 1,168,494,592 |
The repeated main control shows substantial host/run variation. These samples
establish neither a deterministic slowdown nor a speedup; warm candidate RPCs
are slightly slower in both observations. RSS is the sum of family RSS sampled
every 100 ms, so shared pages may count more than once and short peaks may be
missed; it is not a kernel/cgroup peak. Tokenizer processing is outside the timed
SDK runs. Local oracle work, hashing, discovery, serialization, proof export and
cleanup are inside. Cold/warm CPU and memory are not separately measured.
For the first main control and final candidate, `o200k_base` estimates give:
| Cost per first capture | Old complete | Selected components |
| ------------------------------------------------------------ | -----------: | ------------------: |
| Inspection reply bytes | 978,381 | 11,307 |
| Inspection reply estimated tokens | 225,926 | 2,715 |
| Selected family name/description/input-schema catalog tokens | 14,855 | 15,346 |
| Question + input catalog + both call arguments/text replies | 118,430 | 16,893 |
| Full SDK catalog, including output schemas, + both calls | 516,533 | 299,566 |
The primary text projection counts a fixed question, the input catalog once, all
call arguments and every text reply, including export receipts. The SDK transport
row additionally counts output schemas and structuredContent; it is not equated
with model context. Reproduce with `android-component-costs.py --output costs.json
BEFORE_DIRECTORY AFTER_DIRECTORY` (requires tiktoken). The manifest file exported locally is retained
proof, not inline model context. These are tokenizer estimates of scripted SDK
workflows, not billed model usage or a measured autonomous-agent task. They do
not establish fewer model rounds or a generic analysis speedup. Actual provider
work still reads the complete manifest, and schema discovery grows.
## Global catalog compatibility
The actual CI merge `10746a4e8f8c622357cf8dd529f98adc8bf06d2b` combines the feature
with main parent `86d4a21215cb0fbafaf555b5c7d9fcda3f255d82`. Real SDK discovery,
using the same complete contract registrations as the existing profile test, gives:
| Explicit wire profiles | Pure main bytes | CI merge bytes |
| ------------------------------ | --------------: | -------------: |
| Full input / full output | 4,398,561 | 4,425,163 |
| Compact input / compact output | 556,450 | 558,653 |
| Compact input / output off | 302,913 | 305,116 |
| Full input / output off | 1,047,039 | 1,052,239 |
Input presentation defaults to compact; output presentation defaults to full.
For frame-capped hosts, compact/off outputs are explicit compatibility choices.
The bounded compatibility combinations retain the existing one-megabyte test
limit and per-schema budgets. New selectors add 5,200 bytes with full inputs:
2,699 input bytes and 379 description bytes on `inspect_android_package`, plus
2,122 input bytes on the retained-view operation. Compact-input discovery grows
2,203 bytes. These global discovery costs are distinct from the selected-family
task token estimates above.
This change crosses the old test assertion for full input with output off.
That assertion was broader than its default-compact test name and the existing
profile documentation: full inputs deliberately preserve complete guidance and
can dominate the frame even without output schemas. The corrected test retains
both compact-input frame limits, compares every full input schema with its named
canonical advertisement under full/off outputs, preserves the entire tool set,
and verifies that off removes only output advertisements. No production limit is
raised and no full input validation/guidance is discarded. The owning guide now
states this full-profile boundary explicitly.
## Verification boundaries and failed attempts
The real Java regression exercises the existing raw/default-package class selector,
native absence, references and frozen metadata around neighbouring source generation.
An initial assertion compared the entire usage object: JADX legitimately populated
`def_pos` from 0 to 103 after generating source. The revised assertion excludes only
that presentation position and keeps class/method identity, raw DEX invoke IDs,
descriptor/overload facts and invoke offsets. Production bridge behavior and version
are unchanged.
One combined focused run passed 121 checks but hit the existing 30-second MCP test
timeout while advertising every unrelated tool family to the SDK. Its failed receipt
is retained. The Android-specific test now selects the same relevant families as
the real SDK workflow; its named schemas, retained manifest/source-removal behavior,
full/summary compatibility and complete Evidence export assertions remain enabled.
A subsequent run exposed a test assertion that incorrectly expected repeated
acquisitions to reuse the first Evidence ID. Each selected capture now proves its
own retained parent exists and has the same artifact and normalized facts; engine
reuse does not collapse distinct acquisition identities.
ApiDemos contains no URI data declarations. The checked-in manifest/resources and
Smali container permit a second source-built APK with real AAPT2 binary XML and
JADX decoding, including two filters, URI data, metadata, a resource boolean,
explicit process and provider URI grants. Compiled APK/DEX outputs are private.
Static declaration verification does not execute these component classes.
The source-built APK SHA-256 was
`aea6ac64630ad70e93f4d85de0237cc51a7012b077ca0d0e7b24cf22c5737532`.
Its real CLI selected view and retained bundle passed `verify-fixture.py`, including
AAPT2 dump comparison, all source positions, grouping, raw resource references,
metadata names and provider URI grants. The real SDK VIEW query passed the
independent ElementTree oracle on first and warm capture. An initial fixture build
used Smali's CLI and failed because the engine JAR does not bundle optional
JCommander; `BuildDex.java` now calls the already bundled Smali API directly.
The real metadata helper passed all Java markers and downstream CLI/MCP method
body and direct-invoke parity checks for both classes.dex and classes2.dex:
149.436 seconds, 204.026 CPU seconds and 1,070,514,176 sampled peak family RSS bytes.
No production bridge change was needed for default-package selectors.
A genuine complete parent from the old runtime also passed a new CLI retained
component query with an explicitly nonexistent engine JAR. Its newly introduced
fields remained omitted and row coverage remained unknown; no APK/provider
reopening was required. The current source checks cover 111 parser cases,
six projection cases, four CLI early-validation cases and four MCP cases,
including named SDK-converted schemas and retained source-removal behavior.
After the rebase onto `6207843814e5267e3a1d9796d5800346f69dc7da`, type/lint,
all 125 focused cases and the nine documentation conformance cases passed again.
A fresh real ApiDemos SDK qualification also passed the independent oracle:
first/warm RPCs 11.291/0.290 seconds, whole two-capture elapsed 17.008 seconds,
CPU 26.425 seconds and peak sampled family RSS 1,166,245,888 bytes. This is latest-main
integration evidence; it is not paired with a new baseline performance cohort.
Review follow-up: Greptile correctly identified that the original oracle filtered
before comparison, so the timing cohorts above verified selected-receiver facts
and the inventory count, not every unselected component's facts. The oracle now
returns an unfiltered inventory and derives the selected list from it. The driver
checks every retained parent's identity, enabled/raw-enabled/process/permission,
filter grouping/data/priority, metadata names, declaration coverage and component
source location before checking the selected page. The receipt records both the
verified count and scope; historical mode checks the available identity/permission
facets without inventing the new observations. A fresh real ApiDemos first/warm
run passed all 355 rows in both captures (16.538 seconds, 24.977 CPU seconds,
1,176,817,664 sampled peak family RSS bytes). The older receipts remain unchanged. The unfiltered historical-mode run also
passed all 355 available identity/permission rows in both captures; the compiled
manifest fixture passed all three current rows plus its selected VIEW page.
Platform coverage is Linux/JDK21/JADX0.7.1 only. Android devices, emulators, actual
runtime enabling/registration, other JDKs and Windows/macOS are unverified here.
@@ -1,294 +0,0 @@
/** Real APK task through public MCP, with complete retained Evidence and an independent XML oracle. */
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { createReadStream } from "node:fs";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { performance } from "node:perf_hooks";
import { createVerifierCommands } from "../../scripts/lib/verify-commands.mjs";
const { values } = parseArgs({
options: {
repository: { type: "string" },
target: { type: "string" },
output: { type: "string" },
mode: { type: "string" },
"java-home": { type: "string" },
jar: { type: "string" },
action: {
type: "string",
default: "android.provider.Telephony.SMS_RECEIVED",
},
},
});
for (const name of [
"repository",
"target",
"output",
"mode",
"java-home",
"jar",
])
assert.ok(values[name], `Missing --${name}`);
assert.ok(["before", "after"].includes(values.mode));
const repo = resolve(values.repository),
target = resolve(values.target),
output = resolve(values.output);
await mkdir(output, { mode: 0o700 });
const digest = createHash("sha256");
for await (const chunk of createReadStream(target)) digest.update(chunk);
const sha256 = digest.digest("hex");
const require = createRequire(join(repo, "package.json"));
const { Client } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client"))
);
const { StdioClientTransport } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio"))
);
const commands = await createVerifierCommands();
const env = {
PATH: process.env.PATH,
JAVA_HOME: resolve(values["java-home"]),
REA_JADX_MCP_JAR: resolve(values.jar),
REA_JADX_HEAP_MIB: "1024",
REA_JADX_ACTIVE_PROCESSOR_COUNT: "2",
NODE_OPTIONS: "--max-old-space-size=3072",
UV_THREADPOOL_SIZE: "2",
REA_LOG_LEVEL: "silent",
REA_MCP_TOOL_FAMILIES: "android,application,session",
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
};
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(repo, "scripts/rea.mjs"), "mcp"],
cwd: repo,
env,
stderr: "pipe",
maxBufferSize: 16 * 1024 * 1024,
});
const diagnostics = [];
let diagnosticBytes = 0;
let omittedDiagnosticBytes = 0;
transport.stderr?.on("data", (chunk) => {
const bytes = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
const retained = bytes.subarray(0, Math.max(0, 65536 - diagnosticBytes));
if (retained.length > 0) diagnostics.push(retained);
diagnosticBytes += retained.length;
omittedDiagnosticBytes += bytes.length - retained.length;
});
const client = new Client({
name: "rea-real-android-components",
version: "1",
});
let catalog;
const calls = [],
runs = [];
const invoke = async (name, arguments_) => {
const definitions = catalog.tools.filter((tool) => tool.name === name);
assert.equal(definitions.length, 1);
const start = performance.now();
const result = await client.callTool(
{ name, arguments: arguments_ },
{ toolDefinition: definitions[0], timeout: 120000 },
);
calls.push({
name,
arguments: arguments_,
result,
elapsed_ms: performance.now() - start,
});
assert.notEqual(
result.isError,
true,
`${name}: ${JSON.stringify(result.content)}`,
);
return result.structuredContent;
};
const selectedFields = [
"kind",
"declared_name",
"name",
"exported",
"exported_default",
"permission",
"permission_source",
"enabled",
"enabled_declaration",
"process",
"intent_filters",
"meta_data",
"location",
];
const stripLocations = (row) => {
const { location: _location, ...fields } = row;
return {
...fields,
intent_filters: row.intent_filters?.map(
({
location: _filterLocation,
source_index: _filterIndex,
coverage: _coverage,
...filter
}) => ({
...filter,
data: filter.data.map(({ location: _dataLocation, ...data }) => data),
}),
),
meta_data: row.meta_data?.map(
({ location: _metadataLocation, ...metadata }) => metadata,
),
};
};
const assertOracleFacts = (row, expected, source) => {
assert.equal(row.declaration_coverage, "complete");
const cleaned = stripLocations(row);
for (const [key, value] of Object.entries(expected))
assert.deepEqual(
cleaned[key],
value,
`Independent oracle: ${expected.source_index}.${key}`,
);
assert.equal(
source.slice(
row.location.offset,
row.location.offset + row.kind.length + 1,
),
"<" + row.kind,
);
};
try {
await client.connect(transport);
catalog = await client.listTools();
await writeFile(join(output, "catalog.json"), JSON.stringify(catalog));
for (const temperature of ["first-capture", "warm-session"]) {
const firstCall = calls.length;
const start = performance.now();
const result = await invoke(
"inspect_android_package",
values.mode === "before"
? { path: target }
: {
path: target,
detail: "summary",
view: {
kind: "android",
facet: "components",
actions: [values.action],
fields: selectedFields,
limit: 64,
},
},
);
const rpcMs = performance.now() - start;
const normalized = result.normalized_result;
const parentId =
values.mode === "before"
? result.evidence_id
: normalized.parent_evidence_id;
const path = join(output, temperature + "-evidence.json");
await invoke("export_evidence_bundle", { path });
const bundle = JSON.parse(await readFile(path, "utf8"));
const parent = bundle.records.find(
(record) => record.evidence_id === parentId,
);
assert.ok(parent);
assert.equal(parent.subject.digest.sha256, sha256);
const oraclePath = join(
dirname(fileURLToPath(import.meta.url)),
"fixtures/android-components/manifest-oracle.py",
);
const oracle = JSON.parse(
(
await commands.exec(
"python3",
[oraclePath, path, parentId, values.action],
{ timeout: 15000, maxBuffer: 4 * 1024 * 1024 },
)
).stdout,
);
assert.ok(
oracle.components.length > 0,
"The completed task must exercise a real positive action match",
);
const complete = parent.normalized_result;
assert.equal(complete.components.items.length, oracle.inventory.length);
assert.equal(oracle.inventory.length, oracle.source_components);
for (const [index, expected] of oracle.inventory.entries()) {
const row = complete.components.items[index];
if (values.mode === "after")
assertOracleFacts(
{ ...row, source_index: index },
expected,
complete.manifest.text,
);
else {
for (const key of ["kind", "declared_name", "name", "permission"])
assert.deepEqual(
row[key],
expected[key],
`Legacy inventory: ${index}.${key}`,
);
}
}
if (values.mode === "after") {
assert.equal(normalized.facet, "components");
assert.equal(normalized.source_status, "complete");
assert.equal(normalized.source_components, oracle.source_components);
assert.equal(normalized.coverage.exhausted, true);
assert.equal(normalized.coverage.total, oracle.components.length);
assert.equal(normalized.items.length, oracle.components.length);
for (let i = 0; i < oracle.components.length; i++) {
const row = normalized.items[i],
expected = oracle.components[i];
assertOracleFacts(row, expected, complete.manifest.text);
}
} else {
for (const expected of oracle.components) {
const row = normalized.components.items[expected.source_index];
assert.equal(row.name, expected.name);
assert.ok(row.actions.includes(values.action));
}
}
runs.push({
temperature,
rpc_ms: rpcMs,
calls: calls.length - firstCall,
matched_components: oracle.components.length,
source_components: oracle.source_components,
parent_evidence_id: parentId,
answer_verified: true,
inventory_verified: oracle.inventory.length,
inventory_scope:
values.mode === "after"
? "declared-structured-facets"
: "legacy-identity-permission",
});
}
} finally {
// exec drains already adopted owners; adopt the live MCP client only at its final cleanup boundary.
commands.adopt(client, "real Android component MCP client");
await commands.close();
await writeFile(join(output, "stderr.log"), Buffer.concat(diagnostics));
await writeFile(join(output, "calls.json"), JSON.stringify(calls), {
mode: 0o600,
});
await writeFile(
join(output, "report.json"),
JSON.stringify(
{
mode: values.mode,
target,
sha256,
runs,
tool_count: catalog?.tools.length,
omitted_diagnostic_bytes: omittedDiagnosticBytes,
},
null,
2,
),
);
}
console.log(JSON.stringify({ mode: values.mode, runs, verified: true }));
@@ -1,166 +0,0 @@
# Android package summary and exact retained manifest
Continues [#2016](https://github.com/morluto/rea/issues/2016) and
[#2438](https://github.com/morluto/rea/pull/2438). The recurring NewPipe question
asks for APK/package/version identity, all case-sensitive MainActivity matches
and all reported onCreate entries, including explicit null DEX descriptors.
Complete package replies delivered decoded manifest text and duplicate upstream
calls unrelated to those selected facts.
## Compatibility and ownership
MCP `inspect_android_package` accepts explicit `detail: "summary"`; omitted and
`"complete"` requests preserve the complete output and provider request. The
one-shot CLI stays complete. Summary selection happens in the application
workflow, independently of the engine protocol, using the existing retained
analysis-view machinery. No third-party plugin API or persistent analysis cache
is introduced. Servers lacking retention reject summary before provider work.
The complete original Evidence stays under its original ID and session lifetime.
A distinct linked derived view returns all package/SDK/permission declarations,
engine identity, class/resource counts and signature-verification status.
Manifest metadata retains the producer's partial/complete status, nullable
reported byte total, and observed UTF-16 length. Text and duplicate raw calls
remain in the original record, rather than being rewritten under its ID.
Use `inspect_analysis_view` with the parent's ID and
`view: {kind: "android", facet: "manifest", offset: 0, limit: 4096}` to read exact
retained decoded text. Follow `coverage.next_offset`; surrogate boundaries use
the existing text-page implementation. Exhaustion describes the retained text,
not global completeness when the producer says partial. APK signatures and
runtime behavior remain unverified. Engine artifact SHA-256 identifies the JAR;
views require the authenticated APK subject and never substitute the engine
hash for a missing target identity. Older servers require schema discovery and
the original complete request; automatic fallback is not introduced.
## Frozen runtime and real SDK pairs
Source runtime: `28c84e78cf2fbfde4aceed345b773c59eb889f68`; later test-only schema
typing corrections do not relabel these receipts. Compiled dist/bridge file-table
SHA-256: `5332dda13981f043e146b3601ebec1f380d9f698280ca53d75786b638a912e11`.
The frozen old runtime is `8bc1601ca51e417ca472d90ea0367e9418a1a0f2`, digest
`9e7380f537301bf0cfe2926f5619011af848af7c220750e75189498e65d825e1`.
Real NewPipe 0.29.1 bytes, acquisition and digest are pinned in `corpus.json`.
Actual configured JADX headless MCP 0.7.1/Java runs use the same selected
provider settings and search-path digests, one worker and 768 MiB Java heap.
Node is 24.18.0; actual SDK client/server are 2.3.1. Processes and engines are
fresh per trial; host page caches are not flushed. SDK schema mode is selected
in both treatments. Heavy runs are serial, with one CPU, unchanged 3 GiB early
stop/4 GiB hard memory, no swap, and 192 tasks.
All four candidate workflows use three successful public MCP calls. Only the
first package delivery changes. Both pairs pass settings and selected-fact
comparison, including all declarations, manifest status/length, engine identity,
original limitations, matching class rows and onCreate entries. Original replies
and IDs remain separate. A fresh old-runtime complete workflow also passes
comparison against the candidate complete request; both complete package replies
are exactly 119,527 compact client-result JSON bytes. This is scoped compatibility
evidence, not a guarantee for every historical release/client/host.
| Trial | Package reply bytes | SDK workflow seconds | Payload CPU seconds | Kernel peak bytes |
| -------------- | ------------------: | -------------------: | ------------------: | ----------------: |
| sdk-complete-1 | 119,527 | 24.059 | 25.259 | 1,225,076,736 |
| sdk-summary-1 | 8,659 | 23.694 | 24.894 | 1,226,870,784 |
| sdk-summary-2 | 8,659 | 24.407 | 25.611 | 1,228,296,192 |
| sdk-complete-2 | 119,527 | 24.509 | 25.740 | 1,229,844,480 |
Summary replies are 8,659 bytes, about 92.8% smaller; all three replies together
fall from 149,577 to 38,709 bytes. This is a delivery result, not a model-token
or engine-work saving. The SDK timing differences are small. CPU and memory do
not establish a reduction. No provider setting or analysis scope changes.
Transport roundtrip includes server/client work and is not an engine-only timer.
The complete catalog is 2,506,213 compact JSON bytes versus 2,497,635 on the old
runtime: about 0.34% growth, including intervening main changes. Discovery cost
remains visible alongside selected-delivery savings.
## Authentic old Evidence continuation
A real package record from old runtime 8bc160 imports into the candidate and
supports retained summary, exact manifest pages, inline portable summary and
export through actual public SDK tools. The new session explicitly selects an
absent JADX JAR; this workflow performs no package analysis. Eleven successful
calls preserve the original record, original ID, all declarations and all 25,222
manifest UTF-16 units. Manifest UTF-8 SHA-256:
`fee192fe2a6d7892c22cba115184c74c7e2c42ce464e8bb37be6289f2c16881d`.
The exported original record is deeply equal to the imported record.
## Actual Codex task outcomes
Four serial trials run in complete/summary/summary/complete order on the same
frozen candidate. Requested configuration: Codex CLI 0.161.0, `gpt-5.6-terra` high,
three exact selected tool names and the same optional context/discovery capture
settings. The requested alias does not certify the server-resolved model. The
original analyst question is unchanged; equal treatment instructions explicitly
choose package detail and explain the summary carrier. Temporary configuration
points to the frozen modified REA MCP; no global configuration is edited.
| Trial | Input | Cached | Uncached | Output | Seconds | MCP calls | Usage records | Facts |
| ---------------- | ------: | -----: | -------: | -----: | ------: | --------: | ------------: | ----- |
| model-complete-1 | 39,267 | 23,040 | 16,227 | 1,690 | 94.053 | 3 | 3 | Pass |
| model-summary-1 | 128,417 | 93,440 | 34,977 | 3,633 | 150.401 | 8 | 9 | Pass |
| model-summary-2 | 54,437 | 37,120 | 17,317 | 2,185 | 123.677 | 3 | 4 | Pass |
| model-complete-2 | 21,332 | 9,984 | 11,348 | 667 | 20.774 | 0 | 2 | Fail |
Three answers pass the original selected-fact gate against the actual SDK
producer. Each trial preserves APK bytes and removes its isolated profile.
The first summary trial makes five successful calls and three rejected requests:
an invented `case_sensitive` argument, empty class-search input and empty
class-inspection input. It then repeats successful package/search work. These
errors and correction turns remain part of measured task cost; strict existing
schemas correctly reject them. The second summary trial uses the intended three
successful calls, but still does not establish improved total/uncached input or
latency against the factually acceptable first complete control.
The final complete control's first client executor filters `ALL_TOOLS` for the
same three exact names and receives an empty array. It makes zero MCP calls,
reports all requested facts unknown and fails the factual gate. Identical name
filters worked in the other trials. Stderr contains no explanatory error; the
client visibility failure remains unexplained and is not attributed to REA or
summary delivery. Its short duration is not comparable completed work. The
original failed receipt is retained rather than replaced by a successful trial.
These measurements demonstrate no real-model token or speed gain. Smaller raw
MCP envelopes do not establish smaller model context; observed usage also
includes discovery, execution context, corrections and extra responses. Usage
records count captured usage items rather than asserting exact remote request
contents or equating MCP calls with model rounds. The complete default and
opt-in choice remain unchanged; this is useful delivery/retention capability,
not a general performance recommendation.
All model runs have verified payload-completion witnesses with only the observer
remaining and zero memory/task limit events at that phase. Kernel peaks are
602,472,448–1,444,945,920 bytes; the failed no-analysis control is included in that
range and is not evidence of lower analysis memory. Whole-scope final counters
remain unknown and the existing full resource grader stays false. No broader
CPU, memory, OS/provider or model claim is made.
## Resource interpretation
Every SDK run and portable continuation has a verified owned
payload-exited-observer-alive witness, with only the observer remaining and no
memory/task limit events at that phase. Kernel peak and CPU include that opt-in
observer. Whole-scope terminal counters remain unknown; the existing complete
physical resource grader is not changed or called successful. No OOM stress or
resource-limit increase is used.
## Verification and scope
Twenty-seven relevant converted-schema, SDK, domain and existing Android
failure/cancellation/cleanup checks pass. A three-case focused rerun passes after
a test-only schema typing correction. The 1 MiB manifest protocol fixture verifies
original default/full output, reconstruction after source removal, unchanged
exported Evidence and one engine launch. Other cases preserve nulls and partial
status, Unicode boundaries and tagged malformed/missing-identity failures, and
reject unavailable retention or cancelled requests before provider work. These
fixtures verify production boundaries; actual JADX claims use the real NewPipe
runs above. Full supported single-thread compiler and unchanged lint pass under
the same memory safeguards. No other Android OS/device/provider lane is inferred.
Raw SDK/model replies, private profiles and resource receipts stay outside Git.
Reproduce SDK pairs using the existing `measure.mjs` with `--artifact newpipe.apk`,
`--schemas selected`, identical explicit provider configuration and frozen runtime,
and alternating `--delivery complete` / `--delivery summary`. Preserve failed
attempts and run `compare.mjs` on both pair orders. Model findings require their
own identical analyst facts and explicit delivery controls.
File diff suppressed because it is too large. Load diff
@@ -1,171 +0,0 @@
# Apktool shared decoding and table-only projection
The measured question is the same in every comparison: inspect an APK's
metadata/manifest and its default, German and Brazilian Portuguese string tables.
One existing batch removes repeated decoding. An optional table-only projection
then removes repeated primary strings without changing the selected facts.
This is a static resource-query measurement, not an agent evaluation or a claim
about reconstructing an application's behavior. The machine-readable measured
summary is [apktool-string-tables.json](apktool-string-tables.json).
## Revisions, inputs and execution
- Baseline: `8f7d12e30d63f3d3b4cfe2a90336362841438f56`, latest main at issue start.
- Projection source: `406af90023a545fb280601bf51f445e122c36378`.
- Existing shared decode: [#4777](https://github.com/morluto/rea/issues/4777),
integrated by [#4804](https://github.com/morluto/rea/pull/4804).
- Projection follow-up: [#4982](https://github.com/morluto/rea/issues/4982).
- Linux x86-64, kernel 6.1.0-53-cloud-amd64, Node 24.18.0, npm 11.16.0,
Temurin 21.0.12.1, Apktool 3.0.3. The selected JAR's closeout digest is
`dbf930b076c6b9be08d57c449cacefc3bdd6b71ebd59b3066fc0e1f5b14f9423`;
decoder contents were not independently hashed for every request.
- [newpipe](https://github.com/TeamNewPipe/NewPipe/releases/tag/v0.29.1): 11,507,122 bytes; SHA-256 `18447bfb1e06d113edc88df93f471827280de06f6f3d4dc42f56f28b9c1bab79`. Valid projected counts: 1126/833/779; unsupported entries: 1/1/1.
- [apidemos](https://github.com/appium/android-apidemos/releases/tag/v6.0.18): 6,521,217 bytes; SHA-256 `a9eecf37b26cd084855c530db81c2bb1b91f4c1b095a04f47aa7c20e2791f686`. Valid projected counts: 967/29/29; unsupported entries: 8/0/0.
- [settings](https://github.com/appium/io.appium.settings/releases/tag/v8.2.1): 3,099,589 bytes; SHA-256 `e4024b3ee348929252bcaf5eb422bcab9e4ace96703499a84f3332281c03375f`. Valid projected counts: 25/19/19; unsupported entries: 0/0/0.
The counts are default/de/pt-BR respectively. Source paths, XML digests, exact
caller paths, runtime hashes and per-run identities/costs are in the JSON summary.
The APKs were never installed or executed. Input digests were checked before and
after each completed task; provider-owned workspaces were confirmed absent.
Each task uses a fresh production stdio MCP runtime, complete paginated tool
discovery, client inspection, resource call(s), `close_binary`, transport close
and owned cleanup. OS file caches were not flushed. Every decode has a fresh JVM,
APK snapshot, framework directory and workspace: these are not warm retained
engine/database queries, nor cold-disk measurements.
Tasks run serially on one logical CPU, with an 8 GiB family hard limit, 7 GiB
early stop and no swap. Admission requires 16 GiB host availability; running
tasks stop below 8 GiB host availability. CPU and kernel `memory.peak` cover
the complete owned cgroup, including subprocesses, observer and file cache.
Counters are captured at payload exit while the observer is alive; disappeared
final whole-scope counters remain unknown. All completed payload receipts show
successful owned cleanup and zero observed OOM/limit events at that checkpoint.
## Correctness and owning change
Separate native decodes were retained for independent Python Expat validation.
It checks direct string entries, source entity spelling, unsupported markup
counts, ordering and emitted prefixes against the XML bytes. Manifest text and
package identity are checked separately. All six serial/batch pairs agree on
selected strings, manifest, metadata and target identity. All nine projection
pairs additionally preserve table counts and coverage; batch/table-only counts
and coverage match the XML oracle. This qualifies REA's projection of these
native outputs, not independent binary ARSC decoder completeness.
The existing default batch repeats its primary table in top-level `strings` and
`string_tables[0]`. On NewPipe this made the full resource replies larger than
three serial replies despite reducing decode calls: 616,059 → 680,188 bytes.
ApiDemos' larger common manifest produces the opposite tradeoff. The existing
batch remains the compatibility default.
`string_tables_only: true` (CLI `--string-tables-only`) keeps the complete
requested table facts inline in `string_tables` and leaves `strings` empty by
request. An unrequested field does not establish that a source table is empty.
This also supports one table without `locales`. `include_strings: false` still
suppresses string projection. Omitting the new option preserves old requests:
three real legacy workflows produce identical complete MCP replies, including
Evidence identity, at the new source revision.
The resource projection owner computes the output choice and charges aggregate
entries for the copies actually emitted. The same 10,000-entry policy can thus
return five distinct 2,000-entry tables rather than consuming capacity on a
duplicate primary copy. Per-table limits, XML budgets, physical alias reuse,
coverage, partial observations and cancellation remain at the existing owner.
CLI and MCP share this implementation; there is no new tool or cross-call cache.
## Existing shared decode: three serial calls versus one batch
Three alternating fresh-session pairs per APK, all at the same main revision.
The table reports marginal medians; paired speedups are reported separately.
Elapsed/CPU include discovery, client/Java startup, calls, transcript/hash checks
and cleanup. Native oracle creation, tokenization and verification gates are
separate costs in the JSON, not hidden inside these task timings.
| Target | Resource calls, s | Whole elapsed, s | Whole CPU, s | Family peak, MiB |
| -------- | ----------------: | ---------------: | ------------: | ---------------: |
| newpipe | 23.19 → 7.40 | 35.32 → 19.86 | 34.95 → 19.65 | 1341.7 → 1309.5 |
| apidemos | 17.43 → 6.22 | 30.32 → 18.85 | 30.05 → 18.63 | 1187.1 → 1202.7 |
Paired resource-call speedup medians: NewPipe **3.23×** (2.90–3.23), ApiDemos
**2.80×** (2.66–3.09). Instrumentation confirms three versus one actual decoder
invocations. These call timings include protocol/projection costs; they do not
isolate the Java engine phase. NewPipe's whole raw token estimate increases
951,332 → 962,330 with the default batch; ApiDemos decreases
1,054,893 → 948,591. Fewer calls alone do not establish less context.
## New selector: default batch versus table-only batch
Three alternating pairs per APK; NewPipe repetitions 1/4/3, others 1/2/3.
Repetition 4 replaces a refused start, described below. Both variants still
perform one decode and return the same required facts.
| Target | Resource call, s | Whole elapsed, s | Whole CPU, s | Family peak, MiB |
| -------- | ---------------: | ---------------: | ------------: | ---------------: |
| newpipe | 9.18 → 9.35 | 23.95 → 24.85 | 23.65 → 23.82 | 1327.5 → 1323.6 |
| apidemos | 6.46 → 5.81 | 23.28 → 19.79 | 22.90 → 19.47 | 1247.8 → 1197.0 |
| settings | 4.21 → 4.06 | 17.66 → 19.39 | 17.42 → 19.04 | 1191.6 → 1122.4 |
There is no consistent latency or whole-family memory improvement established
for the selector. NewPipe and Settings have higher whole-task elapsed and CPU
medians. Elapsed ranges overlap: NewPipe 23.53–34.46 → 21.59–42.80 s;
ApiDemos 18.55–27.07 → 19.25–25.96 s; Settings 16.55–23.39 → 16.14–21.72 s.
The cause of this variation was not established; a passing or faster run does
not explain a slower one. The selector's supported benefit is reduced duplicate
delivery, with its full task costs retained here.
Full resource reply bytes fall 680,188 → 506,481 for NewPipe,
616,841 → 463,588 for ApiDemos and 38,725 → 34,199 for Settings.
Offline `tiktoken` 0.12.0 / `cl100k_base` estimates count one complete catalog
page-array plus every call name/argument/reply, including inspection and close.
Full replies include both textual and structured copies. Text-only replies are
also counted separately. These are serialization estimates, not actual model
usage; initialization/framing, prompts, client filtering/rendering/replay and
continuations are unmeasured.
| Target | Text reply tokens | Whole raw transport tokens | Whole estimated reduction |
| -------- | ----------------: | -------------------------: | ------------------------: |
| newpipe | 73,661 → 55,797 | 962,330 → 924,352 | 3.95% |
| apidemos | 65,693 → 49,162 | 948,591 → 913,693 | 3.68% |
| settings | 4,387 → 3,942 | 814,859 → 814,029 | 0.10% |
Discovery still has 207 tools; only `decode_android_resources` changes.
The added schema costs 557 catalog bytes / 113 estimated catalog tokens
(805,217 → 805,330). Raw discovery dominates this transport estimate; it does
not establish what a client supplies to its model. Smaller replies and these
estimates do not prove an actual agent context or investigation advantage.
## Verification, attempts and limits
- 94 source tests: provider/resource projection/service, including alias reads,
aggregate output exhaustion, malformed/physical failures, partial facts,
cancellation and cleanup. The table-only capacity, single-table provider
and CLI cases fail on the old implementation (three selected failures,
82 unrelated tests skipped).
- Nine compiled CLI/MCP tests: schema conversion/advertisement, option forwarding,
Evidence parameters, legacy locale grammar and existing boolean choices.
- 36 completed real MCP tasks and three real CLI tasks. Nine exact projection
oracle pairs; three legacy full-reply comparisons; alias/missing, single-table
and omitted-string cases pass. CLI normalized data, subjects and Evidence IDs
match MCP exactly on all three APKs.
- `check:fast`, `build:cached` and `verify:test-discovery` passed; the last
discovered all 1,579 test files exactly once. `npm run check` and `npm run docs:check` also passed.
- An authored test group initially exceeded the 150-line lint limit (153).
That gate failed after 189.31 s / 186.19 CPU s, peak 4,487,798,784 bytes;
build/compiled checks did not run in that attempt. The capacity test was moved
out of that group, and the complete corrected gate passed after 343.97 s /
338.67 CPU s, peak 4,311,293,952 bytes. No rule was disabled.
- `projection-newpipe-after-2` was refused before scope/payload creation because
host availability was below 16 GiB; a following poll showed about 10.5 GiB.
It has no payload timing/counters and is not a completed measurement. The
refusal stays recorded; a fresh whole repetition-4 pair ran after recovery.
No existing receipts were overwritten and resource thresholds were unchanged.
Raw transcripts, native outputs, intermediate failures and per-request workspace
arguments remain in ignored local audit storage; the committed JSON is a curated
summary. Small Settings resources and negative time results are retained.
Actual models, APK execution, other providers/platforms and general complex
software effectiveness were not verified by this checkpoint.
@@ -1,713 +0,0 @@
/** Compare complete real PocketBase create/update/read tasks through public MCP. */
import assert from "node:assert/strict";
import { readFile, writeFile, mkdir, mkdtemp, rm } from "node:fs/promises";
import { join, resolve } from "node:path";
import { tmpdir } from "node:os";
import { createRequire } from "node:module";
import { pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { performance } from "node:perf_hooks";
import { createVerifierCommands } from "../../scripts/lib/verify-commands.mjs";
import { withVerifierCleanup } from "../../scripts/lib/verifier-cleanup.mjs";
const { values } = parseArgs({
options: {
repository: { type: "string" },
mode: { type: "string" },
output: { type: "string" },
origin: { type: "string" },
credentials: { type: "string" },
browser: { type: "string" },
"source-only": { type: "boolean", default: false },
},
});
assert.ok(
values.repository &&
values.output &&
values.origin &&
values.credentials &&
values.browser,
);
assert.ok(["before", "after"].includes(values.mode));
const repo = resolve(values.repository),
output = resolve(values.output),
origin = new URL(values.origin).origin;
assert.equal(
new URL(origin).hostname,
"127.0.0.1",
"Use an owned loopback fixture",
);
await mkdir(output, { mode: 0o700 });
const credentials = JSON.parse(await readFile(values.credentials, "utf8"));
const require = createRequire(join(repo, "package.json"));
const { chromium } = require("playwright-core");
const { Client } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client"))
);
const { StdioClientTransport } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio"))
);
const commands = await createVerifierCommands();
const profile = await mkdtemp(join(tmpdir(), "rea-saved-values-"));
const env = {
PATH: process.env.PATH,
NODE_OPTIONS: "--max-old-space-size=3072",
UV_THREADPOOL_SIZE: "2",
REA_LOG_LEVEL: "silent",
REA_MCP_TOOL_FAMILIES: "browser,application,session",
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
REA_TEST_EMAIL: credentials.email,
REA_TEST_PASSWORD: credentials.password,
REA_TEST_WHOLE_JSON: JSON.stringify({ token: "private-token-729d" }),
};
const calls = [];
const runs = [];
let catalog;
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(repo, "scripts/rea.mjs"), "mcp"],
cwd: repo,
env,
stderr: "pipe",
});
transport.stderr?.resume();
const client = new Client({ name: "rea-saved-values-real-app", version: "1" });
commands.adopt(client, "saved-values MCP client");
const tokenHeader = (saved) => ({
name: "Authorization",
value: saved
? { source: "saved", name: "token" }
: { source: "local_storage", key: "rea_auth", json_pointer: "/token" },
});
const decodeBodies = (value) => {
const bodies = [];
const visit = (x) => {
if (x && typeof x === "object") {
if (x.encoding === "base64" && typeof x.content === "string") {
try {
bodies.push(
JSON.parse(Buffer.from(x.content, "base64").toString("utf8")),
);
} catch {}
}
for (const v of Object.values(x)) visit(v);
}
};
visit(value);
return bodies;
};
async function invoke(name, args) {
const defs = catalog.tools.filter((t) => t.name === name);
assert.equal(defs.length, 1);
const start = performance.now();
const result = await client.callTool(
{ name, arguments: args },
{ toolDefinition: defs[0], timeout: 90000 },
);
calls.push({
name,
arguments: args,
result,
elapsed_ms: performance.now() - start,
});
assert.notEqual(
result.isError,
true,
`${name} failed; retain the local receipt`,
);
return result.structuredContent;
}
const loginActions = (saved) => [
{
step_id: "email",
action: "fill",
locator: { kind: "css", selector: "#email" },
value: { source: "secret", secret_id: "email" },
},
{
step_id: "password",
action: "fill",
locator: { kind: "css", selector: "#password" },
value: { source: "secret", secret_id: "password" },
},
{
step_id: "login",
action: "click",
locator: { kind: "css", selector: "#login" },
...(saved
? {
save_response: {
method: "POST",
path: "/api/collections/_superusers/auth-with-password",
save: [{ name: "token", json_pointer: "/token", secret: true }],
},
}
: {}),
timeout_ms: 10000,
},
{
step_id: "ready",
action: "wait_for",
locator: { kind: "css", selector: '#result[data-ready="yes"]' },
state: "visible",
timeout_ms: 10000,
},
];
const edits = (id, saved) => [
{
step_id: "update",
action: "request",
method: "PATCH",
path: `/api/collections/rea_items/records/${id}`,
headers: [tokenHeader(saved)],
json: { title: `record-${id}`, quantity: saved ? "{saved:quantity}" : 11 },
expected_status: [200],
},
{
step_id: "read",
action: "request",
method: "GET",
path: `/api/collections/rea_items/records/${id}?at=${id}`,
headers: [tokenHeader(saved)],
expected_status: [200],
},
{
step_id: "open",
action: "goto",
destination: { url: `${origin}/api/collections/rea_items/records/${id}` },
wait_until: "load",
},
];
const captureOptions = {
after_each_step: ["url"],
at_end: ["url", "accessibility"],
events: ["network"],
network: { request_body: true, response_body: true, header_values: true },
};
try {
const context = await chromium.launchPersistentContext(profile, {
executablePath: resolve(values.browser),
headless: true,
args: ["--remote-debugging-port=0", "--no-sandbox"],
});
commands.adopt(context, "owned saved-values Chromium context");
const page = context.pages()[0];
await page.goto(origin, { waitUntil: "networkidle" });
const port = (
await readFile(join(profile, "DevToolsActivePort"), "utf8")
).split("\n")[0];
const endpoint = `http://127.0.0.1:${port}`;
const targets = await (await fetch(endpoint + "/json/list")).json();
const target = targets.find(
(t) => t.type === "page" && t.url === origin + "/",
);
assert.ok(target);
const browser = {
mode: "connect",
cdp_endpoint: endpoint,
target_id: target.id,
};
const declared = [
{ secret_id: "email", environment_variable: "REA_TEST_EMAIL" },
{ secret_id: "password", environment_variable: "REA_TEST_PASSWORD" },
];
const scenario = (actions, secrets = declared) => ({
browser,
start_url: { url: origin + "/" },
secrets,
actions,
capture: captureOptions,
detail: "summary",
});
await client.connect(transport);
catalog = await client.listTools();
await writeFile(join(output, "catalog.json"), JSON.stringify(catalog));
for (const temperature of values["source-only"] ? [] : ["cold", "warm"]) {
const saved = values.mode === "after";
const start = performance.now(),
firstCall = calls.length;
const create = {
step_id: "create",
action: "request",
method: "POST",
path: "/api/collections/rea_items/records",
headers: [tokenHeader(saved)],
json: { title: `created-${temperature}`, quantity: 11 },
expected_status: [200, 201],
...(saved
? {
save: [
{ name: "record_id", json_pointer: "/id" },
{ name: "quantity", json_pointer: "/quantity" },
],
}
: {}),
};
const firstActions = [...loginActions(saved), create];
if (saved) firstActions.push(...edits("{saved:record_id}", true));
const first = await invoke(
"capture_browser_scenario",
scenario(firstActions),
);
const firstSummary = first.normalized_result;
assert.ok(
firstSummary.summary.steps.every((step) => step.status === "completed"),
"Initial capture must complete",
);
let final = first;
let id;
let legacyPreReadObserved;
if (!saved) {
const selected = await invoke("inspect_analysis_view", {
source: {
kind: "retained-evidence",
evidence_id: firstSummary.parent_evidence_id,
},
view: {
kind: "browser-scenario",
collection: "events",
step_indices: [firstActions.length],
event_kinds: ["network-content"],
limit: 16,
},
});
assert.equal(selected.normalized_result.coverage.exhausted, true);
const records = decodeBodies(selected.normalized_result).filter(
(b) => b?.collectionName === "rea_items" && typeof b.id === "string",
);
assert.equal(records.length, 1);
id = records[0].id;
final = await invoke(
"capture_browser_scenario",
scenario(edits(id, false), []),
);
}
assert.ok(
final.normalized_result.summary.steps.every(
(step) => step.status === "completed",
),
"Completed task must not hide failed actions",
);
const completedMs = performance.now() - start,
taskCalls = calls.length - firstCall;
const path = join(output, `${temperature}-evidence.json`);
await invoke("export_evidence_bundle", { path });
const bundle = JSON.parse(await readFile(path, "utf8"));
const parent = bundle.records.find(
(r) => r.evidence_id === final.normalized_result.parent_evidence_id,
);
assert.ok(parent);
if (!saved) {
const legacy = bundle.records.find(
(record) => record.evidence_id === firstSummary.parent_evidence_id,
);
assert.ok(legacy);
const login = decodeBodies(legacy.normalized_result).filter(
(body) => typeof body?.token === "string",
);
assert.equal(login.length, 1);
const preReadLogin = decodeBodies({
events: legacy.normalized_result.events.items.filter(
(event) => event.step_index < firstActions.length,
),
}).filter((body) => typeof body?.token === "string");
legacyPreReadObserved = preReadLogin.length === 1;
if (legacyPreReadObserved) {
assert.ok(
!preReadLogin[0].token.startsWith("[REDACTED:"),
"An observed pre-read login response must preserve its unmarked token",
);
const later = legacy.normalized_result.events.items.filter(
(event) => event.step_index >= firstActions.length,
);
assert.ok(
!JSON.stringify({
events: later,
bodies: decodeBodies({ events: later }),
}).includes(preReadLogin[0].token),
"Post-read facts must use the current page-value policy",
);
}
}
const recordBodies = decodeBodies(parent.normalized_result).filter(
(b) => b?.collectionName === "rea_items" && typeof b.id === "string",
);
assert.ok(recordBodies.length >= 2);
if (saved) id = recordBodies[0].id;
assert.ok(recordBodies.every((b) => b.id === id));
assert.ok(
recordBodies.some((b) => b.title === `record-${id}` && b.quantity === 11),
);
const read = await fetch(
`${origin}/api/collections/rea_items/records/${id}`,
);
assert.equal(read.status, 200);
const actual = await read.json();
assert.equal(actual.title, `record-${id}`);
assert.equal(actual.quantity, 11);
const requestBodies = decodeBodies(parent.normalized_result).filter(
(b) => b?.title === `record-${id}` && !Object.hasOwn(b, "collectionName"),
);
assert.ok(requestBodies.length > 0);
assert.ok(requestBodies.every((b) => typeof b.quantity === "number"));
if (saved) {
const loginBodies = decodeBodies(parent.normalized_result).filter(
(b) => typeof b?.token === "string",
);
assert.equal(loginBodies.length, 1);
assert.ok(loginBodies[0].token.startsWith("[REDACTED:"));
}
runs.push({
temperature,
legacy_pre_read_observed: legacyPreReadObserved,
elapsed_ms: completedMs,
task_calls: taskCalls,
evidence_id: parent.evidence_id,
record_id: id,
quantity_type: typeof actual.quantity,
});
}
if (values["source-only"]) {
assert.equal(values.mode, "after");
const sourceScenario = scenario(
[
{
step_id: "source",
action: "request",
method: "GET",
path: "/escaped-values.json?credential=source-token",
save: [
{ name: "token", json_pointer: "/token", secret: true },
{ name: "code", json_pointer: "/code", secret: true },
],
},
{
step_id: "later",
action: "request",
method: "GET",
path: "/escaped-values.json",
},
],
[],
);
const captured = await invoke("capture_browser_scenario", sourceScenario);
assert.ok(
captured.normalized_result.summary.steps.every(
(step) => step.status === "completed",
),
);
const path = join(output, "source-evidence.json");
await invoke("export_evidence_bundle", { path });
const bundle = JSON.parse(await readFile(path, "utf8"));
const parent = bundle.records.find(
(record) =>
record.evidence_id === captured.normalized_result.parent_evidence_id,
);
assert.ok(parent);
const sourceRequests = parent.normalized_result.events.items.filter(
(event) =>
event.kind === "request" &&
event.step_index === 1 &&
event.url.url.includes("/escaped-values.json"),
);
assert.equal(sourceRequests.length, 1);
const transaction = sourceRequests[0].transaction_id;
const sourceEvents = parent.normalized_result.events.items.filter(
(event) => event.transaction_id === transaction,
);
assert.ok(sourceEvents.some((event) => event.kind === "response"));
assert.ok(
!JSON.stringify(sourceEvents).includes("source-token"),
"Source transaction metadata must not retain the value learned from its response",
);
const bodies = decodeBodies(parent.normalized_result).filter(
(body) =>
Object.hasOwn(body ?? {}, "token") && Object.hasOwn(body, "code"),
);
assert.equal(
bodies.length,
2,
"Both actual source and later response must be observed",
);
for (const body of bodies) {
assert.equal(body.token, "[REDACTED:page:source.token]");
assert.equal(body.code, "[REDACTED:page:source.code]");
}
const expected = Buffer.from(
'{"token":"[REDACTED:page:source.token]","code":"[REDACTED:page:source.code]","unknown":1e-400}',
).toString("base64");
assert.equal(
JSON.stringify(parent.normalized_result).split(JSON.stringify(expected))
.length - 1,
2,
"Both source bodies must preserve the unmarked numeric spelling exactly",
);
const metadata = await invoke("capture_browser_scenario", {
...sourceScenario,
capture: {
...sourceScenario.capture,
network: {
request_body: false,
response_body: false,
header_values: false,
},
},
});
assert.ok(
metadata.normalized_result.summary.steps.every(
(step) => step.status === "completed",
),
);
const metadataPath = join(output, "metadata-evidence.json");
await invoke("export_evidence_bundle", { path: metadataPath });
const metadataBundle = JSON.parse(await readFile(metadataPath, "utf8"));
const metadataParent = metadataBundle.records.find(
(record) =>
record.evidence_id === metadata.normalized_result.parent_evidence_id,
);
assert.ok(metadataParent);
const metadataRequest = metadataParent.normalized_result.events.items.find(
(event) =>
event.kind === "request" &&
event.step_index === 1 &&
event.url.url.includes("/escaped-values.json"),
);
assert.ok(metadataRequest);
assert.equal(metadataRequest.url.origin, origin);
assert.ok(metadataRequest.url.url.includes("/escaped-values.json"));
assert.ok(!metadataRequest.url.url.includes("source-token"));
assert.ok(
!metadataParent.normalized_result.events.items.some(
(event) => event.kind === "network-content",
),
);
const countUrl = `${origin}/api/collections/rea_items/records?perPage=1`;
const beforeCount = (await (await fetch(countUrl)).json()).totalItems;
assert.ok(Number.isSafeInteger(beforeCount));
const whole = await invoke("capture_browser_scenario", {
...sourceScenario,
start_url: { url: origin + "/whole-json.html" },
secrets: [
{
secret_id: "whole_json",
environment_variable: "REA_TEST_WHOLE_JSON",
},
],
storage: {
local_storage: [
{
origin,
entries: [
{
name: "whole_json",
value: { source: "secret", secret_id: "whole_json" },
},
],
},
],
},
actions: [
{
step_id: "settled",
action: "wait_for",
locator: { kind: "css", selector: '#result[data-ready="yes"]' },
state: "visible",
timeout_ms: 10000,
},
],
});
assert.ok(
whole.normalized_result.summary.steps.every(
(step) => step.status === "completed",
),
);
const wholePath = join(output, "whole-json-evidence.json");
await invoke("export_evidence_bundle", { path: wholePath });
const wholeBundle = JSON.parse(await readFile(wholePath, "utf8"));
const wholeParent = wholeBundle.records.find(
(record) =>
record.evidence_id === whole.normalized_result.parent_evidence_id,
);
assert.ok(wholeParent);
const post = wholeParent.normalized_result.events.items.find(
(event) => event.kind === "request" && event.method === "POST",
);
assert.ok(post);
const postEvents = wholeParent.normalized_result.events.items.filter(
(event) => event.transaction_id === post.transaction_id,
);
assert.ok(
postEvents.some(
(event) =>
event.kind === "network-content" &&
event.phase === "request" &&
event.body.state === "unavailable",
),
);
const postResponse = postEvents.find((event) => event.kind === "response");
assert.ok(postResponse && [400, 401, 403].includes(postResponse.status));
assert.ok(!JSON.stringify(wholeParent).includes("private-token-729d"));
assert.equal(
(await (await fetch(countUrl)).json()).totalItems,
beforeCount,
);
const followup = await invoke("capture_browser_scenario", {
...sourceScenario,
start_url: { url: origin + "/immediate-followup.html" },
capture: {
...sourceScenario.capture,
events: [...sourceScenario.capture.events, "console"],
after_each_step: [
...sourceScenario.capture.after_each_step,
"screenshot",
],
at_end: [...sourceScenario.capture.at_end, "screenshot"],
},
actions: [
{
step_id: "exchange",
action: "click",
locator: { kind: "css", selector: "#run" },
save_response: {
method: "GET",
path: "/escaped-values.json",
save: [{ name: "token", json_pointer: "/token", secret: true }],
},
timeout_ms: 10000,
},
{
step_id: "settled",
action: "wait_for",
locator: { kind: "css", selector: '#result[data-ready="yes"]' },
state: "visible",
timeout_ms: 10000,
},
],
});
const followupPath = join(output, "followup-evidence.json");
await invoke("export_evidence_bundle", { path: followupPath });
const followupBundle = JSON.parse(await readFile(followupPath, "utf8"));
const followupParent = followupBundle.records.find(
(record) =>
record.evidence_id === followup.normalized_result.parent_evidence_id,
);
assert.ok(followupParent);
assert.equal(await page.locator("#sensitive").innerText(), "source-token");
assert.equal(followupParent.normalized_result.steps.length, 3);
assert.ok(
followupParent.normalized_result.steps.every(
(step) =>
step.artifacts.screenshot.state === "missing" &&
step.completeness.missing_sections.includes("screenshot"),
),
);
assert.ok(
followupParent.normalized_result.completeness.missing_sections.includes(
"screenshot",
),
);
const consoleEvents = followupParent.normalized_result.events.items.filter(
(event) => event.kind === "console" && event.text.includes("REDACTED"),
);
assert.equal(consoleEvents.length, 1);
assert.ok(
!JSON.stringify(followupParent.normalized_result.events.items).includes(
"source-token",
),
);
assert.ok(
followup.normalized_result.summary.steps.every(
(step) => step.status === "completed",
),
);
const followupPost = followupParent.normalized_result.events.items.find(
(event) => event.kind === "request" && event.method === "POST",
);
assert.ok(followupPost);
const followupEvents = followupParent.normalized_result.events.items.filter(
(event) => event.transaction_id === followupPost.transaction_id,
);
assert.ok(!JSON.stringify(followupEvents).includes("source-token"));
const followupBodies = decodeBodies({ events: followupEvents });
assert.ok(
followupBodies.some(
(body) => body?.token === "[REDACTED:page:exchange.token]",
),
);
assert.ok(
followupEvents.some(
(event) => event.kind === "response" && event.status === 403,
),
);
assert.equal(
(await (await fetch(countUrl)).json()).totalItems,
beforeCount,
);
const publicScreenshot = await invoke("capture_browser_scenario", {
...sourceScenario,
start_url: { url: origin + "/immediate-followup.html" },
actions: [
{ step_id: "settle", action: "wait_for_timeout", duration_ms: 1 },
],
capture: {
...sourceScenario.capture,
events: [],
network: {
request_body: false,
response_body: false,
header_values: false,
},
after_each_step: [],
at_end: ["screenshot"],
},
});
const publicPath = join(output, "public-screenshot-evidence.json");
await invoke("export_evidence_bundle", { path: publicPath });
const publicBundle = JSON.parse(await readFile(publicPath, "utf8"));
const publicParent = publicBundle.records.find(
(record) =>
record.evidence_id ===
publicScreenshot.normalized_result.parent_evidence_id,
);
assert.ok(publicParent);
assert.equal(await page.locator("#sensitive").innerText(), "");
const png =
publicParent.normalized_result.steps.at(-1).artifacts.screenshot;
assert.equal(png.state, "captured");
assert.equal(
Buffer.from(png.value.data_base64, "base64")
.subarray(0, 8)
.toString("hex"),
"89504e470d0a1a0a",
);
runs.push({
verification: "source-and-later-JSON-spellings",
verified_bodies: bodies.length,
metadata_only_source_url: true,
whole_json_secret_withheld: true,
immediate_followup_protected: true,
immediate_console_protected: true,
rendered_secret_screenshots_withheld: true,
nonsecret_screenshot_captured: true,
whole_json_status: postResponse.status,
});
}
await writeFile(join(output, "calls.json"), JSON.stringify(calls), {
mode: 0o600,
});
await writeFile(
join(output, "metrics.json"),
JSON.stringify({ mode: values.mode, runs }, null, 2),
);
console.log(JSON.stringify({ mode: values.mode, runs }));
} finally {
await withVerifierCleanup(
() => commands.close(),
[
async () => {
if (commands.cleanupResources.length === 0)
await rm(profile, { recursive: true, force: true });
},
],
);
}
@@ -1,63 +0,0 @@
# Browser scenario summary verification
The verifier runs the actual compiled CLI/MCP and caller-selected Chromium against
TodoMVC's JavaScript ES5 distribution served on loopback. Ten actions add five
todos. Every step captures URL, DOM, accessibility and a screenshot; the oracle
checks all completed steps, the five authored labels, retained events and
transaction identities. Summary runs also read step, text and transaction pages
and authenticate an exported complete parent.
Download the public distribution at the pinned revision into a disposable folder:
```bash
export REA_TODOMVC_APP="$(mktemp -d)"
python3 - "$REA_TODOMVC_APP" <<'PY'
import hashlib, json, sys, urllib.request
from pathlib import Path
root = Path(sys.argv[1])
revision = "ff43b02e59dfa604386bb382034b2cd07c2bcd8a"
files = ["index.html", "app.js", "base.css", "base.js", "controller.js",
"helpers.js", "index.css", "model.js", "store.js", "template.js", "view.js"]
manifest = []
for name in files:
url = f"https://raw.githubusercontent.com/tastejs/todomvc/{revision}/examples/javascript-es5/dist/{name}"
with urllib.request.urlopen(url, timeout=30) as reply:
data = reply.read()
(root / name).write_bytes(data)
manifest.append({"path": name, "url": url, "bytes": len(data),
"sha256": hashlib.sha256(data).hexdigest()})
(root / "provenance.json").write_text(json.dumps({"revision": revision, "files": manifest}, indent=2))
PY
```
Build each selected checkout with the repository's pinned Node/npm toolchain.
Use new output directories; exports never overwrite an earlier run:
```bash
node experiments/analysis-efficiency/browser-scenario-summary.mjs \
--repository /absolute/path/to/baseline --mode complete \
--application "$REA_TODOMVC_APP" --browser /absolute/path/to/chromium \
--output /absolute/path/to/new-baseline-report
node experiments/analysis-efficiency/browser-scenario-summary.mjs \
--mode summary --application "$REA_TODOMVC_APP" \
--browser /absolute/path/to/chromium \
--output /absolute/path/to/new-summary-report
```
`cold` is the first capture in a fresh MCP process; `warm` repeats the capture in
that process with a new owned browser/profile. Neither label establishes cold OS
file caches. Summary mode intentionally performs additional retained queries.
Compare equivalent answers and include those calls when reporting task cost;
smaller replies alone do not establish a faster task.
Add `--verify-boundaries` to a separate summary run for a real failed-step focus,
CLI summary publication, and MCP text readback from the CLI's complete bundle.
Those extra checks make its whole-run resource cost unsuitable for comparison
against a complete-only run.
Reports retain the actual catalog, arguments, MCP text and complete Evidence.
When estimating tokens, name the tokenizer and count discovery, arguments and
all replies needed for the chosen task. Raw serialized catalog estimates are
not actual model usage or a measurement of a client's rendered tool definitions.
Measure CPU and peak process-family memory externally, state the accounting
method, and include cleanup; the verifier's elapsed phase excludes final teardown.
@@ -1,412 +0,0 @@
import assert from "node:assert/strict";
import { createServer } from "node:http";
import { readFile, writeFile, mkdir, readdir } from "node:fs/promises";
import { resolve, join, extname } from "node:path";
import { pathToFileURL, fileURLToPath } from "node:url";
import { createRequire } from "node:module";
import { performance } from "node:perf_hooks";
import { execFile } from "node:child_process";
import { promisify, parseArgs } from "node:util";
// A developer verifier: the caller selects the pinned TodoMVC distribution,
// browser executable, output directory and baseline/final repository checkout.
const { values } = parseArgs({
options: {
repository: {
type: "string",
default: fileURLToPath(new URL("../../", import.meta.url)),
},
mode: { type: "string", default: "summary" },
application: { type: "string" },
browser: { type: "string" },
output: { type: "string" },
port: { type: "string", default: "0" },
"verify-boundaries": { type: "boolean", default: false },
},
});
assert.ok(
values.application && values.browser && values.output,
"Pass --application <TodoMVC javascript-es5 dist> --browser <executable> --output <new directory>",
);
const repo = resolve(values.repository);
const mode = values.mode;
const output = resolve(values.output);
const appRoot = resolve(values.application);
const executablePath = resolve(values.browser);
assert.ok(["complete", "summary"].includes(mode));
assert.ok(
!values["verify-boundaries"] || mode === "summary",
"--verify-boundaries requires summary mode",
);
process.chdir(repo);
const require = createRequire(join(repo, "package.json"));
const { Client } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client")).href
);
const { StdioClientTransport } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio")).href
);
await mkdir(output);
const files = await readdir(appRoot);
const server = createServer(async (request, response) => {
try {
const name =
decodeURIComponent(
new URL(request.url, "http://127.0.0.1").pathname,
).slice(1) || "index.html";
assert.ok(files.includes(name));
const data = await readFile(join(appRoot, name));
response.setHeader(
"Content-Type",
{ ".html": "text/html", ".js": "text/javascript", ".css": "text/css" }[
extname(name)
] ?? "application/octet-stream",
);
response.end(data);
} catch {
response.writeHead(404);
response.end();
}
});
await new Promise((done, reject) => {
server.once("error", reject);
server.listen(Number(values.port), "127.0.0.1", done);
});
const address = server.address();
assert.ok(address && typeof address === "object");
const origin = `http://127.0.0.1:${address.port}`;
const environment = {
PATH: process.env.PATH,
NODE_OPTIONS: "--max-old-space-size=3072",
UV_THREADPOOL_SIZE: "2",
REA_LOG_LEVEL: "silent",
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
};
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(repo, "scripts/rea.mjs"), "mcp"],
cwd: repo,
env: environment,
stderr: "pipe",
});
let stderr = "";
transport.stderr?.on("data", (chunk) => {
if (stderr.length < 65536) stderr += chunk;
});
const client = new Client({
name: "rea-browser-scenario-summary-verifier",
version: "1",
});
const calls = [];
const invoke = async (name, args, tools) => {
const toolDefinition = tools.find((tool) => tool.name === name);
assert.ok(
toolDefinition?.outputSchema,
`Missing named output schema: ${name}`,
);
const started = performance.now();
const result = await client.callTool(
{ name, arguments: args },
{ toolDefinition, timeout: 90000 },
);
calls.push({
name,
arguments: args,
content: result.content,
elapsed_ms: performance.now() - started,
});
assert.notEqual(result.isError, true, JSON.stringify(result));
return result.structuredContent;
};
const allStarted = performance.now();
try {
await client.connect(transport);
const discoveryStarted = performance.now();
const catalog = await client.listTools();
const discoveryMs = performance.now() - discoveryStarted;
await writeFile(join(output, "catalog.json"), JSON.stringify(catalog));
const actions = Array.from({ length: 5 }, (_, index) => [
{
step_id: `fill_${index}`,
action: "fill",
locator: { kind: "css", selector: ".new-todo" },
value: { source: "literal", value: `Review evidence ${index + 1}` },
timeout_ms: 10000,
},
{
step_id: `add_${index}`,
action: "press",
locator: { kind: "css", selector: ".new-todo" },
key: "Enter",
timeout_ms: 10000,
},
]).flat();
const scenario = {
browser: {
mode: "launch",
executable_path: executablePath,
headless: true,
},
start_url: { url: origin, query: [] },
actions,
capture: {
after_each_step: ["url", "dom", "accessibility", "screenshot"],
at_end: ["url", "dom", "accessibility"],
events: ["network"],
},
};
const runs = [];
for (const temperature of ["cold", "warm"]) {
const callStart = calls.length;
const result = await invoke(
"capture_browser_scenario",
{ ...scenario, ...(mode === "summary" ? { detail: "summary" } : {}) },
catalog.tools,
);
const parentId =
mode === "summary"
? result.normalized_result.parent_evidence_id
: result.evidence_id;
const bundlePath = join(output, `${temperature}-evidence.json`);
await invoke("export_evidence_bundle", { path: bundlePath }, catalog.tools);
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
const parent = bundle.records.find((row) => row.evidence_id === parentId);
assert.ok(parent);
const capture = parent.normalized_result;
assert.equal(capture.steps.length, 11);
assert.ok(capture.steps.every((step) => step.status === "completed"));
assert.equal(capture.browser.cleanup, "terminated-owned-process");
const final = capture.steps.at(-1).artifacts.accessibility;
assert.equal(final.state, "captured");
for (let index = 1; index <= 5; index++)
assert.ok(final.value.text.includes(`Review evidence ${index}`));
assert.ok(final.value.text.includes('strong: "5"'));
assert.ok(final.value.text.includes("text: items left"));
const transactionIds = new Set(
capture.events.items
.filter((event) => "transaction_id" in event)
.map((event) => event.transaction_id),
);
const details = {
temperature,
parentId,
steps: capture.steps.length,
retained_events: capture.events.retained,
transactions: transactionIds.size,
};
if (mode === "summary") {
const summary = result.normalized_result.summary;
assert.equal(summary.steps.length, 11);
assert.equal(
summary.final_accessibility.accessibility.value.text,
final.value.text,
);
assert.equal(summary.events.retained, capture.events.retained);
assert.equal(summary.limitation_count, parent.limitations.length);
assert.equal(summary.network.transactions, transactionIds.size);
const source = { kind: "retained-evidence", evidence_id: parentId };
const pagingStarted = performance.now();
for (const stepIndex of [1, 5, 10]) {
const page = await invoke(
"inspect_analysis_view",
{
source,
view: {
kind: "browser-scenario",
collection: "steps",
step_indices: [stepIndex],
artifacts: [],
limit: 1,
},
},
catalog.tools,
);
assert.equal(
page.normalized_result.items.rows[0].step_id,
capture.steps[stepIndex].step_id,
);
}
details.warm_page_total_ms = performance.now() - pagingStarted;
const text = await invoke(
"inspect_analysis_view",
{
source,
view: {
kind: "browser-scenario",
collection: "text",
step_index: 10,
artifact: "accessibility",
offset: 0,
limit: 32,
},
},
catalog.tools,
);
assert.equal(
text.normalized_result.items.text,
final.value.text.slice(0, 32),
);
const tx = await invoke(
"inspect_analysis_view",
{
source,
view: {
kind: "browser-scenario",
collection: "transactions",
limit: 2,
},
},
catalog.tools,
);
assert.deepEqual(
tx.normalized_result.items.rows.map((row) => row.transaction_id),
[...transactionIds].slice(0, 2),
);
details.returned_summary = summary;
}
details.calls = calls.slice(callStart);
runs.push(details);
}
const boundaryChecks = [];
if (values["verify-boundaries"]) {
const failed = await invoke(
"capture_browser_scenario",
{
...scenario,
actions: [
{
step_id: "missing",
action: "click",
locator: {
kind: "css",
selector: "#rea-control-that-does-not-exist",
},
timeout_ms: 100,
},
],
capture: {
after_each_step: ["url", "accessibility"],
at_end: ["url", "accessibility"],
events: ["network"],
},
detail: "summary",
},
catalog.tools,
);
assert.equal(failed.normalized_result.summary.steps[1].status, "failed");
assert.equal(
failed.normalized_result.summary.final_accessibility.focus_step_index,
1,
);
assert.equal(
failed.normalized_result.summary.completeness.equality_eligible,
false,
);
boundaryChecks.push("real failed-step focus and incomplete coverage");
const execute = promisify(execFile);
const bundlePath = join(output, "cli-evidence.json");
const { stdout } = await execute(
process.execPath,
[
join(repo, "scripts/rea.mjs"),
"capture-browser-scenario",
JSON.stringify(scenario),
"--detail",
"summary",
"--evidence_output",
bundlePath,
"--json",
],
{
cwd: repo,
env: environment,
maxBuffer: 64 * 1024 * 1024,
timeout: 90000,
},
);
const cli = JSON.parse(stdout);
const parentId = cli.summary.normalized_result.parent_evidence_id;
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
const parent = bundle.records.find((row) => row.evidence_id === parentId);
assert.ok(parent);
assert.equal(parent.normalized_result.steps.length, 11);
assert.ok(
parent.normalized_result.steps.every(
(step) => step.status === "completed",
),
);
assert.ok(!("detail" in parent.parameters));
assert.equal(
cli.summary.normalized_result.summary.final_accessibility.accessibility
.value.text,
parent.normalized_result.steps.at(-1).artifacts.accessibility.value.text,
);
const text = await invoke(
"inspect_analysis_view",
{
source: {
kind: "evidence-file",
path: bundlePath,
evidence_id: parentId,
},
view: {
kind: "browser-scenario",
collection: "text",
step_index: 10,
artifact: "accessibility",
offset: 0,
limit: 32,
},
},
catalog.tools,
);
assert.equal(
text.normalized_result.items.text,
parent.normalized_result.steps
.at(-1)
.artifacts.accessibility.value.text.slice(0, 32),
);
boundaryChecks.push(
"real CLI summary, complete bundle and MCP evidence-file readback",
);
}
await writeFile(
join(output, "result.json"),
JSON.stringify(
{
repo,
mode,
application: appRoot,
browser: executablePath,
input_paths: {
repository: values.repository,
application: values.application,
browser: values.browser,
output: values.output,
},
origin,
discovery_ms: discoveryMs,
elapsed_ms: performance.now() - allStarted,
runs,
boundaryChecks,
stderr,
},
null,
2,
),
);
console.log(
JSON.stringify({
mode,
discovery_ms: discoveryMs,
runs: runs.map(
({ calls: _calls, returned_summary: _summary, ...rest }) => rest,
),
boundaryChecks,
elapsed_ms: performance.now() - allStarted,
}),
);
} finally {
await client.close();
server.closeAllConnections();
await new Promise((done) => server.close(done));
}
@@ -1,94 +0,0 @@
"""Verify 18 completed entropy tasks and report whole-task resource/token costs."""
import argparse
from pathlib import Path
import hashlib, json, statistics
import tiktoken
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--receipts', type=Path, required=True)
parser.add_argument('--output', type=Path, required=True)
options = parser.parse_args()
b = options.receipts
cohort = b / 'cohort'
index = json.loads((cohort / 'index.json').read_text())
assert len(index) == 18 and all((r['exit'] == 0 for r in index))
assert len({item['name'] for item in index}) == 18
enc = tiktoken.get_encoding('cl100k_base')
compact = lambda d: json.dumps(d, ensure_ascii=False, separators=(',', ':'), allow_nan=False)
count = lambda s: len(enc.encode(s, disallowed_special=()))
digest = lambda p: hashlib.sha256(p.read_bytes()).hexdigest()
rows = []
for item in index:
p = cohort / item['name']
r = json.loads((p / 'report.json').read_text())
catalog = json.loads((p / 'catalog.json').read_text())
tx = json.loads((p / 'transcript.json').read_text())
scope = next((json.loads(l) for l in reversed((cohort / (item['name'] + '.log')).read_text().splitlines()) if l.startswith('{"resource_scope":')))
obs = scope['payload_resource_observation']
events = obs['resource_events']
assert r['oracle_sha256'] == digest(b / f"{item['case']}-oracle.json")
assert item['exit'] == 0 and r['status'] == 'passed' and r['complete_source_export_verified'] and (not r['cleanup_failures']) and (not r['receipt_failures'])
assert scope['abort'] is None and scope['payload_resource_observation_verified'] and (obs['payload_exit_code'] == 0) and obs['payload_cleanup_verified']
assert all((events['memory.events'][k] == 0 for k in ['high', 'max', 'oom', 'oom_kill'])) and events['pids.events']['max'] == 0
assert [e['name'] for e in tx] == ['inspect_byte_entropy', 'export_evidence_bundle']
assert all((not e['reply'].get('isError', False) for e in tx))
inp = count(compact([{k: t[k] for k in ['name', 'description', 'inputSchema']} for t in catalog['tools']]))
args = sum((count(compact({'name': e['name'], 'arguments': e['input']})) for e in tx))
replies = sum((count(c['text']) for e in tx for c in e['reply']['content'] if c['type'] == 'text'))
prompt = count(r['question'])
instructions = count(r['server_instructions'])
rows.append({
**item,
'catalog_tools': len(catalog['tools']),
'catalog_sha256': digest(p / 'catalog.json'),
'parent_evidence_id': r['parent_evidence_id'],
'source_report_sha256': r['source_report_sha256'],
'runtime_identity_sha256': r['runtime_identity_sha256'],
'wall_seconds': scope['elapsed_seconds'],
'cpu_seconds': obs['cpu_stat']['usage_usec'] / 1000000.0,
'kernel_family_peak_bytes': obs['memory_peak_bytes'],
'inspection_ms': r['calls'][0]['elapsed_ms'],
'discovery_ms': r['phases']['discovery_ms'],
'token_estimate': {
'question': prompt,
'instructions': instructions,
'directory_once': inp,
'arguments': args,
'text_replies': replies,
'whole_task': prompt + instructions + inp + args + replies,
'raw_catalog': count(compact(catalog)),
},
'receipt_sha256': {n: digest(p / n) for n in ['report.json', 'transcript.json', 'catalog.json', 'source.evidence.json']},
'payload_exit_events': events,
'payload_cleanup_verified': True,
'whole_scope_events_final': scope['resource_events_final'],
})
summary = {}
for case in ['llvm-small-block', 'llvm-large-block', 'ls']:
rs = [r for r in rows if r['case'] == case]
for k in ['catalog_sha256', 'source_report_sha256', 'parent_evidence_id']:
assert len({r[k] for r in rs}) == 1, (case, k)
summary[case] = {}
for lane in ['before', 'after']:
ls = [r for r in rs if r['lane'] == lane]
assert len(ls) == 3
assert {r['repeat'] for r in ls} == {1, 2, 3}
metrics = {k: [r[k] for r in ls] for k in ['wall_seconds', 'cpu_seconds', 'kernel_family_peak_bytes', 'inspection_ms', 'discovery_ms']}
metrics['whole_task_tokens'] = [r['token_estimate']['whole_task'] for r in ls]
summary[case][lane] = {k: {'median': statistics.median(v), 'min': min(v), 'max': max(v)} for k, v in metrics.items()}
identities = {lane: json.loads((b / f'{lane}-runtime-identity.json').read_text()) for lane in ['before', 'after']}
assert all(identity['source_clean'] for identity in identities.values())
for row in rows:
assert row['runtime_identity_sha256'] == digest(b / f"{row['lane']}-runtime-identity.json")
result = {
'tiktoken': tiktoken.__version__,
'encoding': enc.name,
'actual_model_usage': None,
'revisions': {l: i['source_revision'] for l, i in identities.items()},
'identity_receipts': {l: digest(b / f'{l}-runtime-identity.json') for l in identities},
'oracles': {c: {'sha256': digest(b / f'{c}-oracle.json'), 'artifact': json.loads((b / f'{c}-oracle.json').read_text())['expected']['artifact']} for c in summary},
'summary': summary,
'recordings': rows,
'measurement': 'Three alternating fresh production stdio SDK sessions per lane/case; exact discovered toolDefinition/default validator, fresh HOME, Node compile cache disabled, OS caches unflushed. Each task discovers complete catalog, inspects bytes and exports/compares full source Evidence; all oracle facts checked. Whole process-family CPU and kernel peak include client/server, runtime fingerprint verification, serialization, export and cleanup. Oracle generation and tokenization outside measured tasks. One-CPU quota, 3GiB early stop/4GiB hard/no swap; payload exit/cleanup/events verified, final whole-scope counters unavailable. Token serialization estimate includes question/instructions/all input name+description+schema once/all arguments/all text replies, raw catalog separately. Host framing, replay, model inference quality and billing unmeasured.',
}
options.output.write_text(json.dumps(result, indent=2) + '\n')
print(json.dumps(summary, indent=2))
@@ -1,56 +0,0 @@
"""Independent Python histogram oracle; reads artifact bytes without execution."""
import argparse
from collections import Counter
import hashlib
import json
import math
from pathlib import Path
def entropy(counts, size):
if not size:
return 0
value = -sum((counts[k] / size) * math.log2(counts[k] / size) for k in sorted(counts) if counts[k])
return math.floor(value * 10000 + 0.5) / 10000
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--path', required=True)
parser.add_argument('--output', required=True)
parser.add_argument('--block-size', type=int, required=True)
parser.add_argument('--offset', type=int, default=0)
parser.add_argument('--length', type=int)
args = parser.parse_args()
path = Path(args.path)
before = path.stat()
counts = Counter()
digest = hashlib.sha256()
with path.open('rb') as source:
while chunk := source.read(1024 * 1024):
digest.update(chunk)
counts.update(chunk)
length = args.length if args.length is not None else before.st_size - args.offset
assert 0 <= args.offset <= before.st_size and 0 <= length <= before.st_size - args.offset
blocks, runs = [], []
source.seek(args.offset)
for offset in range(args.offset, args.offset + length, args.block_size):
chunk = source.read(min(args.block_size, args.offset + length - offset))
value = entropy(Counter(chunk), len(chunk))
block = {'offset': offset, 'length': len(chunk), 'entropy': value}
blocks.append(block)
classification = 'high' if value >= 7.2 else 'low' if value <= 1 else None
if classification is not None:
if runs and runs[-1]['classification'] == classification and runs[-1]['offset'] + runs[-1]['length'] == offset:
runs[-1]['length'] += len(chunk)
else:
runs.append({'offset': offset, 'length': len(chunk), 'classification': classification})
after = path.stat()
assert (before.st_dev, before.st_ino, before.st_size, before.st_mtime_ns, before.st_ctime_ns) == (after.st_dev, after.st_ino, after.st_size, after.st_mtime_ns, after.st_ctime_ns)
expected = {'artifact': {'path': args.path, 'bytes': before.st_size, 'sha256': digest.hexdigest()}, 'block_size':args.block_size,'window':{'offset':args.offset,'length':length},'file_entropy':entropy(counts,before.st_size),'thresholds':{'high':7.2,'low':1},'blocks':blocks,'runs':runs}
request = {'path':args.path,'block_size':args.block_size,'offset':args.offset}
if args.length is not None:
request['length'] = args.length
Path(args.output).write_text(json.dumps({'input':request,'expected':expected},indent=2)+'\n')
print(json.dumps({'path':args.path,'sha256':digest.hexdigest(),'blocks':len(blocks),'runs':len(runs)}))
if __name__ == '__main__':
main()
File diff suppressed because it is too large. Load diff
@@ -1,158 +0,0 @@
# Byte entropy: separate physical reads from report resolution
Issue: [#6947](https://github.com/morluto/rea/issues/6947).
The question is to return the full-file SHA-256 and entropy, every block in a
selected byte window, and adjacent high/low runs, preserving complete source
Evidence. `ByteEntropyService` previously used `block_size` for both reporting
and physical reads, including outside the selected window. Small report blocks
therefore caused hundreds of thousands of asynchronous reads of a large file.
The shared service now fills a bounded 64 KiB buffer before calculating the
caller-selected blocks. Supported power-of-two block sizes divide that buffer;
the window starts its own block sequence, and only its final block can be short.
Short physical reads are joined before interpretation. The same stable-file,
whole-file hash, cancellation, capacity and descriptor-cleanup owners remain.
No new cache, parser, contract or output selection is introduced.
## Fixed comparison
Baseline: `1d896e03b776453282beca7fd89338be778bfca2`.
Measured implementation: `3ed3276d0ee79c291e1006901ace5b06626c1d4e`.
Both were clean and built with Node 24.22.1, npm 11.16.0 and SDK 2.3.1.
The runtime receipt fingerprints 2,582 compiled/catalog/entrypoint/dependency
and driver files in each lane, checked before and after every task.
Real artifacts are installed Linux x64 binaries, inspected without execution:
| Artifact | Bytes | SHA-256 |
| ----------------------------------------------------- | ----------: | ------------------------------------------------------------------ |
| LLVM 16 (`/usr/lib/x86_64-linux-gnu/libLLVM-16.so.1`) | 123,379,936 | `f62d254b7f2bf42df8c8b07d46ee3bb4c2cafeca436b2e6bc6ccbe4581f58f40` |
| GNU ls (`/usr/bin/ls`) | 151,344 | `cb30d69b24245bf2ecdc9e7f53bbad19159999970b6d82c0c00c7d32d9e37aa4` |
`byte-entropy-oracle.py` independently computes Python byte histograms,
Shannon entropy, selected blocks, rounded-value run classifications and digests.
It checks file state around its reads. Every returned fact agrees with this
oracle, and both lanes have identical complete report hashes and Evidence IDs.
There are three alternating fresh production stdio MCP sessions per lane/case,
18 tasks total. Each discovers all 222 tools, uses exact discovered
`toolDefinition` validation, inspects the bytes, exports the complete Evidence
bundle and verifies its parent record. Fresh HOME/client/server, Node compile
cache disabled, OS caches unflushed. These are fresh sessions; no warm-runtime
speedup is claimed. Cached-catalog SDK bulk validation was not measured here.
| Task (3 samples per lane) | Whole elapsed median, s | Family CPU median, s | Kernel family peak median, MiB | Whole-task estimated tokens |
| ----------------------------------------------------------- | ----------------------: | -------------------: | -----------------------------: | --------------------------: |
| LLVM: 256-byte blocks, offset 3, length 65,539 (257 blocks) | 29.842 → 8.584 | 25.286 → 6.358 | 629.5 → 672.4 | 62,000 → 62,000 |
| LLVM: whole file, 65,536-byte blocks (1,883 blocks) | 12.314 → 12.115 | 7.581 → 7.325 | 667.9 → 674.3 | 91,900 → 91,900 |
| ls: whole file, 256-byte blocks (592 blocks) | 7.075 → 8.286 | 5.804 → 5.528 | 659.1 → 659.3 | 67,215 → 67,215 |
The large fine-resolution task improves elapsed median by 71.2% and CPU by
74.9%. Elapsed ranges are 28.541–33.290 → 7.981–10.396 seconds; CPU ranges
23.600–25.429 → 6.049–6.633 seconds. Its kernel peak increases by about
43 MiB (ranges 620.4–637.5 → 665.7–673.2 MiB). This is a speed/CPU improvement
with higher measured peak memory, not a memory optimization.
The other controls do not establish a stable whole-task speedup. The small
file's elapsed median is worse; ranges overlap at 6.469–9.892 → 6.169–9.188
seconds. No general latency or memory benefit is claimed. Full-file reading
and hashing remain necessary even when the selected window is short or empty.
A separate single real-file service probe delegated to actual `open/stat/read`
while counting reads: 481,955 → 1,885 physical reads of all 123,379,936 bytes,
maximum request 256 → 65,536 bytes, service elapsed 17.09 → 1.34 seconds.
The complete Evidence objects were equal and independently checked. This probe
locates the work removed; it is not the repeated public-task performance table.
The [machine-readable report](./byte-entropy-reads.json) preserves ranges,
per-sample phase timings, calls, receipt hashes, artifact identities and commands.
`cl100k_base` via tiktoken 0.14.0 counts the fixed question, server instructions,
all tool names/descriptions/input schemas once, every argument and every text
reply including export. Input-directory cost is 56,685 tokens; raw catalog with
output schemas is separately 838,375. Actual model billing, host framing/replay,
model reasoning quality and inference latency were not measured. No token saving
is claimed: replies and catalog remain unchanged.
## Resource and correctness verification
All tasks ran serially under a one-CPU quota, 3 GiB early memory stop, 4 GiB hard
limit, zero swap and 192-task ceiling. Whole process-family CPU and kernel memory
peak include client/server, runtime fingerprint reads, serialization, source
export and cleanup; kernel peak includes charged file cache. Independent oracle
generation and tokenization are outside timed tasks. Payload exits, remaining
owned processes and memory/PID-limit events were checked: all task payloads
exited 0, cleanup verified, no guard stop/OOM. Final whole-scope counters were
unavailable after transient-scope removal and are not represented as verified.
Executed additional checks:
- Six real CLI calls (three cases in both lanes): every oracle fact, complete
CLI Evidence equality and matching MCP parent identity passed.
- 36 tests in four exact source/public-boundary files passed, including short
reads, unaligned windows across buffers, partial tails at minimum/intermediate/
maximum block sizes, empty windows, cancellation through finalization,
identity changes, capacity refusal, retained summaries and cleanup retry.
- Six serial static tasks (typecheck, lint including module boundaries, format,
knip, metadata and required compilation), plus architecture guards passed.
Static family peak was 1.96 GiB; focused-test/preparation peak was 2.15 GiB.
- An initial typecheck failure in the new short-read test's injected callback
was fixed with an explicitly typed positional boundary. Its failed receipt is
retained, not counted as a successful run. Initial local formatter path and
early profile verification attempts also failed before their intended checks;
corrected checks are recorded separately.
The public experiments qualify static byte inspection on these Linux files.
They do not establish behavior on every filesystem or operating system and do
not execute LLVM or ls. CLI/MCP contracts, Evidence fields, input limits and
provider authority remain unchanged; generated catalogs and builds are ignored.
After integrating main `21a20306b`, the six serial static tasks, architecture
guards and all 36 focused cases passed again. Three fresh real MCP/source-export
tasks and three CLI calls matched every oracle fact and the frozen cohort's
report/parent identity. The integrated test/preparation family peaked at
2.23 GiB with verified payload exit/cleanup and no guard stop/OOM. These are
integration qualification runs; they do not replace the fixed performance pins.
The committed cost verifier also reproduced the complete measurement JSON.
## Reproduction
Build both pinned checkouts using their locked toolchain, serially. Supply an
independent installed dependency tree matching the lock, without upgrading host
software. Use the exact artifact digests above. The oracle accepts explicit
`--path`, `--block-size`, `--offset`, `--length` and `--output`; omit length for
the two whole-file controls. For example:
```sh
python3 experiments/analysis-efficiency/byte-entropy-oracle.py \
--path /usr/lib/x86_64-linux-gnu/libLLVM-16.so.1 \
--block-size 256 --offset 3 --length 65539 --output /absolute/oracle.json
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
node experiments/analysis-efficiency/byte-entropy-reads.mjs \
--runtime /absolute/pinned-checkout --oracle /absolute/oracle.json \
--identity /absolute/runtime-identity.json --output /absolute/new-task-directory
```
Runtime identity JSON contains `source_revision`, `source_clean`,
`node_sha256` and `files: [{path, sha256}]` with absolute paths. Fingerprint the
entire compiled runtime and catalogs, scripts, lock/toolchain files, actual SDK
JavaScript dependencies and the driver/helpers. Both sessions must use the same
Node and dependency contents. Use the same one-CPU/memory policy for the harness;
the historical runs additionally pinned the payload to CPUs 0–1 and the initial
observer to CPU 15. Do not compare an unguarded default-concurrency run as an
equivalent resource measurement.
Retained raw receipts are on the investigation host at
`/tmp/rea-perf-byte-entropy-20261024/`. This includes `cohort/index.json`,
per-task catalog/transcript/report/source export/guard logs, runtime identities,
three oracle JSONs, CLI stdout/stderr, boundary profiles and verification logs.
The cost verifier takes that receipt directory and a separate output:
```sh
python3 experiments/analysis-efficiency/byte-entropy-costs.py \
--receipts /absolute/receipt-directory --output /absolute/costs.json
```
It requires all 18 successful verified samples, matching parent/report/catalog
identity across lanes, successful source exports and oracle/runtime receipts.
Failed or stopped attempts cannot enter the published cohort.
@@ -1,154 +0,0 @@
/** Complete entropy question through production stdio, with full source export. */
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { finished } from "node:stream/promises";
import { parseArgs } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { createRetainedQueryCaller } from "./retained-query-calls.mjs";
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
const { values } = parseArgs({
options: Object.fromEntries(
["runtime", "oracle", "identity", "output"].map((name) => [
name,
{ type: "string" },
]),
),
});
for (const value of Object.values(values)) assert.equal(value, resolve(value));
const sha256 = (value) => createHash("sha256").update(value).digest("hex");
const oracleBytes = await readFile(values.oracle);
const oracle = JSON.parse(oracleBytes);
const identityBytes = await readFile(values.identity);
const identity = JSON.parse(identityBytes);
await mkdir(values.output, { mode: 0o700 });
const account = await mkdtemp(join(tmpdir(), "rea-entropy-task-"));
const client = new Client({ name: "entropy-complete-task", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
maxBufferSize: 10485760,
env: {
HOME: account,
TMPDIR: account,
PATH: process.env.PATH,
REA_LOG_LEVEL: "silent",
NODE_OPTIONS: "--max-old-space-size=2048",
NODE_DISABLE_COMPILE_CACHE: "1",
UV_THREADPOOL_SIZE: "2",
},
stderr: "pipe",
});
const serverLog = createWriteStream(join(values.output, "server.log"));
const serverLogFinished = finished(serverLog);
serverLogFinished.catch(() => {});
transport.stderr?.pipe(serverLog);
const report = {
status: "running",
question:
"Return the file SHA-256, file-wide entropy, every entropy block and adjacent high/low run in the requested byte window; preserve the complete source Evidence.",
request: oracle.input,
calls: [],
phases: {},
client:
"Production stdio SDK, exact discovered toolDefinition and default validator",
runtime_identity_sha256: sha256(identityBytes),
oracle_sha256: sha256(oracleBytes),
driver_sha256: sha256(await readFile(new URL(import.meta.url))),
};
const transcript = [];
let definitions;
const call = createRetainedQueryCaller(
{
callTool(params, options) {
return client.callTool(params, {
...options,
toolDefinition: definitions.get(params.name),
});
},
},
(receipt, event) => {
report.calls.push(receipt);
transcript.push(event);
},
);
const observe = async (name, args, phase) => {
const result = await call(name, args, phase);
assert.notEqual(result.isError, true, JSON.stringify(result));
assert.ok(result.structuredContent);
return result.structuredContent;
};
const verifyRuntime = async () => {
assert.equal(sha256(await readFile(values.oracle)), sha256(oracleBytes));
assert.equal(sha256(await readFile(values.identity)), sha256(identityBytes));
assert.equal(sha256(await readFile(process.execPath)), identity.node_sha256);
assert.equal(
sha256(await readFile(new URL(import.meta.url))),
report.driver_sha256,
);
for (const item of identity.files)
assert.equal(sha256(await readFile(item.path)), item.sha256, item.path);
};
try {
await verifyRuntime();
const start = performance.now();
await client.connect(transport);
report.server_instructions = client.getInstructions() ?? "";
const catalog = await client.listTools();
definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
report.phases.discovery_ms = performance.now() - start;
const parent = await observe("inspect_byte_entropy", oracle.input, "inspect");
for (const [key, value] of Object.entries(oracle.expected))
assert.deepEqual(parent.normalized_result[key], value, key);
report.parent_evidence_id = parent.evidence_id;
report.source_report_sha256 = sha256(
JSON.stringify(parent.normalized_result),
);
report.artifact = parent.normalized_result.artifact;
await observe(
"export_evidence_bundle",
{ path: join(values.output, "source.evidence.json") },
"export",
);
const bundle = JSON.parse(
await readFile(join(values.output, "source.evidence.json")),
);
assert.deepEqual(
bundle.records.find((item) => item.evidence_id === parent.evidence_id),
parent,
);
report.complete_source_export_verified = true;
report.status = "passed";
} catch (cause) {
report.status = "failed";
report.error = String(cause);
} finally {
try {
await verifyRuntime();
} catch (cause) {
report.status = "failed";
report.identity_failure = String(cause);
}
await finalizeRetainedQueryReceipt({
client,
transport,
account,
serverLog,
serverLogFinished,
report,
transcript,
output: values.output,
});
if (
report.status !== "passed" ||
report.cleanup_failures.length ||
report.receipt_failures.length
)
process.exitCode = 1;
}
@@ -1,32 +0,0 @@
"""Count all actual client JSON inputs and preserve bounded family resource receipts."""
from pathlib import Path
import argparse, json, hashlib
import tiktoken
parser=argparse.ArgumentParser(description=__doc__)
parser.add_argument('--root',type=Path,required=True)
parser.add_argument('--output',type=Path,required=True)
args=parser.parse_args();lab=args.root;enc=tiktoken.get_encoding('cl100k_base')
compact=lambda value:json.dumps(value,ensure_ascii=False,separators=(',',':'),allow_nan=False)
count=lambda value:len(enc.encode(compact(value),disallowed_special=()))
def resources(root):
scope=json.loads(Path(str(root)+'.log').read_text().splitlines()[-1]);o=scope['payload_resource_observation']
assert scope['abort'] is None and scope['payload_resource_observation_verified'] and o['payload_exit_code']==0 and o['payload_cleanup_verified']
for event in o['resource_events'].values():assert not any(event.values())
return {'wall_seconds':scope['elapsed_seconds'],'cpu_seconds':o['cpu_stat']['usage_usec']/1e6,'family_peak_mib':o['memory_peak_bytes']/1048576,'payload_cleanup_verified':True,'final_whole_scope_counters_verified':scope['resource_counters_verified']}
wire=[];sdk=[]
for root in sorted(lab.glob('final-wire-*')):
if not root.is_dir():continue
report=json.loads((root/'report.json').read_text());requests=json.loads((root/'model-requests.json').read_text());assert report['status'] in ['passed','expected-incomplete-task'] and not report['cleanup_failures'] and report['appserver_exit_code']==0 and report['owned_scope_drain']['remaining_processes']==[]
rows=[]
for index,captured in enumerate(requests):
body=captured['body'];namespace=[n for n in body.get('tools',[]) if n.get('type')=='namespace' and n.get('name')=='mcp__rea_audit'];compaction=not namespace
if compaction:assert 'CONTEXT CHECKPOINT COMPACTION' in compact(body['input'][-1])
else:assert len(namespace)==1 and len(namespace[0]['tools'])==210
rows.append({'index':index+1,'kind':'compaction' if compaction else 'normal','whole_json_tokens_proxy':count(body),'rea_namespace_tokens_proxy':0 if not namespace else count(namespace[0]),'rea_function_count':0 if not namespace else len(namespace[0]['tools'])})
wire.append({'case':root.name,'task_completed':report['task_completed'],'mode':report['mode'],'budget':report['observations']['tool_output_budget'],'requests':len(rows),'compactions':sum(r['kind']=='compaction' for r in rows),'whole_json_tokens_proxy':sum(r['whole_json_tokens_proxy'] for r in rows),'request_details':rows,**resources(root)})
for root in sorted(lab.glob('final-sdk-*')):
if not root.is_dir():continue
report=json.loads((root/'report.json').read_text());assert report['status']=='passed' and not report['cleanup_failures'] and not report['receipt_failures']
sdk.append({'case':root.name,'provider':report['provider'],'symbol':report['symbol'],'detail':report['detail'],'warm':report['warm'],'tool_calls':len(report['calls']),'exported_bundle_bytes':(root/'evidence.json').stat().st_size,'disassembly_calls':[r for r in report['calls'] if r['phase'].startswith('disassembly:')],**resources(root)})
receipt={'measurement_scope':'Actual public SDK and Codex0.163 app-server with real GDB/BFD. Local deterministic SSE fixture and observed-ID checkpoint; no AI inference, model usage/quality/acceptance or billing claim. Whole compact JSON serialization tokens are proxies and include all schemas/replays/compaction. Incomplete default baseline is excluded from completed-workflow gains. Fresh processes have unflushed OS caches; warm SDK cases repeat the selected disassembly in the same server. Native target never executed.','tokenizer':{'package':tiktoken.__version__,'encoding':enc.name},'runtime_pins':{variant:{k:v for k,v in json.loads((lab/(variant+'-manifest.json')).read_text()).items() if k!='files'} for variant in ['baseline','candidate']},'native':json.loads((lab/'native-prerequisite-manifest.json').read_text()),'wire':wire,'sdk':sdk}
args.output.write_text(json.dumps(receipt,indent=2)+'\n');print(json.dumps({'wire':[{k:v for k,v in row.items() if k!='request_details'} for row in wire],'sdk':[{k:v for k,v in row.items() if k!='disassembly_calls'} for row in sdk]},indent=2))
@@ -1,22 +0,0 @@
from pathlib import Path
import struct,hashlib,json,argparse
parser=argparse.ArgumentParser(description='Independent ELF64 little-endian native-command byte oracle');parser.add_argument('--input',type=Path,required=True);parser.add_argument('--output',type=Path,required=True);args=parser.parse_args();path=args.input;b=path.read_bytes();assert b[:6]==b'\x7fELF\x02\x01'
shoff=struct.unpack_from('<Q',b,40)[0];entsize,n,strings=struct.unpack_from('<HHH',b,58)
assert entsize==64 and shoff+n*entsize<=len(b)
sections=[struct.unpack_from('<IIQQQQIIQQ',b,shoff+i*entsize) for i in range(n)]
rows=[]
for si,s in enumerate(sections):
if s[1]!=11:continue
text=sections[s[6]];names=b[text[4]:text[4]+text[5]]
assert s[9]==24 and s[5]%s[9]==0
for ordinal,off in enumerate(range(s[4],s[4]+s[5],24)):
name,info,other,section,value,size=struct.unpack_from('<IBBHQQ',b,off)
end=names.find(b'\0',name);assert end>=name
if names[name:end] not in [b'execute_command',b'execute_command_internal'] or info&15!=2:continue
target=sections[section];file_offset=target[4]+value-target[3]
assert target[4]<=file_offset and file_offset+size<=target[4]+target[5]
rows.append({'symbol_table_section':si,'symbol_ordinal':ordinal,'name':names[name:end].decode('ascii'),'address':value,'bytes':size,'file_offset':file_offset,'first_16_bytes_hex':b[file_offset:file_offset+16].hex()})
assert rows
result={'path':str(path),'artifact_sha256':hashlib.sha256(b).hexdigest(),'artifact_bytes':len(b),'elf_class':64,'byte_order':'little','elf_machine':struct.unpack_from('<H',b,18)[0],'symbols':rows,'scope':'Independent ELF64 little-endian dynamic symbol/section mapping; no target execution or REA parser.'}
args.output.write_text(json.dumps(result,indent=2)+'\n')
print(json.dumps(result))
@@ -1,212 +0,0 @@
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { finished } from "node:stream/promises";
import { pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { createRetainedQueryCaller } from "./retained-query-calls.mjs";
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
const { values } = parseArgs({
options: {
runtime: { type: "string" },
input: { type: "string" },
oracle: { type: "string" },
output: { type: "string" },
provider: { type: "string" },
symbol: { type: "string" },
detail: { type: "string", default: "complete" },
warm: { type: "boolean", default: false },
},
});
for (const key of ["runtime", "input", "oracle", "output"])
assert.equal(values[key], resolve(values[key]));
assert.ok(["gdb", "objdump"].includes(values.provider));
assert.ok(["complete", "summary"].includes(values.detail));
const oracle = JSON.parse(await readFile(values.oracle, "utf8"));
assert.ok(oracle.symbols.some((row) => row.name === values.symbol));
const inputBytes = await readFile(values.input);
assert.equal(
createHash("sha256").update(inputBytes).digest("hex"),
oracle.artifact_sha256,
);
const require = createRequire(join(values.runtime, "package.json"));
const { Client } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client"))
);
const { StdioClientTransport } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio"))
);
await mkdir(values.output, { mode: 0o700 });
const account = await mkdtemp(join(tmpdir(), "rea-command-views-"));
const client = new Client({ name: "command-observation-task", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
env: {
HOME: account,
PATH: process.env.PATH,
NODE_OPTIONS: "--max-old-space-size=2048",
UV_THREADPOOL_SIZE: "1",
REA_LOG_LEVEL: "silent",
},
stderr: "pipe",
});
const serverLog = createWriteStream(join(values.output, "server.log"));
const serverLogFinished = finished(serverLog);
serverLogFinished.catch(() => {});
transport.stderr?.pipe(serverLog);
const transcript = [];
const report = {
question:
"Read architecture and the complete byte-annotated disassembly of one selected native function; export complete Evidence. No inferior execution.",
provider: values.provider,
symbol: values.symbol,
detail: values.detail,
warm: values.warm,
calls: [],
observations: [],
};
const call = createRetainedQueryCaller(client, (receipt, event) => {
report.calls.push(receipt);
transcript.push(event);
});
let sessionId;
try {
await client.connect(transport);
const catalog = await client.listTools();
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
const originalCall = client.callTool.bind(client);
client.callTool = (input, options) =>
originalCall(input, {
...options,
timeout: 60000,
toolDefinition: definitions.get(input.name),
});
const observe = async (name, input, phase = name) => {
const response = await call(name, input, phase);
assert.ok(!response.isError, JSON.stringify(response));
report.observations.push({
name,
phase,
evidence: response.structuredContent,
});
return response.structuredContent;
};
const completeParents = [];
const disassemble = async (phase) => {
const selection = values.detail === "summary" ? { detail: "summary" } : {};
const result =
values.provider === "gdb"
? await observe(
"gdb_console",
{
session_id: sessionId,
command: `disassemble /r ${values.symbol}`,
...selection,
},
phase,
)
: await observe(
"inspect_with_objdump",
{
path: values.input,
operation: "disassemble",
symbol: values.symbol,
...selection,
},
phase,
);
completeParents.push(result);
};
if (values.provider === "gdb") {
const started = await observe("start_gdb_session", {});
sessionId = started.session_id;
for (const command of [
"set auto-load off",
"set debuginfod enabled off",
`file ${values.input}`,
"show architecture",
])
await observe("gdb_console", { session_id: sessionId, command });
} else
await observe("inspect_with_objdump", {
path: values.input,
operation: "file_headers",
});
await disassemble("disassembly:fresh");
if (values.warm) await disassemble("disassembly:repeat");
if (sessionId) {
await observe("close_gdb_session", { session_id: sessionId });
sessionId = undefined;
}
await observe("export_evidence_bundle", {
path: join(values.output, "evidence.json"),
});
const bundle = JSON.parse(
await readFile(join(values.output, "evidence.json"), "utf8"),
);
for (const delivered of completeParents) {
const parent =
values.detail === "complete"
? delivered
: bundle.records.find(
(record) =>
record.evidence_id ===
delivered.normalized_result.parent_evidence_id,
);
assert.ok(parent);
assert.equal(parent.normalized_result.completion_status, "complete");
assert.equal(parent.normalized_result.output_truncated, false);
if (values.provider === "gdb") assert.equal(parent.subject, null);
else assert.equal(parent.subject.digest.sha256, oracle.artifact_sha256);
if (values.detail === "summary") {
const selected = delivered.normalized_result.summary;
const { available_counts: counts, ...facts } = selected;
const { console, target, log, ...records } = parent.normalized_result;
assert.deepEqual(
values.provider === "gdb" ? facts : selected,
values.provider === "gdb" ? records : parent.normalized_result,
);
if (values.provider === "gdb")
assert.equal(counts.console_utf16_units, console.length);
assert.equal(delivered.raw_result, null);
}
}
report.status = "passed";
} catch (error) {
report.status = "failed";
report.failure = String(error?.stack ?? error);
process.exitCode = 1;
} finally {
if (sessionId)
try {
const closed = await call(
"close_gdb_session",
{ session_id: sessionId },
"cleanup:gdb",
);
assert.ok(!closed.isError);
} catch (error) {
report.gdb_cleanup_failure = String(error);
process.exitCode = 1;
}
if (
!(await finalizeRetainedQueryReceipt({
report,
client,
transport,
account,
serverLog,
serverLogFinished,
output: values.output,
transcript,
}))
)
process.exitCode = 1;
}
@@ -1,28 +0,0 @@
from pathlib import Path
import json,re,hashlib,argparse
parser=argparse.ArgumentParser(description='Match every native instruction byte against independently mapped ELF file ranges');parser.add_argument('--root',type=Path,required=True);args=parser.parse_args();lab=args.root;oracle=json.loads((lab/'elf-oracle.json').read_text());image=Path(oracle['path']).read_bytes();assert hashlib.sha256(image).hexdigest()==oracle['artifact_sha256'];rows=[]
for directory in sorted(lab.iterdir()):
bundle=directory/'evidence.json'
if not bundle.is_file():continue
for parent in json.loads(bundle.read_text())['records']:
if parent['operation'] not in ['gdb_console','inspect_with_objdump']:continue
params=parent['parameters'];gdb=parent['operation']=='gdb_console'
symbol=params.get('command','').removeprefix('disassemble /r ') if gdb else params.get('symbol')
if symbol not in {s['name'] for s in oracle['symbols']}:continue
spec=next(s for s in oracle['symbols'] if s['name']==symbol)
report=parent['normalized_result'];assert report['completion_status']=='complete' and not report['output_truncated']
if gdb:
assert parent['subject'] is None
console='\n'.join(line for line in report['records'] if line.startswith('~'));assert console==report['console']
text=''.join(json.loads(line[1:]) for line in report['records'] if line.startswith('~'))
pattern=r'^\s*(?:=>\s*)?0x([0-9a-f]+)[^:]*:\s*((?:[0-9a-f]{2}(?:\s+|$))+)'
else:
assert parent['subject']['digest']['sha256']==oracle['artifact_sha256'];text=report['stdout'];pattern=r'^\s*([0-9a-f]+):\s*((?:[0-9a-f]{2}(?:\s+|$))+)'
address=spec['address'];collected=bytearray();count=0
for line in text.splitlines():
m=re.match(pattern,line)
if not m:continue
actual=int(m[1],16);raw=bytes.fromhex(m[2]);assert actual==address+len(collected),(directory,actual,address+len(collected));collected.extend(raw);count+=1
expected=image[spec['file_offset']:spec['file_offset']+spec['bytes']];assert collected==expected,(directory,len(collected),len(expected));rows.append({'case':directory.name,'mode':'gdb' if gdb else 'objdump','symbol':symbol,'byte_bearing_lines':count,'bytes':len(collected),'exact_file_bytes':True,'parent_evidence_id':parent['evidence_id']})
assert rows
(lab/'native-final-verification.json').write_text(json.dumps({'status':'passed','artifact_sha256':oracle['artifact_sha256'],'records':rows},indent=2)+'\n');print(json.dumps(rows))
File diff suppressed because it is too large. Load diff
@@ -1,139 +0,0 @@
# Retained command observations
Addresses [#5584](https://github.com/morluto/rea/issues/5584). GDB command Evidence
repeats ordered MI records in stream groups and a raw mirror; objdump repeats
stdout/stderr in its raw mirror. Explicit `detail: summary` retains the complete
parent and projects existing owner facts once. Default complete Evidence is
unchanged. This adds no tool, MI decoder, live-state cache or command authority.
GDB summaries preserve **every ordered record**, including asynchronous and
unrecognized records, correlated MI result, command, completion/truncation,
invalid-UTF-8 and process-exit facts. They omit the convenience console/target/log
groupings and report their original UTF-16 sizes. Objdump summaries preserve
opaque stdout/stderr, exit/signal, completion and pre/post digest-change facts.
Both carry source parameters/provider/profile, parent identity, limitations and
unknowns. Subjectless observations stay subjectless. Historical observations do
not establish current target state or global coverage. The existing
`inspect_analysis_view` summary operation works after debugger close.
The domain codec now owns the command fact shapes used by contracts and GDB's
capture type; protocol interpretation stays with its provider. MCP requires
retention before command execution. CLI uses the existing shared projection and
nonoverwriting bundle publisher after provider cleanup:
```sh
rea inspect-with-objdump /usr/bin/bash disassemble --symbol execute_command \
--detail summary --evidence_output /tmp/bash-command-evidence.json --json
```
## Real input and correctness
Linux x86-64, Node 24.18.0/npm 11.16.0; GNU GDB 13.1 (Debian 13.1-3),
GNU objdump 2.40, installed Bash ELF 1,265,648 bytes, SHA-256
`55b89ab22bee4792a210f493a53fb066accd5d30b69837c28d98be5ff863efcf`.
The [receipt](command-observation-views.json) pins native binaries, baseline main
`c2858d2aa0916f0dd2d073d848b8c2195d9bf6b3`, source patch and packaged runtime digests.
No target execution, attach, GUI or database mutation occurred.
An [independent ELF reader](command-observation-oracle.py) maps dynamic symbols
to exact file ranges: `execute_command`, 239 bytes at `0x4ba50`, and
`execute_command_internal`, 15,428 bytes at `0x47e00`. The
[verifier](command-observation-verify.py) checks every disassembled byte and its
contiguous address against those ranges. GDB reports 66/3,365 instructions;
objdump byte lines also include continuations, so their count is not an
instruction count. Ordered summary facts match each exported complete parent.
GDB Evidence has a null subject; the external oracle does not manufacture a
target binding. Actual CLI default objdump Evidence matches the baseline exactly,
including identity; symbol/pathless summaries preserve it in bundles. CLI checks
also cover rejected overwrite, missing destination before dispatch, and a real
GDB command with a caller-owned PTY that remains available after debugger exit.
## Whole tasks and resource costs
The [SDK driver](command-observation-queries.mjs) discovers all 210 tools, reads
architecture and symbol-scoped disassembly, closes GDB where applicable, exports
complete Evidence and finalizes owned resources. GDB uses 8 calls; objdump uses 3.
Warm cases repeat disassembly in the same process, adding one call. Fresh-process
cases have **unflushed OS caches**, not disk-cold inputs. Native commands are
rerun; results are not reused as current live state.
The [actual Codex 0.163 app-server driver](command-observation-wire.py) uses a
loopback deterministic SSE fixture, no AI inference or paid service. All 210
function schemas are present in every normal model-facing request. Matched tasks
use full input/output profiles; only disassembly selects summary. The
[counter](command-observation-costs.py) counts compact JSON serialization with
`tiktoken 0.12.0 / cl100k_base` across **every** request, including schemas,
replayed results and the actual client's local compaction request. The fixture
answers compaction with observed identifiers and the retained-body location.
These are serialization-token proxies, not model rendering, acceptance, answer
quality, actual usage or billing. Compaction behavior is qualified only for this
client/configuration; no inference quality is established.
| Completed GDB task | JSON token proxy before → after | Requests | Wall seconds | CPU seconds | Family peak MiB |
| ------------------------------ | ------------------------------: | -------: | ------------: | ------------: | --------------: |
| Small, 30,000 per-tool budget | 605,133 → 580,070 | 9 → 9 | 7.906 → 8.618 | 7.195 → 7.832 | 788.9 → 785.9 |
| Large, 350,000 per-tool budget | 1,269,405 → 1,091,290 | 10 → 9 | 9.551 → 9.127 | 8.832 → 8.370 | 805.2 → 770.8 |
Whole-task context proxies decrease **4.14% / 14.03%**, rather than the much
larger reply-byte reduction. The large baseline makes one compaction request;
the candidate makes none. The small task is slower. These single paired samples
do not establish a general speedup. With the client's default output budget,
the small baseline delivers truncated invalid inner JSON and does **not**
complete the question; summary completes it. That incomplete baseline is
excluded from the percentage comparison.
| SDK task | Reply bytes before → after | Whole wall seconds | CPU seconds | Family peak MiB |
| ------------- | -------------------------: | -----------------: | ------------: | --------------: |
| GDB small | 54,123 → 18,009 | 6.631 → 5.447 | 6.102 → 5.240 | 536.3 → 535.8 |
| GDB large | 2,763,983 → 700,453 | 6.965 → 6.660 | 6.729 → 6.451 | 595.0 → 545.8 |
| Objdump small | 22,467 → 15,990 | 7.599 → 7.699 | 7.110 → 7.264 | 527.8 → 532.5 |
| Objdump large | 964,889 → 487,219 | 7.737 → 7.499 | 7.334 → 6.935 | 537.0 → 536.6 |
Same-process repeat disassembly takes 1,100 → 1,016 ms for GDB and
1,070 → 939 ms for objdump in these observations. Whole warm-task CPU improves
7.613 → 7.162 s for GDB but increases 7.819 → 7.859 s for objdump. Timings include
existing ownership checks; no check was removed or cached.
Retaining a summary also retains an additional derived record: large GDB bundles
grow 1,310,073 → 1,640,022 bytes; objdump 474,288 → 712,656. This trades retained
storage for smaller delivery; long-session heap growth is unmeasured. Retained-data lifetime follows the existing investigation ledger and cleanup
owners. Output still scales
with the requested complete disassembly; summary is not a fixed-size preview.
Each experiment runs in one owned systemd scope, one CPU, no swap, 4 GiB hard /
3 GiB early memory thresholds and host-memory admission/stop thresholds. Recorded
payload exit and cleanup are verified, with zero OOM/resource events. Final
whole-scope counters disappear after teardown and remain explicitly unverified.
The receipt includes CPU time, wall time, peak family memory, warm cases and
bundle growth. Build/check scopes use separate bounded budgets. Host-memory
check stops and initial fixture failures are preserved locally and are not passes.
## Coverage and reproduction
320 focused tests cover the new SDK schemas/views plus existing GDB queue,
malformed output, cancellation, process exit, target identity and cleanup cases,
objdump symbol selection and command admission. Real provider evidence is from
the CLI/SDK/client runs above, not simulated-provider tests.
Run the oracle for the selected Bash image, then use the SDK driver with
`--runtime`, `--input`, `--oracle`, `--output`, `--provider gdb|objdump`,
`--symbol`, `--detail complete|summary` and optional `--warm`. The wire driver
also takes the fixed Codex/Node paths, `--mode`, `--symbol` and `--budget`; it always performs the
export (eight tool calls). Run each through `resource-run.py --observe-payload-exit`, preserving
its sibling `.log`; the verifier and counter accept `--root` for those results.
Default Rizin execution returned ENOENT with no configured alternative. Cutter
reported no live bridge; no configured selector or default bridge directory was
present. Real Rizin/Cutter performance and GUI workflows remain unverified.
Their command semantics were not changed. Existing objdump native symbol
selection was reused. No claim covers arbitrary GDB target execution or another
host client's output budget.
Review follow-up adds nine CLI publication regressions (16 CLI tests including the
existing cleanup cases), three deterministic delivery-oracle tests, and a focused
strict typecheck of the new CLI boundary and its transitive source. Real client
small/default and large/generous summary runs pass the strengthened oracle and
export complete parents; only the explicit complete/default baseline may finish
with an incomplete answer. The unused export switch was removed: export is
mandatory in this question. Product source and the original paired measurements
are unchanged; the receipt records these additional validation runs separately.
@@ -1,301 +0,0 @@
"""Actual Codex wire capture with a deterministic local SSE backend; no AI inference."""
import argparse
import http.server
import threading
import asyncio
import hashlib
import json
import os
from pathlib import Path
import re
import time
import traceback
import subprocess
from command_observation_delivery import verify_command_delivery
async def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--codex", type=Path, required=True)
parser.add_argument("--node", type=Path, required=True)
parser.add_argument("--input-profile", choices=["full", "compact"], default="full")
parser.add_argument("--output-profile", choices=["full", "off"], default="full")
parser.add_argument("--input", type=Path, required=True)
parser.add_argument("--oracle", type=Path, required=True)
parser.add_argument("--expected-tools", type=int, default=210)
parser.add_argument("--mode", choices=["complete", "summary"], required=True)
parser.add_argument("--question", choices=["disassembly"], default="disassembly")
parser.add_argument("--symbol", choices=["execute_command", "execute_command_internal"], required=True)
parser.add_argument("--budget", type=int, default=30000)
args = parser.parse_args()
oracle=json.loads(args.oracle.read_text())
target=str(args.input)
assert args.input.is_absolute() and args.oracle.is_absolute() and args.output.is_absolute()
assert hashlib.sha256(args.input.read_bytes()).hexdigest()==oracle["artifact_sha256"]
def delivered(pieces):
assert isinstance(pieces,str) and pieces.startswith("Wall time: "), pieces[:300]
_,sep,payload=pieces.partition("Output:\n");assert sep
try: value=json.loads(payload)
except json.JSONDecodeError: return {"_invalid_json_payload":payload}
return value["result"] if "result" in value and "normalized_result" not in value else value
def outputs_of(body):
return {item["call_id"]:delivered(item["output"]) for item in body.get("input",[]) if item.get("type")=="function_call_output"}
steps=8
args.output.mkdir(mode=0o700)
audit_home = args.output / "account"
codex_state = audit_home / ".codex"
mcp_home = audit_home / "mcp"
workspace = audit_home / "workspace"
for directory in [codex_state, mcp_home, workspace]:
directory.mkdir(parents=True)
captured = []
known_outputs = {}
state = {"normal_requests":0,"compactions":0}
class Capture(http.server.BaseHTTPRequestHandler):
def log_message(self, *_):
pass
def do_POST(self):
size = int(self.headers["Content-Length"])
assert 0 < size <= 16 * 1024 * 1024
assert self.headers.get("Content-Encoding", "identity") == "identity"
body = json.loads(self.rfile.read(size))
captured.append({"path": self.path, "body": body})
(args.output / "model-requests.json").write_text(json.dumps(captured) + "\n")
http_index = len(captured)
known_outputs.update(outputs_of(body))
names = [t.get("name", "") for ns in body.get("tools", []) if ns.get("type") == "namespace" and ns.get("name") == "mcp__rea_audit" for t in ns.get("tools", [])]
if not names:
last=body.get('input',[])[-1]
assert 'CONTEXT CHECKPOINT COMPACTION' in json.dumps(last)
state['compactions']+=1;assert state['compactions']<=1
evidence=known_outputs['call_local_6'];assert 'normalized_result' in evidence
captured_result=evidence['normalized_result'];command_report=captured_result.get('summary',captured_result)
checkpoint={'fixture':'deterministic protocol checkpoint; no AI inference or model-quality claim','task':'Full native Bash function disassembly has been delivered to this compaction request. Retain complete Evidence and finish owned cleanup.','path':target,'artifact_sha256_external_oracle':oracle['artifact_sha256'],'symbol':args.symbol,'session_id':known_outputs['call_local_1']['session_id'],'parent_evidence_id':captured_result.get('parent_evidence_id',evidence['evidence_id']),'subject':evidence['subject'],'completion_status':command_report['completion_status'],'output_truncated':command_report['output_truncated'],'native_console_utf16_units':command_report.get('available_counts',{}).get('console_utf16_units',len(command_report.get('console','').encode('utf-16-le'))//2),'full_body_location':'same-session retained Evidence; export next to '+str(args.output/'evidence.json'),'remaining':['close_gdb_session','export_evidence_bundle']}
text=json.dumps(checkpoint)
item={"type":"message","id":f"msg_checkpoint_{state['compactions']}","role":"assistant","status":"completed","content":[{"type":"output_text","text":text,"annotations":[]}]}
captured[-1]['kind']='deterministic-compaction'
else:
state['normal_requests']+=1;index=state['normal_requests'];assert index<=steps+1
captured[-1]['kind']='tool-planning-or-final'
if index<=steps:
if index==1:
operation="start_gdb_session";arguments={}
else:
session=known_outputs["call_local_1"]["session_id"]
if index<=6:
command=['set auto-load off','set debuginfod enabled off',f'file {target}','show architecture',f'disassemble /r {args.symbol}'][index-2]
operation="gdb_console";arguments={"session_id":session,"command":command}
if index==6 and args.mode=="summary":arguments["detail"]="summary"
elif index==7:
operation="close_gdb_session";arguments={"session_id":session}
else:
operation="export_evidence_bundle";arguments={"path":str(args.output/'evidence.json')}
selected=[name for name in names if name.endswith(operation)];assert len(selected)==1
item={"type":"function_call","id":f"fc_local_{index}","call_id":f"call_local_{index}","name":selected[0],"namespace":"mcp__rea_audit","arguments":json.dumps(arguments),"status":"completed"}
else:
item={"type":"message","id":"msg_local_done","role":"assistant","status":"completed","content":[{"type":"output_text","text":"Scripted local GDB disassembly task complete.","annotations":[]}]}
response_id=f"resp_local_{http_index}"
base = {"id": response_id, "object": "response", "model": body.get("model"), "status": "completed", "output": [item]}
events = [
{"type": "response.created", "response": {**base, "status": "in_progress", "output": []}},
{"type": "response.output_item.added", "output_index": 0, "item": {**item, "status": "in_progress"}},
{"type": "response.output_item.done", "output_index": 0, "item": item},
{"type": "response.completed", "response": base},
]
wire = "".join("event: " + e["type"] + "\ndata: " + json.dumps(e) + "\n\n" for e in events).encode()
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.send_header("Content-Length", str(len(wire)))
self.end_headers()
self.wfile.write(wire)
endpoint = http.server.HTTPServer(("127.0.0.1", 0), Capture)
endpoint_thread = threading.Thread(target=endpoint.serve_forever, daemon=True)
endpoint_thread.start()
task_path = f"{args.node.parent}:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
mcp_env = {"HOME": str(mcp_home), "PATH": task_path,
"NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1",
"REA_LOG_LEVEL": "silent", "REA_MCP_INPUT_SCHEMA_PROFILE": args.input_profile,
"REA_MCP_OUTPUT_SCHEMA_PROFILE": args.output_profile}
config = 'model_provider = "rea_capture"\nmodel = "rea-wire-probe"\n[analytics]\nenabled = false\n[mcp_servers.rea_audit]\n'
config += f'command = {json.dumps(str(args.node))}\n'
config += f'args = {json.dumps([str(args.runtime / "scripts/rea.mjs"), "mcp"])}\n'
config += 'startup_timeout_sec = 60\ntool_timeout_sec = 60\n'
config += '[mcp_servers.rea_audit.env]\n'
config += ''.join(f'{key} = {json.dumps(value)}\n' for key, value in mcp_env.items())
for name in ['start_gdb_session','gdb_console','close_gdb_session','export_evidence_bundle']:
config += f'[mcp_servers.rea_audit.tools.{name}]\napproval_mode = "approve"\n'
if args.budget: config += f'output_token_limit = {args.budget}\n'
config += f'\n[model_providers.rea_capture]\nname = "Local scripted capture"\nbase_url = "http://127.0.0.1:{endpoint.server_port}/v1"\nwire_api = "responses"\nrequires_openai_auth = false\nsupports_websockets = false\nrequest_max_retries = 0\nstream_max_retries = 0\n'
(codex_state / "config.toml").write_text(config)
(args.output / "declared-config.toml").write_text(config)
environment = {"HOME": str(audit_home), "CODEX_HOME": str(codex_state), "PATH": task_path,
"NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1"}
report = {"question": "Return architecture and complete byte-annotated disassembly for one selected native Bash function and export complete Evidence. No target execution.",
"client": "Codex app-server 0.163.0", "model_turns_started": 1, "backend": "local deterministic SSE fixture; no AI model or paid service",
"actual_model_usage": None, "input_profile": args.input_profile,
"output_profile": args.output_profile, "calls": [], "observations": [],
"unexpected_server_requests": [], "cleanup_failures": []}
client_version = subprocess.check_output([str(args.codex), "--version"], env=environment, cwd=workspace, text=True, timeout=10).strip()
assert client_version == "codex-cli 0.163.0", client_version
transcript = []
pending = {}
completed_turn = asyncio.Event()
next_id = 0
stderr = (args.output / "server.log").open("wb")
process = await asyncio.create_subprocess_exec(str(args.codex), "app-server", "--stdio",
stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE, stderr=stderr,
cwd=workspace, env=environment, limit=16 * 1024 * 1024)
async def send(message):
encoded = (json.dumps(message, separators=(",", ":")) + "\n").encode()
transcript.append({"direction": "client", "bytes": len(encoded), "message": message})
process.stdin.write(encoded)
await process.stdin.drain()
async def read_messages():
try:
while line := await process.stdout.readline():
value = json.loads(line)
transcript.append({"direction": "server", "bytes": len(line), "message": value})
if value.get("method") == "turn/completed":
report["turn"] = value["params"]["turn"]
completed_turn.set()
if "id" in value and "method" not in value:
future = pending.pop(value["id"], None)
if future and not future.done():
future.set_result(value)
elif "id" in value:
report["unexpected_server_requests"].append(value["method"])
await send({"id": value["id"], "error": {"code": -32601, "message": "Unexpected request in scripted metadata probe"}})
raise RuntimeError("app-server stdout closed")
except Exception as error:
for future in pending.values():
if not future.done():
future.set_exception(error)
reader = asyncio.create_task(read_messages())
async def request(method, params):
nonlocal next_id
assert method in ["initialize", "thread/start", "mcpServerStatus/list", "mcpServer/tool/call", "turn/start"]
next_id += 1
message = {"id": next_id, "method": method, "params": params}
future = asyncio.get_running_loop().create_future()
pending[next_id] = future
start = time.monotonic()
await send(message)
reply = await asyncio.wait_for(future, 90)
report["calls"].append({"method": method, "elapsed_ms": (time.monotonic() - start) * 1000,
"error": reply.get("error")})
if "error" in reply:
raise RuntimeError(json.dumps(reply["error"])[:4000])
return reply["result"]
try:
await request("initialize", {"clientInfo": {"name": "rea_host_probe", "title": "REA host compatibility probe", "version": "1"},
"capabilities": {"experimentalApi": True, "explicitGatewayOauth": True}})
await send({"method": "initialized"})
created = await request("thread/start", {"cwd": str(workspace), "ephemeral": True,
"approvalPolicy": "never", "sandbox": "read-only"})
thread_id = created["thread"]["id"]
assert not created["thread"].get("turns", [])
inventory = await request("mcpServerStatus/list", {"threadId": thread_id,
"serverName": "rea_audit", "detail": "toolsAndAuthOnly", "limit": 10})
assert inventory.get("nextCursor") is None
assert len(inventory["data"]) == 1
server = inventory["data"][0]
assert server["name"] == "rea_audit" and server.get("toolsError") is None
tools = list(server["tools"].values())
names = [tool["name"] for tool in tools]
assert len(names) == len(set(names)) == args.expected_tools
assert "gdb_console" in names
output_schemas = sum(tool.get("outputSchema") is not None for tool in tools)
assert output_schemas == (args.expected_tools if args.output_profile == "full" else 0)
raw = json.dumps(inventory, separators=(",", ":"), ensure_ascii=False).encode()
(args.output / "catalog.json").write_bytes(raw)
report["catalog"] = {"tools": len(tools), "output_schemas": output_schemas,
"bytes": len(raw), "sha256": hashlib.sha256(raw).hexdigest(),
"server_info": server.get("serverInfo"), "runtime_status": server.get("runtimeStatus")}
await request("turn/start", {"threadId":thread_id,"input":[{"type":"text","text":f"Read architecture and the complete byte-annotated disassembly of {args.symbol} in the selected Bash executable. Retain and export complete Evidence, then close the owned debugger. The local fixture scripts the required tools; no AI inference or target execution is used."}]})
await asyncio.wait_for(completed_turn.wait(), 90)
assert report["turn"]["status"] == "completed", report["turn"]
assert state["normal_requests"]==steps+1
assert len(captured)==steps+1+state["compactions"]
outputs=known_outputs
assert len(outputs)==steps
bundle=json.loads((args.output/'evidence.json').read_text())
evidence=next(e for e in bundle['records'] if e['operation']=='gdb_console' and e['parameters']['command']==f'disassemble /r {args.symbol}')
assert evidence['subject'] is None
assert evidence['normalized_result']['completion_status']=='complete' and not evidence['normalized_result']['output_truncated']
delivered_result=outputs['call_local_6']
delivery_complete=verify_command_delivery(delivered_result,evidence,args.mode,args.budget)
if delivery_complete and args.mode=='summary':
assert next(e for e in bundle['records'] if e['evidence_id']==delivered_result['evidence_id'])==delivered_result
report['task_completed']=delivery_complete
report['observations']={'parent_evidence_id':evidence['evidence_id'],'exported':True,'native_console_utf16_units':len(evidence['normalized_result']['console'].encode('utf-16-le'))//2,'client_delivered_full_evidence':delivery_complete,'selected_symbol':args.symbol,'tool_output_budget':args.budget or 'client default','subject':evidence['subject']}
report["captured_requests"]=len(captured)
report["compaction_requests"]=state["compactions"]
report["compaction_backend"]="Deterministic checkpoint of observed identifiers and retained-body location; no AI inference or quality claim."
report["mode"]=args.mode
report["request_tool_counts"] = [len(c["body"].get("tools", [])) for c in captured]
assert not report["unexpected_server_requests"]
assert hashlib.sha256(args.input.read_bytes()).hexdigest()==oracle["artifact_sha256"]
report["status"] = "passed" if report["task_completed"] else "expected-incomplete-task"
except Exception:
report["status"] = "failed"
report["failure"] = traceback.format_exc()[-6000:]
finally:
process.stdin.close()
try:
await asyncio.wait_for(process.wait(), 15)
except asyncio.TimeoutError:
report["cleanup_failures"].append("app-server did not exit after stdin close; terminated owned process")
process.terminate()
try:
await asyncio.wait_for(process.wait(), 5)
except asyncio.TimeoutError:
process.kill()
await process.wait()
await reader
# app-server can acknowledge exit before its MCP child finishes EOF
# cleanup. Wait on the allocated resource scope; never target a PID.
try:
groups = [line[3:] for line in Path("/proc/self/cgroup").read_text().splitlines()
if line.startswith("0::")]
assert len(groups) == 1
group = Path("/sys/fs/cgroup") / groups[0].lstrip("/")
assert re.fullmatch(r"rea-efficiency-[0-9a-f]{32}\.scope", group.name)
observer = os.getppid()
observer_args = Path(f"/proc/{observer}/cmdline").read_bytes().split(b"\0")
assert any(Path(os.fsdecode(value)).name == "resource_payload.py" for value in observer_args if value)
expected = {os.getpid(), observer}
until = time.monotonic() + 10
while True:
actors = {int(value) for value in (group / "cgroup.procs").read_text().splitlines()}
assert expected <= actors
remaining = sorted(actors - expected)
if not remaining or time.monotonic() >= until:
break
await asyncio.sleep(0.05)
report["owned_scope_drain"] = {"remaining_processes": remaining}
if remaining:
report["cleanup_failures"].append("owned MCP child did not settle after app-server exit")
except (OSError, ValueError, AssertionError) as error:
report["owned_scope_drain"] = {"status": "unavailable", "reason": type(error).__name__}
report["cleanup_failures"].append("allocated resource-scope cleanup could not be verified")
endpoint.shutdown()
endpoint.server_close()
endpoint_thread.join(5)
stderr.close()
report["appserver_exit_code"] = process.returncode
if process.returncode != 0:
report["cleanup_failures"].append(f"app-server exit code: {process.returncode}")
(args.output / "transcript.json").write_text(json.dumps(transcript) + "\n")
(args.output / "report.json").write_text(json.dumps(report, indent=2) + "\n")
print(json.dumps({key: report.get(key) for key in ["status", "failure", "catalog", "observations", "cleanup_failures", "appserver_exit_code"]}))
return 0 if report["status"] in ["passed","expected-incomplete-task"] and not report["cleanup_failures"] else 1
raise SystemExit(asyncio.run(main()))
@@ -1,52 +0,0 @@
#!/usr/bin/env node
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { readFile } from "node:fs/promises";
import { isDeepStrictEqual } from "node:util";
import canonicalize from "canonicalize";
const [baselinePath, candidatePath, recordPath] = process.argv.slice(2);
assert.ok(baselinePath && candidatePath && recordPath);
const record = JSON.parse(await readFile(recordPath, "utf8"));
const fingerprinted = record.trials.at(-1);
const digest = (value) =>
createHash("sha256").update(canonicalize(value)).digest("hex");
const readSemantic = async (path, label) => {
const evidence = JSON.parse(await readFile(path, "utf8"));
const analysis = evidence.normalized_result;
assert.equal(analysis.root_artifact_sha256, record.input.asar_sha256);
const semantic = analysis.semantic_graph;
assert.equal(semantic.application_graph_id, analysis.graph.graph_id);
const rawSha = digest(semantic);
assert.equal(
rawSha,
fingerprinted[label].profile.analysis_fingerprints.semantic_graph_sha256,
);
const { application_graph_id, graph_id, ...content } = semantic;
return { content, rawSha, application_graph_id, graph_id };
};
// Read sequentially; retain semantic content rather than complete parent graphs.
const before = await readSemantic(baselinePath, "baseline");
const after = await readSemantic(candidatePath, "candidate");
assert.ok(
isDeepStrictEqual(before.content, after.content),
"Semantic content differs after excluding only the two parent commitments",
);
console.log(
JSON.stringify({
excluded_top_level_fields: ["application_graph_id", "graph_id"],
content_deep_equal: true,
baseline_raw_sha256: before.rawSha,
candidate_raw_sha256: after.rawSha,
baseline_content_sha256: digest(before.content),
candidate_content_sha256: digest(after.content),
baseline_application_graph_id: before.application_graph_id,
candidate_application_graph_id: after.application_graph_id,
baseline_graph_id: before.graph_id,
candidate_graph_id: after.graph_id,
compared_fields: Object.keys(before.content).sort(),
serialization:
"RFC 8785 canonical JSON; arrays and all remaining fields retained",
}),
);
@@ -1,59 +0,0 @@
# Workflow coverage reconciliation, 2026-10-21
The checked snapshot is main `264862bf2`, with **46 first-level `src/`
directories and 212 production stdio MCP tools**. Directory counts and tool
counts are different inventories: one application workflow can cross several
directories, and one provider can expose many independently costly operations.
This is a source/history review, not a new runtime qualification of all tools.
The search includes this experiment directory, owning contracts and source
history, original feature PRs and related open/closed issues. Missing experiment
files do not establish missing audits. In particular, the model and IL2CPP
feature PRs contain substantial real-task evidence absent from this directory.
Historical measurements retain their original revisions, platforms and limits.
| Directories / workflow | Evidence already present | Remaining coverage boundary |
| ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `kuna`, `ghidra` | Earlier native entry work and [Ghidra projection](ghidra-evidence-projection.md) | Selected questions only; database reuse is a separate [research issue #3136](https://github.com/morluto/rea/issues/3136). |
| `ida` | [Windows native correctness and completed-task costs](ida-native-windows.md), including real `whoami.exe` | Warm selected-task gains do not establish cold analysis, all string formats, all operations or other IDA platforms. |
| `hopper`, `jeb` | [Hopper string records](hopper-string-records.md), [Linux runtime](hopper-linux-python-runtime.md), [real JEB CE](jeb-linux-native.md) | Preserve negative whole-task results; other editions, platforms and workflows are separate scopes. |
| `rizin`, `cutter` | Existing command/session adapters and lifecycle fixes | No comparable dedicated real completed-task performance record located in the searched records. Actual engine/GUI availability must be established before selecting a task. |
| `gdb`, `objdump` | [Real Bash command observations](command-observation-views.md) | Selected read-only disassembly is measured; other commands, large outputs and running-target observation are separate tasks. |
| `adb`, `frida`, `inspector` | [Frida native status/history experiment](frida-status-views.md) | Frida status does not qualify arbitrary scripts, ADB/device transfers, browser Inspector events or live target freshness. |
| `go`, `rust` | [Go build/dependency selection on real binaries](go-build-views.md); Rust cargo-auditable support [#2456](https://github.com/morluto/rea/pull/2456) | Go is already measured. No comparable dedicated Rust dependency-investigation cost record located in this review. |
| `javascript`, `browser`, `electron` | [Agent fact workflow](javascript-facts-agent-history.json), lifecycle experiments and [#4415](https://github.com/morluto/rea/issues/4415) | Real Joplin preparation remains exploratory. Individual JS features and browser/runtime workflows need their own evidence. |
| `android`, `apktool` | [Package summary](android-package-summary.md), [three-APK shared resource decoding](apktool-string-tables.md) | These do not qualify all JADX class/method/reference operations or physical-device workflows. #4777 is already fixed. |
| `dotnet`, `il2cpp` | [Managed summaries/pages](managed-artifact-summary.md), IL2CPP metadata [#3807](https://github.com/morluto/rea/pull/3807) and native links/seeds [#5327](https://github.com/morluto/rea/pull/5327) | IL2CPP already has real Windows/Android row comparisons, task costs and Ghidra import. Unsupported versions/generics/platforms remain separate correctness and coverage work. |
| `model` | GGUF/safetensors [#5603](https://github.com/morluto/rea/pull/5603): real models, independent readers and complete-task costs | Existing delivery is already audited. [#5774](https://github.com/morluto/rea/issues/5774) isolates omitted-string validation cost within that producer. |
| `native`, `artifacts` | [Native inventory](native-inventory-views.md), [entry views](native-entry-views.md); real NIB output reductions [#1460](https://github.com/morluto/rea/pull/1460), [#1477](https://github.com/morluto/rea/pull/1477) | Inventory does not qualify Apple UI/resource, signatures, dyld or runtime traces. NIB byte reductions alone do not establish token, time, CPU or memory savings for a completed question. |
| `firmware` | Region pilot plus real full OpenWrt extraction/publication [#4602](https://github.com/morluto/rea/pull/4602) | Extraction is already measured; noisy timings establish no general speedup. Other image layouts, extraction trees and publication filesystems remain unqualified. |
| `flutter`, `sqlite`, `wasm`, `evm`, `pcap`, `signals` | [Dart AOT](flutter-aot-summary.md), [SQLite controls](sqlite-existing-controls.md), [WAT framing](wasm-string-framing.md), [EVM](evm-interface-summary.md), [PCAP](pcap-small-captures.md), [signals](signal-selection-workflows.md) | Extend a specific question. Preserve cases where existing controls suffice and small tasks regress; these records do not qualify each format or tool exhaustively. |
| `application`, `domain`, `server`, `cli`, `contracts` | [Mixed retained investigations](retained-investigation-workflows.md), [immutable parent validation](immutable-parent-validation.md), [capture validation](retained-capture-validation.md) | Shared selected-query paths are already optimized. Reconstruction/environment composition and new fact families require their own end-to-end questions. |
| `browser` / recorded captures | [SAZ selection](saz-selection-workflows.md) and [real qlog producer/retained reads](retained-capture-validation.md) | [#3133](https://github.com/morluto/rea/issues/3133) still includes actual model/task questions; producer measurements do not close those requirements. |
| `filesystem`, `process`, `reference`, `providers`, `captures`, `windows`, `main`, `composition`, `config`, `evaluation` | Cross-workflow resource/cleanup checks, [process traces](process-trace-validation.md), [MCP profile](mcp-profile-identity.md) and Windows lifecycle checkpoints | Infrastructure coverage follows the exercised workflow. No blanket startup/resource claim; reference imports and `compare_source_to_bundle` need a dedicated completed source-tree question. |
Every one of the 46 directories appears above; repeated `browser` highlights a
distinct workflow, not another directory. The table records located evidence,
not proof that every historical PR was inspected. Open issue state by itself
does not prove missing implementation or absent measurements.
## Three bounded follow-up questions
1. **Rust dependency identity:** inspect a public cargo-auditable binary, find
exact dependency/version edges and retain complete source Evidence. Compare
existing delivery first; independently decode its auditable section. Include
a small binary and account for all discovery, queries and serialization.
2. **Apple UI graph lookup:** on macOS, locate a specific control/outlet in a
real Transmission/XLD NIB graph using existing summary/document pages.
Compare equivalent complete questions and independently check archive
records. Historical byte reductions alone are not the completion criterion.
3. **Reference/source comparison:** import a pinned public source tree and
answer one source-to-bundle identity question through existing workflows.
Independently check file manifests, symlink/unknown semantics and selected
matches; measure repeated traversal and retained representation before
proposing an index or cache.
Each lane stops with a measured benefit, an evidenced no-change decision, or an
explicit unavailable prerequisite. Rizin/Cutter and device/live workflows stay
visible as additional gaps; no new execution, injection or mutation authority is
implied by this review. No all-provider run or new API is justified by counts.
@@ -1,149 +0,0 @@
# Directory decompilation: capacity, resume and source Evidence
The directory-export feature requested in #6090 already shipped in #6170.
This follow-up verifies the current public workflow and fixes a source-provenance
gap found while combining it with existing Ghidra projects (#1048).
## Completed task and correctness
The analyst asks for all functions as local pseudocode files, without putting
their bodies in agent context. The committed `decompile-directory.mjs` driver
uses the production stdio MCP entrypoint and installed SDK. It discovers the
catalog, exports an initial chunk, closes/reopens the provider, resumes all
remaining work, and resumes again to verify settled files. It authenticates an
explicitly exported Evidence bundle and checks the original target/project.
Every written file is checked against its index SHA-256 and address/profile
header. Existing file hashes and modification times must survive both resumes.
Comparison facts include every address, name, status, body range and pseudocode
digest. An independent fresh-import investigation from #1048 additionally checks
the complete inventory and two nonempty pseudocode observations. This establishes
export preservation of provider observations; it does not establish discovery of
every actual function or equivalence to original source.
| Real artifact | Provider inventory | Written | External | Failed |
| ----------------------- | -----------------: | ------: | -------: | -----: |
| `/usr/bin/ls` | 652 | 541 | 111 | 0 |
| system `libcrypto.so.3` | 14,338 | 14,194 | 143 | 1 |
Target SHA-256 values are respectively
`cb30d69b24245bf2ecdc9e7f53bbad19159999970b6d82c0c00c7d32d9e37aa4` and
`aa01e614af634bc7c8b678c9250b5c4f613c61989bacf0a5d6aed6570e7dcabd`.
These installed ELF x86-64 binaries were analyzed statically, never executed.
The large run explicitly selected a five-second per-function deadline; function
`0x387f00` exceeded it, remained a recorded failure, and was retried on resume.
Later functions were still exported. No disappearance of that failure is claimed.
The resulting large tree occupies about 107 MiB on this filesystem.
## Source-provenance correction
Before this change, `BinarySession` retained provider `sourceEvidence` only inside
its snapshot-eligible branch. Ghidra procedure reads without a document selector
are not snapshot eligible. A composed directory export therefore lost saved
project reports even though the provider supplied them. The workflow also ignored
source-record IDs, and the function-dossier adapter copied observation fields
manually without those links.
The session now authenticates/retains source records independently of replay
eligibility. Plain application helpers also return authenticated immutable source
records with their outcome, including late cancellation. Both adapters explicitly
commit them before delivering links; an adapter without a retention owner refuses
the response. Source collection borrows ledger records without constructing or
hashing an intermediate bundle. The workflow links distinct source IDs without copying their payloads.
The function adapter uses the common observation-to-Evidence constructor,
preserving sources through normal and late-cancelled completion. Queries with no
source records avoid the extra recording path. Complete provider bodies remain
in the exported function files; this does not add per-function payload retention.
A real test project contains historical seed and disabled-analyzer reports saved
by REA's actual Ghidra pre-scripts. The unmodified runtime completed the export
but failed the provenance oracle: its bundle had two workflow records and no
historical source. The fixed runtime retains exactly one historical source and
links the workflow to it. All 652 inventory entries and the sampled pseudocode
still match fresh import; source project content stays unchanged. The report is
historical, with remaining original settings/completeness explicitly unknown.
## Measured costs
Baseline runtime: `19447516a4bdbc9437d112ebd3f12a5fd062db83`.
Initial fixed runtime: `f50763fd21527bdf3bab944633617b9df61dded4`, rebased on
main `f02df45a4`. Both use Node 24.22.1, locked dependencies, actual Ghidra 12.1.4
and JDK 21.0.12.1+1. The fixed driver later delegates transcript publication to
the existing cleanup/receipt owner; the protocol and task are unchanged.
| Completed task | Wall s | Family CPU s | Peak family GiB | Full-wire token estimate |
| --------------------------------- | ------: | -----------: | --------------: | -----------------------: |
| ls, before | 51.411 | 78.984 | 0.943 | 57,082 |
| ls, initial fix | 50.937 | 79.891 | 1.201 | 57,072 |
| ls with historical reports, fixed | 52.827 | 80.236 | 1.097 | 57,207 |
| libcrypto, baseline capacity run | 506.387 | 459.818 | 1.502 | 57,322 |
The plain ls pair has identical complete comparison facts. Runtime is similar;
the higher sampled memory in the fixed run is reported rather than treated as an
improvement. These sequential shared-host samples establish no generic speed or
memory ratio. [PR #7250](https://github.com/morluto/rea/pull/7250) records the final
large-run comparison and CI/review results.
The failed historical baseline is excluded from completed-task comparisons.
The complete MCP replies stayed around 11–12 KiB, including repeated textual and
structured representations, profile and caveats. The normalized result is only
the manifest. For example, the large continuation costs about 3,159 estimated
wire tokens; its complete task is 57,322 because discovery alone costs 47,049.
Both numbers include actual protocol content, rather than only pseudocode bytes.
Estimates use `cl100k_base` on the actual JSON-RPC messages, including discovery,
requests, responses and notifications, excluding only transport framing and the
driver's local transcript wrapper. They are not actual model billing. Host
projection, prompts/reasoning and later agent file reads remain unmeasured.
The historical source increases response token cost slightly while preserving
provenance. No token-reduction claim is made for this correctness fix.
CPU and memory cover the driver, REA and provider process family. Each run uses
CPU affinity 0,1 and an 8-GiB sampled family-RSS guard; Java heaps are 1536M for ls
and 3072M for crypto. RSS is sampled every 100 ms and summed: shared pages may be
counted repeatedly and short peaks may be missed. CPU is waited child usage.
Reported completed runs had no resource stop, OOM or cleanup failure.
Project preparation is excluded and documented separately in
[project reuse](ghidra-project-reuse.md). Provider starts are process-cold, with
host filesystem caches unflushed; verified resume occurs in the second session.
The index and inventory still scale with function count. This 14,338-item run
does not establish a larger capacity guarantee. Real Windows/macOS filesystems,
other decompilers and native DWARF directory layout remain unverified. The current
workflow reports unavailable DWARF layout explicitly instead of inventing it.
Actual CLI `decompile-all` calls additionally selected only `0x10a420`, reported
651 excluded entries and `debug_layout_unavailable` for requested dwarf layout,
then resumed in a new process. The existing file's hash and modification time
stayed unchanged; the returned historical source ID stayed stable. This CLI check
verifies the returned manifest/link and local function file, not offline retrieval
of the separately retained source record after the CLI process exits.
## Reproduce and inspect
Prepare an owned project using the documented
[existing-project route](../../docs/installation.md#reuse-an-existing-ghidra-project).
Use fresh receipt/output paths; the driver refuses existing output directories.
```sh
node experiments/analysis-efficiency/decompile-directory.mjs \
--runtime /absolute/rea --output /absolute/new-receipt \
--target /usr/bin/ls --project /absolute/ls-analysis.gpr --program ls \
--ghidra "$GHIDRA_INSTALL_DIR" --java "$JAVA_HOME"
```
Use `--chunk 512 --heap 3072M --decompile-seconds 5` for the crypto case.
`--control /absolute/previous-receipt` compares all exported facts;
`--golden /absolute/fresh-import-receipt` checks the #1048 driver's inventory and
pseudocode oracle. `--history required` additionally requires a linked historical
source. The driver allows failed functions to succeed on a later retry, verifies only
previously written files remain unchanged, and reloads the final index before
comparison. Complete local receipts include the transcript, resource report, index,
function files, comparison facts and authenticated Evidence export. The resource
report is supplied by the external process-family sampler, not the driver.
The current nine-file suite passed 144 tests. Against unmodified main source,
the three source-retention/constructor suites reproduce seven failures and 70
passes; the fixed full suite passes. This includes invalid source commitments,
live and cursor-dependent reads, shared source deduplication, selected/null/provider
subjects, and source links on late-cancelled completion.
@@ -1,328 +0,0 @@
/** Real-provider directory export, cross-session resume and retained provenance. */
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { execFile } from "node:child_process";
import { createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readFile, stat, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { finished } from "node:stream/promises";
import { pathToFileURL } from "node:url";
import { parseArgs, promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
const { values } = parseArgs({
options: Object.fromEntries(
[
"runtime",
"output",
"target",
"project",
"program",
"ghidra",
"java",
"control",
"heap",
"chunk",
"decompile-seconds",
"history",
"golden",
].map((key) => [key, { type: "string" }]),
),
});
for (const key of [
"runtime",
"output",
"target",
"project",
"ghidra",
"java",
]) {
assert.ok(values[key], `${key} is required`);
values[key] = resolve(values[key]);
}
const chunk = Number(values.chunk ?? 32);
assert.ok(Number.isSafeInteger(chunk) && chunk > 0);
await mkdir(values.output);
const digest = (bytes) => createHash("sha256").update(bytes).digest("hex");
const report = {
calls: [],
runtime: values.runtime,
target: values.target,
project: values.project,
};
let client, transport, account, serverLog, serverLogFinished;
// This workflow has a fixed number of public calls; pseudocode stays on disk.
const transcript = [];
let phase = "initialize";
const started = performance.now();
const tree = join(values.output, "tree");
const readIndex = async () =>
JSON.parse(await readFile(join(tree, "decompile-index.json"), "utf8"));
const fileStates = async (index) => {
const states = {};
for (const row of index.functions.filter((row) => row.status === "written")) {
const path = join(tree, row.relative_path);
const metadata = await stat(path);
states[row.address] = {
sha256: digest(await readFile(path)),
mtime_ms: metadata.mtimeMs,
};
assert.equal(states[row.address].sha256, row.sha256);
}
return states;
};
try {
report.revision = (
await promisify(execFile)("git", [
"-C",
values.runtime,
"rev-parse",
"HEAD",
])
).stdout.trim();
report.target_sha256 = digest(await readFile(values.target));
const { resolveGhidraProject } = await import(
pathToFileURL(join(values.runtime, "dist/ghidra/GhidraProjectSnapshot.js"))
.href
);
const { ArtifactResourceScope } = await import(
pathToFileURL(
join(values.runtime, "dist/artifacts/ArtifactResourceScope.js"),
).href
);
const fingerprint = async () => {
const resources = new ArtifactResourceScope();
try {
return await resolveGhidraProject(
{
path: values.project,
...(values.program ? { programName: values.program } : {}),
},
{ path: values.target },
resources,
);
} finally {
await resources.close();
}
};
const original = await fingerprint();
report.project_sha256 = original.sha256;
account = await mkdtemp(join(tmpdir(), "rea-directory-investigation-"));
transport = new StdioClientTransport({
command: process.execPath,
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
cwd: values.runtime,
maxBufferSize: 10 * 1024 * 1024,
env: {
PATH: process.env.PATH,
HOME: account,
USERPROFILE: account,
JAVA_HOME: values.java,
GHIDRA_INSTALL_DIR: values.ghidra,
GHIDRA_HEADLESS_MAXMEM: values.heap ?? "1536M",
JAVA_TOOL_OPTIONS: "-XX:ActiveProcessorCount=2",
NODE_OPTIONS: process.env.NODE_OPTIONS ?? "--max-old-space-size=3072",
REA_GHIDRA_PROJECT_PATH: values.project,
...(values.program ? { REA_GHIDRA_PROGRAM_NAME: values.program } : {}),
REA_GHIDRA_STARTUP_TIMEOUT_MS: "600000",
...(values["decompile-seconds"]
? { REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS: values["decompile-seconds"] }
: {}),
REA_MCP_TOOL_FAMILIES: "direct,enhanced,application,session",
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
REA_MCP_OUTPUT_SCHEMA_PROFILE: "compact",
},
stderr: "pipe",
});
const start = transport.start.bind(transport);
transport.start = async () => {
const receive = transport.onmessage;
transport.onmessage = (message) => {
transcript.push({ phase, direction: "response", message });
receive?.(message);
};
await start();
};
const send = transport.send.bind(transport);
transport.send = async (message) => {
transcript.push({ phase, direction: "request", message });
await send(message);
};
serverLog = createWriteStream(join(values.output, "server.log"));
serverLogFinished = finished(serverLog);
serverLogFinished.catch(() => {});
transport.stderr.pipe(serverLog);
client = new Client({ name: "rea-directory-investigation", version: "1" });
await client.connect(transport);
phase = "discovery";
const catalog = await client.listTools({}, { timeout: 15000 });
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
report.tools = catalog.tools.length;
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
const call = async (name, args, label) => {
phase = label;
const at = performance.now();
const result = await client.callTool(
{ name, arguments: args },
{ timeout: 900000, toolDefinition: definitions.get(name) },
);
report.calls.push({
name,
phase,
elapsed_ms: performance.now() - at,
response_bytes: Buffer.byteLength(JSON.stringify(result)),
is_error: result.isError ?? false,
});
assert.notEqual(
result.isError,
true,
`${name}: ${JSON.stringify(result).slice(0, 800)}`,
);
return result.structuredContent;
};
const open = () =>
call("open_binary", { path: values.target, provider_id: "ghidra" }, "open");
await open();
const first = await call(
"decompile_all",
{ output_directory: tree, limit: chunk },
"first-chunk",
);
report.first = first.normalized_result;
const firstFiles = await fileStates(await readIndex());
await call("close_binary", {}, "close-first-session");
await open();
const remaining = await call(
"decompile_all",
{ output_directory: tree, resume: true },
"cross-session-resume",
);
report.resume = remaining.normalized_result;
report.resume_evidence_links = remaining.evidence_links?.length ?? 0;
let index = await readIndex();
const allFiles = await fileStates(index);
for (const [address, state] of Object.entries(firstFiles))
assert.deepEqual(allFiles[address], state);
report.first_files_unchanged = true;
const settled = await call(
"decompile_all",
{ output_directory: tree, resume: true },
"verified-resume",
);
report.verified_resume = settled.normalized_result;
if (report.resume.counts.failed === 0)
assert.equal(report.verified_resume.counts.written, 0);
assert.equal(report.verified_resume.counts.remaining, 0);
index = await readIndex();
const settledFiles = await fileStates(index);
for (const [address, state] of Object.entries(allFiles))
assert.deepEqual(settledFiles[address], state);
assert.equal(index.functions.length, report.resume.counts.selected);
const facts = [];
for (const row of index.functions) {
const fact = {
address: row.address,
name: row.name,
status: row.status,
body: row.body,
};
if (row.status === "written") {
const text = await readFile(join(tree, row.relative_path), "utf8");
assert.ok(text.includes(`// address: ${row.address}\n`));
assert.ok(
text.includes(
`// analysis_profile: ${index.analysis_profile_digest}\n`,
),
);
fact.pseudocode_sha256 = digest(text.slice(text.indexOf("\n\n") + 2));
}
facts.push(fact);
}
report.function_count = facts.length;
report.written = facts.filter((row) => row.status === "written").length;
report.failed = facts.filter((row) => row.status === "failed").length;
report.external = facts.filter((row) => row.status === "external").length;
report.facts_sha256 = digest(JSON.stringify(facts));
await writeFile(join(values.output, "facts.json"), JSON.stringify(facts));
const evidencePath = join(values.output, "complete-evidence.json");
await call(
"export_evidence_bundle",
{ path: evidencePath },
"complete-evidence-export",
);
await call("close_binary", {}, "close");
await client.close();
await transport.close();
const { parseEvidenceBundle } = await import(
pathToFileURL(join(values.runtime, "dist/domain/evidenceBundle.js")).href
);
const bundle = parseEvidenceBundle(
JSON.parse(await readFile(evidencePath, "utf8")),
);
report.evidence_records = bundle.records.length;
const retained = bundle.records.find(
(record) => record.evidence_id === remaining.evidence_id,
);
assert.ok(retained);
report.retained_links = retained.evidence_links.length;
const byId = new Set(bundle.records.map((record) => record.evidence_id));
for (const id of retained.evidence_links) assert.ok(byId.has(id));
report.all_source_links_resolve = true;
if (values.history === "required") {
const historical = bundle.records.filter(
(record) =>
record.operation === "health" &&
record.parameters.facet === "historical-project-analysis-reports",
);
assert.equal(historical.length, 1);
assert.equal(historical[0].normalized_result.origin, "caller-project");
assert.ok(retained.evidence_links.includes(historical[0].evidence_id));
report.historical_source_retained_and_linked = true;
}
if (values.golden) {
const golden = JSON.parse(
await readFile(join(values.golden, "facts.json"), "utf8"),
);
const labels = (rows) =>
rows
.map((row) => [row.address, row.name ?? row.value])
.sort((a, b) => a[0].localeCompare(b[0], "en"));
assert.deepEqual(labels(facts), labels(golden.procedures));
for (const item of golden.pseudocode) {
const fact = facts.find((row) => row.address === item.procedure);
assert.equal(fact?.pseudocode_sha256, digest(item.result));
}
report.fresh_import_inventory_and_sampled_pseudocode_match = true;
}
if (values.control) {
const control = JSON.parse(
await readFile(join(values.control, "facts.json"), "utf8"),
);
assert.deepEqual(facts, control);
report.control_facts_match = true;
}
assert.deepEqual(await fingerprint(), original);
assert.equal(digest(await readFile(values.target)), report.target_sha256);
report.source_unchanged = true;
report.outcome = "passed";
} catch (cause) {
report.outcome = "failed";
report.failure = String(cause);
process.exitCode = 1;
} finally {
report.elapsed_ms = performance.now() - started;
const published = await finalizeRetainedQueryReceipt({
report,
client,
transport,
account,
serverLog,
serverLogFinished,
output: values.output,
transcript,
});
if (!published) process.exitCode = 1;
}
@@ -1,115 +0,0 @@
#!/usr/bin/env python3
"""Check verifier result/exit semantics with controlled replies, not real apps."""
import argparse
import json
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
import textwrap
def write(root, name, source):
path = root / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(textwrap.dedent(source))
def check(arguments, root, case):
shutil.copyfile(arguments.verifier, root / "probe.mjs")
write(root, "package.json", '{"type":"module"}')
write(root, "node_modules/playwright-core/package.json",
'{"type":"module","exports":"./index.js"}')
write(root, "node_modules/playwright-core/index.js", """
let reads = 0;
const page = {
locator: () => ({ waitFor: async () => {} }),
getByText: () => ({ waitFor: async () => {} }),
async evaluate() {
const after = reads++ > 0;
return { title: 'controlled oracle', invalid_file_dialog: !after,
unsaved_changes: after,
diagram_svg_elements: after && process.env.REA_DRAWIO_ORACLE_CASE !== 'insertion-failure' ? 2 : 0,
cells: 0 };
},
};
export const _electron = { async launch() {
return { process: () => ({exitCode: 0, signalCode: null}),
firstWindow: async () => page, close: async () => {} };
}};
""")
write(root, "dist/browser/PlaywrightElectronActiveProvider.js", """
export class PlaywrightElectronActiveProvider {
constructor(environment, launch) { this.launch = launch; }
}
""")
write(root, "dist/browser/PlaywrightElectronActiveActions.js", """
export const runWithExecutionLimits = operation => operation;
""")
write(root, "dist/application/javascript/ElectronActiveObservationService.js", """
export async function captureElectronScenario(provider, input) {
const app = await provider.launch({});
await app.close();
const name = process.env.REA_DRAWIO_ORACLE_CASE;
const capture = { actions: input.actions.map(action => ({...action,
status: name === 'action-failure' ? 'failed' : 'completed'})),
coverage: {}, retention: {}, ipc: {events: [], observed: 0, dropped: 0} };
if (name === 'no-capture') return {ok: false, error: {reason: 'startup_failed'}};
if (name === 'partial') return {ok: false, error: {reason: 'cleanup_failed',
partialObservation: {capture}}};
return {ok: true, value: {evidence_id: 'ev_controlled',
normalized_result: capture}};
}
""")
result = subprocess.run(
[arguments.node, str(root / "probe.mjs"), "--runtime-root", str(root),
"--executable", arguments.node, "--application", str(root),
"--profile-root", str(root), "--dismiss-error", "--selector", "controlled"],
env={**os.environ, "REA_DRAWIO_ORACLE_CASE": case},
capture_output=True, text=True, timeout=15,
)
replies = []
for line in result.stdout.splitlines():
try:
replies.append(json.loads(line))
except ValueError:
pass
receipt = next((value for value in replies if "workflow_status" in value), None)
expected_success = case == "complete"
expected_provider = case not in ("partial", "no-capture")
expected_actions = case not in ("action-failure", "no-capture")
expected_inserted = case != "insertion-failure"
matched = (
(result.returncode == 0) is expected_success
and receipt is not None
and receipt["provider_ok"] is expected_provider
and receipt["cleanup_verified"] is expected_provider
and receipt["actions_completed"] is expected_actions
and receipt["shape_inserted"] is expected_inserted
and (receipt["workflow_status"] == "verified") is expected_success
)
return {"case": case, "exit_code": result.returncode,
"expected_success": expected_success, "summary_present": receipt is not None,
"matched_expected": matched}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--node", required=True)
parser.add_argument("--verifier", type=Path,
default=Path(__file__).with_name("drawio-electron-lifecycle.mjs"))
arguments = parser.parse_args()
receipts = []
with tempfile.TemporaryDirectory(prefix="rea-drawio-oracles-") as directory:
for case in ("complete", "partial", "action-failure", "insertion-failure", "no-capture"):
root = Path(directory) / case
root.mkdir()
receipts.append(check(arguments, root, case))
print(json.dumps({"oracle_cases": receipts, "real_provider_verified": False}))
assert all(receipt["matched_expected"] for receipt in receipts), receipts
if __name__ == "__main__":
main()
@@ -1,191 +0,0 @@
#!/usr/bin/env node
// Real draw.io ASAR, renderer and declared actions; DOM reads are independent
// verification probes. No fabricated IPC channels or replacement app callbacks.
import assert from "node:assert/strict";
import { mkdtemp, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { _electron } from "playwright-core";
const { values } = parseArgs({
options: {
"runtime-root": { type: "string" },
executable: { type: "string" },
application: { type: "string" },
"profile-root": { type: "string" },
selector: { type: "string" },
"dismiss-error": { type: "boolean", default: false },
},
});
for (const key of ["runtime-root", "executable", "application", "profile-root"])
assert.ok(values[key], `Missing --${key}`);
const runtime = resolve(values["runtime-root"]);
const { PlaywrightElectronActiveProvider } = await import(
pathToFileURL(
join(runtime, "dist/browser/PlaywrightElectronActiveProvider.js"),
)
);
const { captureElectronScenario } = await import(
pathToFileURL(
join(
runtime,
"dist/application/javascript/ElectronActiveObservationService.js",
),
)
);
const { runWithExecutionLimits } = await import(
pathToFileURL(
join(runtime, "dist/browser/PlaywrightElectronActiveActions.js"),
)
);
const profile = await mkdtemp(
join(resolve(values["profile-root"]), "drawio-ui-"),
);
const controller = new AbortController();
const timer = setTimeout(
() => controller.abort(new Error("Verifier deadline")),
40_000,
);
let application;
let child;
let page;
let before;
let after;
let checkedFinalState = false;
const inspect = () =>
page.evaluate(() => {
const diagram = document.querySelector(".geDiagramContainer svg");
return {
title: document.title,
invalid_file_dialog:
document.body.innerText.includes("Invalid file data"),
unsaved_changes: document.body.innerText.includes("Unsaved changes"),
diagram_svg_elements:
diagram?.querySelectorAll("rect,path,ellipse,text").length ?? null,
cells: document.querySelectorAll(".geDiagramContainer [data-cell-id]")
.length,
};
});
const started = performance.now();
try {
const actions = values["dismiss-error"]
? [
{
step_id: "dismiss-invalid-file",
kind: "click",
window_index: 0,
selector: 'button:has-text("OK"):visible',
},
]
: [];
if (values.selector !== undefined)
actions.push({
step_id: "insert-shape",
kind: "click",
window_index: 0,
selector: values.selector,
});
actions.push({
step_id: "settle-renderer",
kind: "wait",
window_index: 0,
duration_ms: 500,
});
const result = await captureElectronScenario(
new PlaywrightElectronActiveProvider(process.env, async (options) => {
application = await _electron.launch(options);
child = application.process();
page = await application.firstWindow({ timeout: 15_000 });
await page
.locator(".geDiagramContainer")
.waitFor({ state: "visible", timeout: 15_000 });
if (values["dismiss-error"])
await page
.getByText("Invalid file data", { exact: true })
.waitFor({ state: "visible", timeout: 15_000 });
before = await inspect();
return new Proxy(application, {
get(target, key) {
if (key === "close")
return async () => {
if (!checkedFinalState) {
checkedFinalState = true;
after = await inspect();
}
return target.close();
};
const value = Reflect.get(target, key, target);
return typeof value === "function" ? value.bind(target) : value;
},
});
}),
{
executable_path: resolve(values.executable),
application_path: resolve(values.application),
args: ["--no-sandbox", "--disable-gpu", `--user-data-dir=${profile}`],
actions,
},
{ signal: controller.signal },
);
const capture = result.ok
? result.value.normalized_result
: result.error.partialObservation?.capture;
await writeFile(join(profile, "result.json"), JSON.stringify(result));
const actionsCompleted =
capture?.actions.every((action) => action.status === "completed") === true;
const inserted =
values.selector === undefined
? null
: after?.diagram_svg_elements > before?.diagram_svg_elements;
console.log(
JSON.stringify({
profile,
runtime,
elapsed_ms: performance.now() - started,
deadline_triggered: controller.signal.aborted,
selected_shape: values.selector ?? null,
before,
after,
actions_completed: actionsCompleted,
shape_inserted: inserted,
provider_ok: result.ok,
cleanup_verified: result.ok,
reason: result.ok ? null : result.error.reason,
coverage: capture?.coverage,
retention: capture?.retention,
ipc: capture?.ipc,
actions: capture?.actions,
workflow_status:
result.ok && actionsCompleted && inserted !== false
? "verified"
: "partial-or-failed",
}),
);
assert.ok(actionsCompleted, "Declared real-app actions did not complete");
assert.ok(before && after, "Real renderer state was not verified");
assert.notEqual(inserted, false, "Click did not insert a diagram shape");
if (values["dismiss-error"]) assert.equal(after.invalid_file_dialog, false);
assert.equal(
controller.signal.aborted,
false,
"Verifier deadline was reached",
);
assert.ok(result.ok, "Provider cleanup remains unverified");
} finally {
clearTimeout(timer);
if (
application !== undefined &&
child?.exitCode === null &&
child?.signalCode === null
)
await runWithExecutionLimits(
application.close(),
undefined,
performance.now() + 5_000,
).catch(() => {
process.exitCode = 1;
});
// The resource scope owns any unresolved descendants; no PID/path-only kill.
if (child?.exitCode === null && child?.signalCode === null) process.exit(1);
}
@@ -1,154 +0,0 @@
#!/usr/bin/env python3
"""Check verifier exit semantics with controlled replies, not real providers/apps.
The JS case owns a short-lived Node child so the main process has exited while
the provider reply still reports incomplete cleanup. The CLI cases substitute
only Popen. These are negative-oracle regressions, not runtime compatibility or
performance evidence; real Joplin trials remain separately required.
"""
import argparse
import contextlib
import importlib.util
import io
import json
from pathlib import Path
import shutil
import subprocess
import tempfile
import textwrap
from unittest.mock import patch
def write(root, name, text):
path = root / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(textwrap.dedent(text))
def check_js(arguments, root, complete):
shutil.copyfile(arguments.javascript, root / "probe.mjs")
write(root, "package.json", '{"type":"module"}')
write(root, "node_modules/playwright-core/package.json",
'{"type":"module","exports":"./index.js"}')
write(root, "node_modules/playwright-core/index.js", """
import { spawn } from 'node:child_process';
import { once } from 'node:events';
export const _electron = { async launch() {
const child = spawn(process.execPath, ['-e', 'setTimeout(() => {}, 30000)'],
{ stdio: 'ignore' });
await once(child, 'spawn');
return { process: () => child, async close() {
if (child.exitCode !== null || child.signalCode !== null) return;
const exited = once(child, 'exit'); child.kill(); await exited;
}};
}};
""")
write(root, "dist/browser/PlaywrightElectronActiveProvider.js", """
export class PlaywrightElectronActiveProvider {
constructor(environment, launch) { this.launch = launch; }
}
""")
write(root, "dist/browser/PlaywrightElectronActiveActions.js", """
export const runWithExecutionLimits = operation => operation;
""")
write(root, "dist/application/javascript/ElectronActiveObservationService.js", f"""
import {{ once }} from 'node:events';
export async function captureElectronScenario(provider, input, options) {{
const app = await provider.launch({{}});
try {{
void app.firstWindow();
await once(options.signal, 'abort');
}} finally {{ await app.close(); }}
const error = Object.assign(new Error('controlled cancellation'), {{
_tag: 'BrowserObservationError', reason: 'cancelled',
userCategory: 'cancelled', cleanupIncomplete: {str(not complete).lower()},
partialObservation: {{capture: {{application: {{}},
actions: [{{status: 'cancelled'}}], coverage: {{}}, windows: [], limitations: []}}}}
}});
return {{ok: false, error}};
}}
""")
result = subprocess.run(
[arguments.node, str(root / "probe.mjs"), "--runtime-root", str(root),
"--executable", arguments.node, "--application", str(root),
"--profile-root", str(root), "--case", "window-wait"],
capture_output=True, text=True, timeout=15,
)
receipts = [json.loads(line) for line in result.stdout.splitlines()]
receipt = next(value for value in receipts if "cleanup_phase" in value)
assert receipt["launched_process_exited"] is True
assert receipt["cleanup_phase"] == ("verified" if complete else "unverified")
assert (result.returncode == 0) is complete, (result.returncode, result.stdout, result.stderr)
return {"verifier": "shared", "controlled_cleanup_complete": complete,
"main_exited": True, "exit_code": result.returncode}
def check_cli(arguments, root, code, category, cleanup):
spec = importlib.util.spec_from_file_location("cli_cancellation_probe", arguments.python)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
error = {"code": code, "category": category, "details": {"partial_observation": {
"kind": "electron-active-observation", "capture": {
"application": {"cleanup": cleanup}, "actions": [{"status": "cancelled"}]}}}}
class ExitedCLI:
def __init__(self, command, **options):
request = json.loads(command[3])
(Path(request["args"][-1]) / "log-main-process.txt").touch()
self.returncode = None
self.output = options["stdout"]
def poll(self):
return self.returncode
def wait(self, timeout):
raise subprocess.TimeoutExpired("controlled CLI", timeout)
def send_signal(self, signal):
self.output.write(json.dumps(error).encode())
self.output.flush()
self.returncode = 143
supplied = argparse.Namespace(profile_root=root, node=arguments.node,
runtime_root=root, executable=Path(arguments.node), application=root)
output = io.StringIO()
passed = True
with patch.object(module.subprocess, "Popen", ExitedCLI), contextlib.redirect_stdout(output):
try:
module.run(supplied)
except AssertionError:
passed = False
expected = code == "cancelled" and category == "cancelled" and cleanup == "terminated-owned-process"
assert passed is expected, (code, category, cleanup, output.getvalue())
receipt = json.loads(output.getvalue())
assert receipt["exit_code"] == 143
return {"verifier": "cli", "code": code, "category": category,
"application_cleanup": cleanup, "accepted": passed}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--node", required=True)
parser.add_argument("--javascript", type=Path, default=Path(__file__).with_name("electron-cancellation.mjs"))
parser.add_argument("--python", type=Path, default=Path(__file__).with_name("joplin-cli-cancellation.py"))
arguments = parser.parse_args()
receipts = []
with tempfile.TemporaryDirectory(prefix="rea-cancellation-oracles-") as directory:
root = Path(directory)
for complete in (False, True):
case = root / str(complete)
case.mkdir()
receipts.append(check_js(arguments, case, complete))
for code, category, cleanup in [
("cleanup_incomplete", "cancelled", "unverified"),
("cleanup_incomplete", "cancelled", "terminated-owned-process"),
("cancelled", "execution_failure", "terminated-owned-process"),
("cancelled", "cancelled", "terminated-owned-process"),
]:
receipts.append(check_cli(arguments, root, code, category, cleanup))
print(json.dumps({"oracle_cases": receipts, "real_provider_verified": False}))
if __name__ == "__main__":
main()
@@ -1,273 +0,0 @@
#!/usr/bin/env node
// Real applications run unchanged. Fault cases hold a REA-side Playwright reply;
// they do not establish that the application naturally exhibits that failure.
import assert from "node:assert/strict";
import { mkdtemp, readFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { _electron } from "playwright-core";
const { values } = parseArgs({
options: {
executable: { type: "string" },
application: { type: "string" },
"profile-root": { type: "string" },
"runtime-root": { type: "string" },
case: { type: "string", default: "normal-wait" },
},
});
const cases = ["normal-wait", "window-wait", "initial-state", "final-state"];
assert.equal(
process.platform,
"linux",
"This verifier checks process identity through Linux /proc",
);
assert.ok(cases.includes(values.case));
assert.ok(values.executable && values.application && values["profile-root"]);
const runtime = resolve(
values["runtime-root"] ?? fileURLToPath(new URL("../..", import.meta.url)),
);
const { PlaywrightElectronActiveProvider } = await import(
pathToFileURL(
join(runtime, "dist/browser/PlaywrightElectronActiveProvider.js"),
)
);
const { captureElectronScenario } = await import(
pathToFileURL(
join(
runtime,
"dist/application/javascript/ElectronActiveObservationService.js",
),
)
);
const { runWithExecutionLimits } = await import(
pathToFileURL(
join(runtime, "dist/browser/PlaywrightElectronActiveActions.js"),
)
);
const profile = await mkdtemp(join(resolve(values["profile-root"]), "joplin-"));
const controller = new AbortController();
const reason = new Error(`real-application cancellation: ${values.case}`);
const started = performance.now();
let application;
let child;
let initialBirth;
let evaluateCalls = 0;
let injected = false;
let cancelledAt;
let cancelTimer;
let watchdog;
let rejectDeadline;
const deadline = new Promise((_, reject) => {
rejectDeadline = reject;
});
const birth = async (pid) => {
try {
const text = await readFile(`/proc/${pid}/stat`, "utf8");
const fields = text.slice(text.lastIndexOf(")") + 2).split(" ");
return fields[0] === "Z" ? null : fields[19];
} catch (cause) {
if (cause.code === "ENOENT") return null;
throw cause;
}
};
const armCancellation = () => {
if (cancelTimer !== undefined) return;
cancelTimer = setTimeout(
() => {
cancelledAt = performance.now();
controller.abort(reason);
watchdog = setTimeout(
() =>
rejectDeadline(
new Error("Cancellation did not settle within 20 seconds"),
),
20_000,
);
},
values.case === "normal-wait" ? 1_000 : 100,
);
};
const launch = async (options) => {
application = await _electron.launch(options);
child = application.process();
initialBirth = await birth(child.pid);
assert.ok(initialBirth, "Launched process identity is unavailable");
return new Proxy(application, {
get(target, key) {
if (key === "evaluate")
return (...args) => {
evaluateCalls += 1;
if (
(values.case === "initial-state" && evaluateCalls === 1) ||
(values.case === "final-state" && evaluateCalls === 5)
) {
injected = true;
armCancellation();
return new Promise(() => {});
}
const result = target.evaluate(...args);
if (values.case === "normal-wait" && evaluateCalls === 4)
void result.then(armCancellation, () => undefined);
return result;
};
if (key === "firstWindow" && values.case === "window-wait")
return () => {
injected = true;
armCancellation();
return new Promise(() => {});
};
const value = Reflect.get(target, key, target);
return typeof value === "function" ? value.bind(target) : value;
},
});
};
let record;
try {
const input = {
executable_path: resolve(values.executable),
application_path: resolve(values.application),
args: [
"--no-sandbox",
"--disable-gpu",
`--user-data-dir=${profile}/chromium`,
"--profile",
profile,
],
actions: values.case.endsWith("state")
? []
: [{ step_id: "settle", kind: "wait", duration_ms: 60_000 }],
};
const result = await Promise.race([
captureElectronScenario(
new PlaywrightElectronActiveProvider(process.env, launch),
input,
{ signal: controller.signal },
),
deadline,
]);
assert.equal(result.ok, false, "Expected a typed cancellation");
const error = result.error;
const partial = error.partialObservation?.capture;
const causes = [];
const seen = new Set();
for (
let cause = error;
cause instanceof Error && !seen.has(cause);
cause = cause.cause
) {
seen.add(cause);
causes.push({
tag: cause._tag ?? null,
reason: cause.reason ?? null,
message: cause.message,
diagnostics: cause.diagnostics ?? null,
});
}
record = {
case: values.case,
fault_injected: injected,
input,
profile,
evaluate_calls: evaluateCalls,
elapsed_ms: performance.now() - started,
cancellation_to_result_ms:
cancelledAt === undefined ? null : performance.now() - cancelledAt,
tag: error._tag,
reason: error.reason,
user_category: error.userCategory,
diagnostics: error.diagnostics ?? null,
message: error.message,
user_message: error.userMessage ?? null,
cleanup: error.cleanup ?? null,
cause: error.cause instanceof Error ? error.cause.message : null,
causes,
cleanup_incomplete: error.cleanupIncomplete,
partial:
partial === undefined
? null
: {
application: partial.application,
actions: partial.actions,
coverage: partial.coverage,
window_count: partial.windows.length,
limitations: partial.limitations,
},
launched_process_exited:
child !== undefined && (await birth(child.pid)) !== initialBirth,
};
assert.equal(controller.signal.aborted, true);
assert.equal(error.userCategory, "cancelled");
if (values.case !== "normal-wait") assert.equal(injected, true);
if (values.case.endsWith("wait"))
assert.ok(partial?.actions.some((action) => action.status === "cancelled"));
if (values.case === "initial-state") assert.equal(partial, undefined);
if (values.case === "final-state") {
assert.ok(
partial?.limitations.some((text) =>
text.includes("initial pre-action snapshot"),
),
);
assert.ok(
causes.some(
(cause) =>
cause.diagnostics?.final_state_read === "cancellation_grace_expired",
),
);
}
// Cleanup is a separate oracle. A known Linux lineage limitation must remain
// visible, rather than turning cancellation coverage into a cleanup claim.
record.cancellation_phase = "verified";
record.cleanup_phase =
record.launched_process_exited && !error.cleanupIncomplete
? "verified"
: "unverified";
record.workflow_status =
record.cleanup_phase === "verified" ? "verified" : "partial";
console.log(JSON.stringify(record));
assert.equal(
record.cleanup_phase,
"verified",
"Provider cleanup remains unverified; cancellation alone is a partial result",
);
} catch (cause) {
console.log(
JSON.stringify({
case: values.case,
fault_injected: injected,
evaluate_calls: evaluateCalls,
elapsed_ms: performance.now() - started,
cancellation_to_failure_ms:
cancelledAt === undefined ? null : performance.now() - cancelledAt,
completed: false,
observed: record ?? null,
verifier_error: cause instanceof Error ? cause.message : String(cause),
}),
);
throw cause;
} finally {
clearTimeout(cancelTimer);
clearTimeout(watchdog);
if (
application !== undefined &&
child?.exitCode === null &&
child?.signalCode === null
)
await runWithExecutionLimits(
application.close(),
undefined,
performance.now() + 5_000,
).catch((cause) => {
console.error(
JSON.stringify({
verifier_cleanup: "unverified",
message: cause.message,
}),
);
process.exitCode = 1;
});
// The external resource scope owns any leftovers; do not leave a failed
// verifier awaiting an application whose ownership the provider cannot prove.
if (child?.exitCode === null && child?.signalCode === null) process.exit(1);
}
@@ -1,89 +0,0 @@
#!/usr/bin/env python3
"""Serial native/hooked producer controls; not real-application evidence."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
def sha256(path):
with path.open("rb") as stream:
return hashlib.file_digest(stream, "sha256").hexdigest()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--hook-root", type=Path, required=True)
parser.add_argument("--node", type=Path, action="append", required=True)
parser.add_argument("--electron", type=Path)
parser.add_argument("--output", type=Path, required=True)
arguments = parser.parse_args()
arguments.output.mkdir(parents=True, exist_ok=True)
packet = Path(tempfile.mkdtemp(prefix="native-once-", dir=arguments.output))
stage = packet / "staged"
module = stage / "node_modules/electron"
module.mkdir(parents=True)
fixture = module / "index.js"
fixture.write_text(
'const { EventEmitter } = require("node:events");\n'
"module.exports = { app: new EventEmitter(), ipcMain: new EventEmitter() };\n"
)
sources = [
arguments.hook_root / "scripts/electron-active-hook.cjs",
arguments.hook_root / "scripts/electron-active-hook-boundaries.cjs",
Path(__file__).with_name("electron-once-runtime.mjs"),
]
pins = {path.name: sha256(path) for path in sources}
pins["electron-fixture"] = sha256(fixture)
for source in sources:
shutil.copyfile(source, stage / source.name)
engines = [(engine, False) for engine in arguments.node]
if arguments.electron is not None:
engines.append((arguments.electron, True))
results = []
for index, (engine, run_as_node) in enumerate(engines):
selected = engine.absolute()
environment = dict(os.environ)
if run_as_node:
environment["ELECTRON_RUN_AS_NODE"] = "1"
execution = subprocess.run(
[str(selected), str(stage / sources[-1].name), str(stage / sources[0].name)],
env=environment,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=15,
check=False,
)
(packet / f"engine-{index}.stdout").write_bytes(execution.stdout)
(packet / f"engine-{index}.stderr").write_bytes(execution.stderr)
receipt = {
"selected_binary": str(engine),
"absolute_binary": str(selected),
"binary_sha256": sha256(selected),
"electron_run_as_node": run_as_node,
"exit_code": execution.returncode,
"stdout_sha256": hashlib.sha256(execution.stdout).hexdigest(),
"stderr_sha256": hashlib.sha256(execution.stderr).hexdigest(),
}
if execution.returncode == 0:
receipt["result"] = json.loads(execution.stdout)
results.append(receipt)
report = {"hook_root": str(arguments.hook_root), "source_sha256": pins,
"engines": results, "scope": __doc__}
(packet / "report.json").write_text(json.dumps(report, indent=2) + "\n")
if execution.returncode != 0:
raise RuntimeError(f"Engine failed; preserved receipt at {packet}")
print(json.dumps({"packet": str(packet), "engines": [
{"node": row["result"]["node"], "electron": row["result"]["electron"],
"cases": row["result"]["cases"],
"disagreements": row["result"]["disagreements"]} for row in results
]}))
if __name__ == "__main__":
main()
@@ -1,111 +0,0 @@
"""Verify completed BEAM selection/export tasks and count a declared token projection."""
import argparse
import hashlib
import json
from pathlib import Path
import statistics
import tiktoken
def compact(value):
return json.dumps(value, ensure_ascii=False, separators=(',', ':'), allow_nan=False)
def digest(path):
return hashlib.sha256(path.read_bytes()).hexdigest()
def inspect(path, encoding):
report = json.loads((path / 'report.json').read_text())
catalog = json.loads((path / 'catalog.json').read_text())
transcript = json.loads((path / 'transcript.json').read_text())
scope = next(json.loads(line) for line in reversed(path.with_suffix('.log').read_text().splitlines())
if line.startswith('{"resource_scope":'))
observed = scope.get('payload_resource_observation') or {}
events = observed.get('resource_events', {})
assert report['status'] == 'passed' and report['runtime_identity_sha256']
assert report['complete_source_export_verified'] is True
assert not report['cleanup_failures'] and not report['receipt_failures']
assert scope['abort'] is None and scope['payload_resource_observation_verified'] is True
assert observed['payload_exit_code'] == 0 and observed['payload_cleanup_verified'] is True
assert all(events.get('memory.events', {}).get(key) == 0 for key in ['high', 'max', 'oom', 'oom_kill'])
assert events.get('pids.events', {}).get('max') == 0
assert len(catalog['tools']) == report['catalog_tools'] == 221
assert transcript[0]['name'] == 'inspect_erlang_beam'
assert transcript[-1]['name'] == 'export_evidence_bundle'
assert all(event['name'] == 'inspect_analysis_view' for event in transcript[1:-1])
assert all('reply' in event and not event['reply'].get('isError', False) for event in transcript)
count = lambda text: len(encoding.encode(text, disallowed_special=()))
directory = [{key: tool[key] for key in ['name', 'description', 'inputSchema']} for tool in catalog['tools']]
request = transcript[0]['input']
question = report['question']
assert ('detail' in request) == (report['lane'] == 'selected')
arguments = sum(count(compact({'name': event['name'], 'arguments': event['input']})) for event in transcript)
replies = sum(count(block['text']) for event in transcript for block in event['reply']['content'] if block['type'] == 'text')
inputs, prompt = count(compact(directory)), count(question)
instructions = count(report['server_instructions'])
return dict(case=path.name, lane=report['lane'], artifact=report['artifact'],
answer_count=report['answer_count'], answer_sha256=report['answer_sha256'],
parent_evidence_id=report['parent_evidence_id'], source_report_sha256=report['source_report_sha256'],
runtime_identity_sha256=report['runtime_identity_sha256'], catalog_sha256=digest(path / 'catalog.json'),
calls=report['calls'], phases=report['phases'], wall_seconds=scope['elapsed_seconds'],
cpu_seconds=observed['cpu_stat']['usage_usec'] / 1e6,
kernel_family_peak_bytes=observed['memory_peak_bytes'],
events_at_payload_exit=events, payload_cleanup_verified=True,
final_whole_scope_events=scope.get('resource_events_final'),
token_proxy=dict(question=prompt, server_instructions=instructions, input_catalog_once=inputs, arguments=arguments,
text_replies=replies, combined=prompt + instructions + inputs + arguments + replies,
complete_transport_catalog=count(compact(catalog))),
receipt_sha256={file.name: digest(file) for file in [path / 'report.json', path / 'catalog.json',
path / 'transcript.json', path / 'evidence.json', path.with_suffix('.log')]})
def summarize(rows):
result = {}
for artifact in sorted({row['artifact']['sha256'] for row in rows}):
cases = [row for row in rows if row['artifact']['sha256'] == artifact]
for field in ['answer_count', 'answer_sha256', 'parent_evidence_id', 'source_report_sha256']:
assert len({row[field] for row in cases}) == 1, field
result[artifact] = {}
for lane in ['complete', 'selected']:
samples = [row for row in cases if row['lane'] == lane]
assert len(samples) == 3
for field in ['runtime_identity_sha256', 'catalog_sha256']:
assert len({row[field] for row in samples}) == 1, field
metrics = {key: [row[key] for row in samples] for key in ['wall_seconds', 'cpu_seconds', 'kernel_family_peak_bytes']}
metrics['projected_tokens'] = [row['token_proxy']['combined'] for row in samples]
metrics['mcp_reply_bytes'] = [sum(call['reply_bytes'] for call in row['calls']) for row in samples]
result[artifact][lane] = {key: dict(median=statistics.median(values), minimum=min(values), maximum=max(values))
for key, values in metrics.items()}
return result
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--cohort', type=Path, required=True)
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
manifest = json.loads((args.cohort / 'manifest.json').read_text())
index = json.loads((args.cohort / 'index.json').read_text())
assert len(index) == 18 and all(case['exit'] == 0 for case in index)
for file, expected in manifest['driver_files'].items():
assert digest(Path(file)) == expected, file
identities = {}
for lane, path in manifest['runtime_identities'].items():
identity = json.loads(Path(path).read_text())
assert identity['source_clean'] is True
identities[lane] = dict(sha256=digest(Path(path)), source_revision=identity['source_revision'])
encoding = tiktoken.get_encoding('cl100k_base')
rows = [inspect(args.cohort / case['name'], encoding) for case in index]
assert all(row['runtime_identity_sha256'] == identities[row['lane']]['sha256'] for row in rows)
receipt = dict(tiktoken=tiktoken.__version__, encoding=encoding.name, actual_model_usage=None,
cohort=manifest, commands=index, runtime_identities=identities,
projection='Scripted fresh SDK tasks, not model runs. One fixed question, server instructions and complete input directory (name/description/inputSchema), every argument and MCP text reply including export. Raw catalog including output schemas reported separately. No framing, host injection, history replay, inference quality or billable usage measured.',
resources='CPU quota100%, payload affinity0-1, initial observer15. Early stop3GiB, hard4GiB, swap0. Fresh HOME/client/server; Node compile cache disabled, OS caches unflushed. Whole process-family CPU and kernel memory peak include file cache, runtime identity checks, hashing, assertions, SDK/server serialization and cleanup. Oracle generation and tokenization outside timed tasks. Payload-exit counters verified; final whole-scope counters unavailable.',
summary=summarize(rows), recordings=rows)
args.output.write_text(json.dumps(receipt, ensure_ascii=False, indent=2) + '\n')
if __name__ == '__main__':
main()
@@ -1,108 +0,0 @@
"""Independent struct oracle for the fixed public OTP 27 benchmark artifacts.
Reads bytes only. It deliberately refuses compact atom layouts; format-wide
coverage belongs to the production reader and its separate regressions.
"""
from pathlib import Path
import getopt
import hashlib
import json
import struct
import sys
def decode(path):
data = path.read_bytes()
assert len(data) <= 64 * 1024 * 1024
u32 = lambda at: struct.unpack_from('>I', data, at)[0]
assert data[:4] == b'FOR1' and data[8:12] == b'BEAM'
assert u32(4) + 8 == len(data)
chunks, selected, at = [], {}, 12
while at < len(data):
name = data[at:at + 4].decode('ascii')
size = u32(at + 4)
end = at + 8 + size
padded = (end + 3) & ~3
assert end <= padded <= len(data)
chunks.append(dict(index=len(chunks), name=name,
location=dict(offset=at, bytes=padded - at),
payload=dict(location=dict(offset=at + 8, bytes=size),
bytes=size, sha256=hashlib.sha256(data[at + 8:end]).hexdigest())))
selected[name] = (at + 8, end)
at = padded
assert at == len(data)
atom_name = 'AtU8' if 'AtU8' in selected else 'Atom'
start, end = selected[atom_name]
n = u32(start)
assert n < 65536, 'Fixed OTP 27 oracle does not decode compact atom tables'
at, atoms = start + 4, []
for i in range(n):
size = data[at]
at += 1
name = data[at:at + size].decode('utf-8' if atom_name == 'AtU8' else 'latin1')
atoms.append(dict(index=i + 1, name=name))
at += size
assert at == end
def atom(i):
return atoms[i - 1]['name'] if 0 < i <= len(atoms) else None
def table(name, keys):
if name not in selected:
return None
start, end = selected[name]
if name == 'FunT' and start == end:
return None
n, width = u32(start), len(keys)
assert start + 4 + n * 4 * width == end
rows = []
for i in range(n):
values = struct.unpack_from('>' + 'I' * width, data, start + 4 + i * width * 4)
row = dict(zip(keys, values))
if 'module_index' in row:
row['module'] = atom(row['module_index'])
row['function'] = atom(row['function_index'])
rows.append(row)
return rows
exports = table('ExpT', ['function_index', 'arity', 'label'])
imports = table('ImpT', ['module_index', 'function_index', 'arity'])
lambdas = table('FunT', ['function_index', 'arity', 'label', 'index', 'free_vars', 'old_unique'])
code = None
if 'Code' in selected:
start, end = selected['Code']
size = u32(start)
assert size >= 16 and start + 4 + size <= end
code = dict(zip(['instruction_set', 'opcode_max', 'labels', 'functions'],
struct.unpack_from('>IIII', data, start + 4)))
code.update(extra_header_bytes=size - 16,
instructions=dict(offset=start + 4 + size, bytes=end - start - 4 - size))
return dict(artifact=dict(path=str(path), sha256=hashlib.sha256(data).hexdigest(), bytes=len(data)),
form=dict(location=dict(offset=0, bytes=len(data)), size=len(data) - 8),
chunks=chunks, atoms=atoms, atom_encoding='utf8' if atom_name == 'AtU8' else 'latin1',
exports=exports, imports=imports, lambdas=lambdas, code=code,
module=atoms[0]['name'],
coverage=dict(status='complete', examined_chunks=len(chunks), atoms=len(atoms)))
def main():
options, remaining = getopt.getopt(sys.argv[1:], '', ['input=', 'output=', 'name='])
assert not remaining
inputs = [v for k, v in options if k == '--input']
outputs = [v for k, v in options if k == '--output']
names = [v for k, v in options if k == '--name']
assert len(inputs) == len(outputs) == 1 and names
path = Path(inputs[0])
assert path.is_absolute()
expected = decode(path)
answer = [dict(source_index=i, **row) for i, row in enumerate(expected['exports'] or [])
if row['function'] in names]
result = dict(oracle='Independent Python struct decoding of fixed OTP27 bytes; no BEAM execution. '
'Every table, chunk range/hash and Code header checked. Other encodings/malformed inputs unqualified.',
source='https://www.erlang.org/doc/apps/stdlib/beam_lib.html',
expected=expected, names=names, answer=answer)
Path(outputs[0]).write_text(json.dumps(result, ensure_ascii=False, indent=2) + '\n')
if __name__ == '__main__':
main()
@@ -1,534 +0,0 @@
{
"method": "3alternating freshsessions perlane; SDK cachesentirecatalog thenusesdefaultvalidator withouttoolDefinition override. Diagnostic client startup control, separate from18exactnamedtasks. Samequestion/fullsourceexports/oracle; allpayloadexit/cleanupverified.",
"summary": {
"716d78394b7c3da983e36aac7aadd6673e5041677ba7de20413cc2704d958d55": {
"complete": {
"wall_seconds": {
"median": 19.764,
"minimum": 17.257,
"maximum": 21.577
},
"cpu_seconds": {
"median": 16.276557,
"minimum": 15.287268,
"maximum": 16.561897
},
"kernel_family_peak_bytes": {
"median": 1073950720,
"minimum": 1041948672,
"maximum": 1078980608
},
"projected_tokens": {
"median": 76907,
"minimum": 76907,
"maximum": 76907
},
"mcp_reply_bytes": {
"median": 147003,
"minimum": 147003,
"maximum": 147003
}
},
"selected": {
"wall_seconds": {
"median": 19.375,
"minimum": 18.052,
"maximum": 22.397
},
"cpu_seconds": {
"median": 16.867381,
"minimum": 16.139353,
"maximum": 17.069666
},
"kernel_family_peak_bytes": {
"median": 1105092608,
"minimum": 1059520512,
"maximum": 1117036544
},
"projected_tokens": {
"median": 58090,
"minimum": 58090,
"maximum": 58090
},
"mcp_reply_bytes": {
"median": 7469,
"minimum": 7469,
"maximum": 7469
}
}
}
},
"recordings": [
{
"case": "erl_lint-1-complete",
"lane": "complete",
"artifact": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam",
"sha256": "716d78394b7c3da983e36aac7aadd6673e5041677ba7de20413cc2704d958d55",
"bytes": 233968
},
"answer_count": 4,
"answer_sha256": "34ce3091dbea9ac8cd20831cf6443c7a1de6e02b753246cafa3e0e22b83907c9",
"parent_evidence_id": "ev_df4bfa8b8f0391ad6291542fdccf9062ae3bb8dbabf01c2b166ca6f3493051fc",
"source_report_sha256": "db64350ae25bff9e77f5e167f4a4ac095ed872e09c15de5ce3b2728c078a16d2",
"runtime_identity_sha256": "1b4b40330a2e21d31c06a07d10937cb6a3b37cd0bcbf3d129380341d486b916e",
"catalog_sha256": "f4a683209df5bed1ed25ae42c690fdd803348742132e88f77ba2e4e8333ca562",
"calls": [
{
"name": "inspect_erlang_beam",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam"
},
"phase": "inspect",
"elapsed_ms": 18092.161941,
"reply_bytes": 146627,
"is_error": false
},
{
"name": "export_evidence_bundle",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/sdk-cache-cohort/erl_lint-1-complete/evidence.json"
},
"phase": "export",
"elapsed_ms": 52.40052199999991,
"reply_bytes": 376,
"is_error": false
}
],
"phases": {
"discovery_ms": 1018.838796
},
"wall_seconds": 21.577,
"cpu_seconds": 16.561897,
"kernel_family_peak_bytes": 1073950720,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"payload_cleanup_verified": true,
"final_whole_scope_events": null,
"token_proxy": {
"question": 32,
"server_instructions": 513,
"input_catalog_once": 55409,
"arguments": 78,
"text_replies": 20875,
"combined": 76907,
"complete_transport_catalog": 829075
},
"receipt_sha256": {
"report.json": "3717e681ac50a70db8bd7303ac18d87bd2028d4321104a793421695a82579290",
"catalog.json": "f4a683209df5bed1ed25ae42c690fdd803348742132e88f77ba2e4e8333ca562",
"transcript.json": "c718ac6e9b6c63533bc28d18c90c73a03abc1697516ae216a444b6dd55827255",
"evidence.json": "ca66367cd1bb325d7d4e47f958c86234d01aaf749b68778ac5212cac246aacb3",
"erl_lint-1-complete.log": "a90d1345cd09e90865c8a75f732abe2561c2299d0084718e91205f2cb904f38d"
}
},
{
"case": "erl_lint-1-selected",
"lane": "selected",
"artifact": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam",
"sha256": "716d78394b7c3da983e36aac7aadd6673e5041677ba7de20413cc2704d958d55",
"bytes": 233968
},
"answer_count": 4,
"answer_sha256": "34ce3091dbea9ac8cd20831cf6443c7a1de6e02b753246cafa3e0e22b83907c9",
"parent_evidence_id": "ev_df4bfa8b8f0391ad6291542fdccf9062ae3bb8dbabf01c2b166ca6f3493051fc",
"source_report_sha256": "db64350ae25bff9e77f5e167f4a4ac095ed872e09c15de5ce3b2728c078a16d2",
"runtime_identity_sha256": "a74d203a5a12d7eb82aaa1e0fda6595b9647af2255f318bf116953777052460f",
"catalog_sha256": "9c09aa2944f10c806df78d1cfa2e1daeddc0a487c432701cb67fa65e14213b0c",
"calls": [
{
"name": "inspect_erlang_beam",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam",
"detail": "summary",
"view": {
"kind": "beam",
"collection": "exports",
"names": ["module", "exprs", "expr"],
"fields": ["function_index", "function", "arity", "label"],
"offset": 0,
"limit": 64
}
},
"phase": "inspect",
"elapsed_ms": 18394.333454,
"reply_bytes": 7093,
"is_error": false
},
{
"name": "export_evidence_bundle",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/sdk-cache-cohort/erl_lint-1-selected/evidence.json"
},
"phase": "export",
"elapsed_ms": 58.59494199999972,
"reply_bytes": 376,
"is_error": false
}
],
"phases": {
"discovery_ms": 1056.9951270000001
},
"wall_seconds": 22.397,
"cpu_seconds": 17.069666,
"kernel_family_peak_bytes": 1117036544,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"payload_cleanup_verified": true,
"final_whole_scope_events": null,
"token_proxy": {
"question": 32,
"server_instructions": 513,
"input_catalog_once": 56359,
"arguments": 120,
"text_replies": 1066,
"combined": 58090,
"complete_transport_catalog": 834792
},
"receipt_sha256": {
"report.json": "c3c17495f33ee4ea9daabd7a353f698fa0a804451c4bbdbe9eb1adf905ab483a",
"catalog.json": "9c09aa2944f10c806df78d1cfa2e1daeddc0a487c432701cb67fa65e14213b0c",
"transcript.json": "fd8c589a6e0dbcd0783ef5a526b51db7a3c208cd4067986ed99804bda06c4498",
"evidence.json": "2373bc2e57e40aac881f3ef91fa77187b098dd61d6cf44d37a4b8fe56d022b40",
"erl_lint-1-selected.log": "fdd55def718ffa7c972b3cb9b6be1b7f66a7fa5fb1c371cc492051ffb0d08f62"
}
},
{
"case": "erl_lint-2-selected",
"lane": "selected",
"artifact": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam",
"sha256": "716d78394b7c3da983e36aac7aadd6673e5041677ba7de20413cc2704d958d55",
"bytes": 233968
},
"answer_count": 4,
"answer_sha256": "34ce3091dbea9ac8cd20831cf6443c7a1de6e02b753246cafa3e0e22b83907c9",
"parent_evidence_id": "ev_df4bfa8b8f0391ad6291542fdccf9062ae3bb8dbabf01c2b166ca6f3493051fc",
"source_report_sha256": "db64350ae25bff9e77f5e167f4a4ac095ed872e09c15de5ce3b2728c078a16d2",
"runtime_identity_sha256": "a74d203a5a12d7eb82aaa1e0fda6595b9647af2255f318bf116953777052460f",
"catalog_sha256": "9c09aa2944f10c806df78d1cfa2e1daeddc0a487c432701cb67fa65e14213b0c",
"calls": [
{
"name": "inspect_erlang_beam",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam",
"detail": "summary",
"view": {
"kind": "beam",
"collection": "exports",
"names": ["module", "exprs", "expr"],
"fields": ["function_index", "function", "arity", "label"],
"offset": 0,
"limit": 64
}
},
"phase": "inspect",
"elapsed_ms": 15135.096698,
"reply_bytes": 7093,
"is_error": false
},
{
"name": "export_evidence_bundle",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/sdk-cache-cohort/erl_lint-2-selected/evidence.json"
},
"phase": "export",
"elapsed_ms": 37.63045200000124,
"reply_bytes": 376,
"is_error": false
}
],
"phases": {
"discovery_ms": 1335.57118
},
"wall_seconds": 19.375,
"cpu_seconds": 16.139353,
"kernel_family_peak_bytes": 1059520512,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"payload_cleanup_verified": true,
"final_whole_scope_events": null,
"token_proxy": {
"question": 32,
"server_instructions": 513,
"input_catalog_once": 56359,
"arguments": 120,
"text_replies": 1066,
"combined": 58090,
"complete_transport_catalog": 834792
},
"receipt_sha256": {
"report.json": "d25a255b106e1bb232237474915e2900e313ba354aded0d27dad81ea74cfe0c6",
"catalog.json": "9c09aa2944f10c806df78d1cfa2e1daeddc0a487c432701cb67fa65e14213b0c",
"transcript.json": "2c0f92c25f97bb825b99133b84d792c7aac315b4c100986d75cb01939af6ce77",
"evidence.json": "2373bc2e57e40aac881f3ef91fa77187b098dd61d6cf44d37a4b8fe56d022b40",
"erl_lint-2-selected.log": "011553e956860fb3ce9d4e54ca7d4a2baf88a260786d82f60b1d0b70974fdbb7"
}
},
{
"case": "erl_lint-2-complete",
"lane": "complete",
"artifact": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam",
"sha256": "716d78394b7c3da983e36aac7aadd6673e5041677ba7de20413cc2704d958d55",
"bytes": 233968
},
"answer_count": 4,
"answer_sha256": "34ce3091dbea9ac8cd20831cf6443c7a1de6e02b753246cafa3e0e22b83907c9",
"parent_evidence_id": "ev_df4bfa8b8f0391ad6291542fdccf9062ae3bb8dbabf01c2b166ca6f3493051fc",
"source_report_sha256": "db64350ae25bff9e77f5e167f4a4ac095ed872e09c15de5ce3b2728c078a16d2",
"runtime_identity_sha256": "1b4b40330a2e21d31c06a07d10937cb6a3b37cd0bcbf3d129380341d486b916e",
"catalog_sha256": "f4a683209df5bed1ed25ae42c690fdd803348742132e88f77ba2e4e8333ca562",
"calls": [
{
"name": "inspect_erlang_beam",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam"
},
"phase": "inspect",
"elapsed_ms": 15133.678018999999,
"reply_bytes": 146627,
"is_error": false
},
{
"name": "export_evidence_bundle",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/sdk-cache-cohort/erl_lint-2-complete/evidence.json"
},
"phase": "export",
"elapsed_ms": 44.53432299999986,
"reply_bytes": 376,
"is_error": false
}
],
"phases": {
"discovery_ms": 1000.9708420000002
},
"wall_seconds": 19.764,
"cpu_seconds": 15.287268,
"kernel_family_peak_bytes": 1041948672,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"payload_cleanup_verified": true,
"final_whole_scope_events": null,
"token_proxy": {
"question": 32,
"server_instructions": 513,
"input_catalog_once": 55409,
"arguments": 78,
"text_replies": 20875,
"combined": 76907,
"complete_transport_catalog": 829075
},
"receipt_sha256": {
"report.json": "61aa7d7c78db145af4399b22d57aa39c2008c479530b28a49d401a6d564e46f7",
"catalog.json": "f4a683209df5bed1ed25ae42c690fdd803348742132e88f77ba2e4e8333ca562",
"transcript.json": "3729bcc5547ab06f80b578c24bbb7937a438b0c190ee6680d0d7868cb5885159",
"evidence.json": "ca66367cd1bb325d7d4e47f958c86234d01aaf749b68778ac5212cac246aacb3",
"erl_lint-2-complete.log": "7bb8d637df3cb7ba542345c9d93ea09fe967d11c90cf5383083db7cd2c5fafb8"
}
},
{
"case": "erl_lint-3-complete",
"lane": "complete",
"artifact": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam",
"sha256": "716d78394b7c3da983e36aac7aadd6673e5041677ba7de20413cc2704d958d55",
"bytes": 233968
},
"answer_count": 4,
"answer_sha256": "34ce3091dbea9ac8cd20831cf6443c7a1de6e02b753246cafa3e0e22b83907c9",
"parent_evidence_id": "ev_df4bfa8b8f0391ad6291542fdccf9062ae3bb8dbabf01c2b166ca6f3493051fc",
"source_report_sha256": "db64350ae25bff9e77f5e167f4a4ac095ed872e09c15de5ce3b2728c078a16d2",
"runtime_identity_sha256": "1b4b40330a2e21d31c06a07d10937cb6a3b37cd0bcbf3d129380341d486b916e",
"catalog_sha256": "f4a683209df5bed1ed25ae42c690fdd803348742132e88f77ba2e4e8333ca562",
"calls": [
{
"name": "inspect_erlang_beam",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam"
},
"phase": "inspect",
"elapsed_ms": 13658.772345000001,
"reply_bytes": 146627,
"is_error": false
},
{
"name": "export_evidence_bundle",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/sdk-cache-cohort/erl_lint-3-complete/evidence.json"
},
"phase": "export",
"elapsed_ms": 48.42420299999867,
"reply_bytes": 376,
"is_error": false
}
],
"phases": {
"discovery_ms": 1121.564358
},
"wall_seconds": 17.257,
"cpu_seconds": 16.276557,
"kernel_family_peak_bytes": 1078980608,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"payload_cleanup_verified": true,
"final_whole_scope_events": null,
"token_proxy": {
"question": 32,
"server_instructions": 513,
"input_catalog_once": 55409,
"arguments": 78,
"text_replies": 20875,
"combined": 76907,
"complete_transport_catalog": 829075
},
"receipt_sha256": {
"report.json": "d683aab80d34c4f0fb474981c5715a4ba1ee706bf4e74be10bf69c40358ec011",
"catalog.json": "f4a683209df5bed1ed25ae42c690fdd803348742132e88f77ba2e4e8333ca562",
"transcript.json": "12380e1e67368b8df7787d54be828c953d4e2ef63007c04cd9fc912b42cc42b5",
"evidence.json": "ca66367cd1bb325d7d4e47f958c86234d01aaf749b68778ac5212cac246aacb3",
"erl_lint-3-complete.log": "404f609e4dc10bbf0d8f1d3076751b2cb8b271ff401b00ea161a00944a55d558"
}
},
{
"case": "erl_lint-3-selected",
"lane": "selected",
"artifact": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam",
"sha256": "716d78394b7c3da983e36aac7aadd6673e5041677ba7de20413cc2704d958d55",
"bytes": 233968
},
"answer_count": 4,
"answer_sha256": "34ce3091dbea9ac8cd20831cf6443c7a1de6e02b753246cafa3e0e22b83907c9",
"parent_evidence_id": "ev_df4bfa8b8f0391ad6291542fdccf9062ae3bb8dbabf01c2b166ca6f3493051fc",
"source_report_sha256": "db64350ae25bff9e77f5e167f4a4ac095ed872e09c15de5ce3b2728c078a16d2",
"runtime_identity_sha256": "a74d203a5a12d7eb82aaa1e0fda6595b9647af2255f318bf116953777052460f",
"catalog_sha256": "9c09aa2944f10c806df78d1cfa2e1daeddc0a487c432701cb67fa65e14213b0c",
"calls": [
{
"name": "inspect_erlang_beam",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/real-beam/erl_lint.beam",
"detail": "summary",
"view": {
"kind": "beam",
"collection": "exports",
"names": ["module", "exprs", "expr"],
"fields": ["function_index", "function", "arity", "label"],
"offset": 0,
"limit": 64
}
},
"phase": "inspect",
"elapsed_ms": 14405.966256000002,
"reply_bytes": 7093,
"is_error": false
},
{
"name": "export_evidence_bundle",
"input": {
"path": "/tmp/rea-perf-static-tool-audit-20261024/sdk-cache-cohort/erl_lint-3-selected/evidence.json"
},
"phase": "export",
"elapsed_ms": 31.32568199999878,
"reply_bytes": 376,
"is_error": false
}
],
"phases": {
"discovery_ms": 1107.032897
},
"wall_seconds": 18.052,
"cpu_seconds": 16.867381,
"kernel_family_peak_bytes": 1105092608,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"payload_cleanup_verified": true,
"final_whole_scope_events": null,
"token_proxy": {
"question": 32,
"server_instructions": 513,
"input_catalog_once": 56359,
"arguments": 120,
"text_replies": 1066,
"combined": 58090,
"complete_transport_catalog": 834792
},
"receipt_sha256": {
"report.json": "732e5b4f880d3a981d5362ece88578a2da9e68e80e97eca2dec2ae5856ebbdee",
"catalog.json": "9c09aa2944f10c806df78d1cfa2e1daeddc0a487c432701cb67fa65e14213b0c",
"transcript.json": "e62dc35d939b21418989093a8addadbe533d358a046feb7d99cecab10cedc2d4",
"evidence.json": "2373bc2e57e40aac881f3ef91fa77187b098dd61d6cf44d37a4b8fe56d022b40",
"erl_lint-3-selected.log": "2f35909a9ad5efa9e8ddeffe665887e8a9e9c97c13b4a030b765dd1ceecb14ef"
}
}
]
}
File diff suppressed because it is too large. Load diff
@@ -1,125 +0,0 @@
# Selected BEAM exports: completed-task measurements
Issue #6938. The analyst asks for every exported function with a small set of
exact names, preserving arities, original table/atom indexes and labels. Both
lanes export and validate the **complete** parent report, not only the answer.
Baseline `e09b06369d923e9cccedd91f6839ba643dfbd893`; selected delivery
`bcc3f7611` (full SHA and source-clean runtime identities in the JSON receipt).
The reader, format budgets and complete Evidence identity are unchanged.
Node 24.22.1, SDK client/core/server 2.3.1, npm 11.16.0; Linux x86-64. The
18-task cohort alternates three fresh complete/selected sessions per artifact.
Fresh HOME/client/server and disabled Node compile cache; OS caches unflushed.
This is a scripted client task, not an actual model investigation.
## Results
Medians, before → after. Seconds and kernel process-family MiB (includes file
cache), not parent-only RSS. Ranges and every receipt hash are in
[the 18-task JSON](erlang-beam-views.json).
| Real artifact | Whole-task estimated tokens | Wall s | Family CPU s | Family peak MiB |
| ------------- | --------------------------: | ------------: | ------------: | --------------: |
| erl_lint | 76,907 → 58,090 | 6.773 → 6.870 | 5.581 → 5.312 | 667.0 → 630.9 |
| erl_parse | 72,499 → 58,087 | 6.367 → 6.974 | 5.627 → 5.555 | 666.1 → 657.5 |
| pg2 | 57,733 → 58,018 | 6.365 → 7.576 | 5.295 → 5.615 | 648.2 → 626.9 |
For erl_lint the estimate drops 24.5%; erl_parse drops 19.9%. Small pg2 grows 0.5%
(285 tokens), so selected delivery does not benefit every input. Complete stays
the default. No stable general elapsed/CPU/memory improvement is established;
the small selected task has a higher wall/CPU median. Full parsing/retention and
cold client setup remain. Matching source-report, parent-ID and answer digests
agree across all six tasks for each artifact.
The projection counts one question, server instructions, all 221 input-tool
names/descriptions/schemas once, and every argument/text reply including export.
`tiktoken 0.14.0/cl100k_base` serialization estimates are not actual model usage.
Input directory cost grows 55,409→56,359tokens; the complete transport catalog
including output schemas grows 829,075→834,792tokens and is reported separately.
Protocol framing, host injection/history replay, output-schema presentation to
a model and inference quality are unmeasured. Reported context reduction is
specific to this declared client projection.
## SDK cached-catalog control
Six additional alternating fresh tasks use the SDK's cached catalog/default
validator without an explicit `toolDefinition`. They answer/export the same
question and pass the same independent oracle and cleanup checks. They are
separate from the 18-task primary cohort; do not pool their startup costs.
Median wall 19.764→19.375s (ranges 17.257–21.577 / 18.052–22.397);
family CPU 16.277→16.867s (ranges 15.287–16.562 / 16.139–17.070);
family peak 1024.2→1053.9MiB.
No general cached-client startup improvement is claimed. See
[all six control receipts](erlang-beam-sdk-cache.json).
## Correctness and resources
The unmodified Debian `erlang-base_27.3.4.1+dfsg-1+deb13u3_amd64.deb` supplies
actual OTP 27 compiler/stdlib modules: erl_lint 233,968 bytes (915 atoms, 119 imports,
112 lambdas), erl_parse 608,240 bytes (1,035 atoms), and pg2 1,152 bytes (4 atoms).
The package URL/SHA256, every selected artifact digest, source revision, driver
hash and runtime fingerprint are retained in the JSON records. The independent
[Python struct oracle](erlang-beam-oracle.py) checks every decoded table, chunk
range/payload digest and Code header, then the selected export answer. It reads
bytes only and deliberately qualifies these fixed OTP 27 layouts, not every
historical/compact format. No Erlang installation, emulator or target execution
was used. Default CLI and selected CLI on all three inputs also matched every
oracle fact and parent ID and published exactly the complete source Evidence.
Each of the 24 SDK tasks exited 0 with payload cleanup verified and no guard stop,
OOM or task-cap event. Aggregate CPUQuota 100%, payload affinity0–1,3 GiB early
stop/4 GiB hard limit, no swap, TasksMax 192. CPU/peak counters are observed after
the payload exits while the observer lives; final whole-scope counters remain
unavailable. Costs include SDK discovery/schema validation, runtime/dependency
hashing, assertions, source retention/export and cleanup. Sample acquisition,
oracle generation and tokenization occur outside timed tasks. Node, all compiled dist
files, selected entrypoints, 59 actual SDK files and driver files are hashed
before/after; the source revisions were clean when fingerprints were created.
Public regressions additionally exercise all five collections, duplicate names
and arities, selectors/fields/pages, missing versus empty tables, unresolved atom
names, unavailable retention, source deletion, offline bundles and refusal before
file reads. They establish those boundaries; real new-view qualification here is
OTP 27 on Linux, not every platform or provider.
## Reproducing the question
Build clean pinned checkouts with the repository's pinned toolchain and acquire
the public package linked in the JSON. Keep extracted files outside the repo.
Generate each oracle, for example:
```sh
python3 experiments/analysis-efficiency/erlang-beam-oracle.py \
--input /samples/erl_lint.beam --output /runs/erl_lint-oracle.json \
--name module --name exprs --name expr
node experiments/analysis-efficiency/erlang-beam-views.mjs \
--runtime /built/rea --identity /runs/runtime-identity.json \
--oracle /runs/erl_lint-oracle.json --output /runs/erl_lint-selected \
--lane selected
```
A runtime fingerprint supplies `compiled_files` and `entrypoints` with relative
paths/SHA256, `driver_files` relative to this experiment directory, actual SDK
`dependency_files` with absolute paths/SHA256, and `node_sha256`. Source revision
and clean-state fields record preparation. Create fingerprints before timing;
the driver verifies them before and after each task. The publication records
fingerprint/receipt hashes and full commands; the original complete fingerprint
files, SDK transcripts, complete Evidence bundles and resource logs remain at
`/tmp/rea-perf-static-tool-audit-20261024/`. The local cohort controller is retained
there too. These local receipts are not shipped binaries or installed providers.
Wrap each invocation with `resource-run.py --observe-payload-exit`, setting the
pinned Node PATH/heap, process affinity and resource policy as in its recorded
command. Use a fresh output directory/session each time, alternate lanes three
times per artifact, and count with
[erlang-beam-costs.py](erlang-beam-costs.py). `--page-size` can qualify extra
retained-page calls; `--sdk-cache` selects the separate cached-catalog control.
The18 published primary tasks use the exact discovered toolDefinition/default
validator and 64-row pages, requiring two calls each including full-source export.
The initial exploratory baseline used earlier main `e2463372`; it is not mixed
into this table. Failed development checks remain in the local record: an
incorrect formatter path and copied service import, followed by exact complete
contract narrowing in old test consumers; corrected type/format/public checks
passed. These were not successful benchmark samples.
@@ -1,255 +0,0 @@
/** Compare a completed selected-export question through production stdio MCP. */
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { finished } from "node:stream/promises";
import { parseArgs } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
const { values } = parseArgs({
options: {
...Object.fromEntries(
["runtime", "oracle", "output", "identity"].map((k) => [
k,
{ type: "string" },
]),
),
lane: { type: "string", default: "complete" },
"page-size": { type: "string", default: "64" },
"sdk-cache": { type: "boolean", default: false },
},
});
assert.ok(["complete", "selected"].includes(values.lane));
const pageSize = Number(values["page-size"]);
assert.ok(Number.isSafeInteger(pageSize) && pageSize > 0);
for (const key of ["runtime", "oracle", "output", "identity"])
assert.equal(values[key], resolve(values[key]));
const { createRetainedQueryCaller } =
await import("./retained-query-calls.mjs");
const { finalizeRetainedQueryReceipt } =
await import("./retained-query-finalization.mjs");
const hash = (x) => createHash("sha256").update(x).digest("hex"),
oracleBytes = await readFile(values.oracle),
expected = JSON.parse(oracleBytes),
identityBytes = await readFile(values.identity),
identity = JSON.parse(identityBytes);
assert.equal(
hash(await readFile(expected.expected.artifact.path)),
expected.expected.artifact.sha256,
);
await mkdir(values.output, { mode: 0o700 });
const account = await mkdtemp(join(tmpdir(), "rea-beam-task-"));
const client = new Client({ name: "beam-selected-exports", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
maxBufferSize: 10485760,
env: {
HOME: account,
TMPDIR: account,
PATH: process.env.PATH,
REA_LOG_LEVEL: "silent",
NODE_OPTIONS: "--max-old-space-size=2048",
NODE_DISABLE_COMPILE_CACHE: "1",
UV_THREADPOOL_SIZE: "2",
},
stderr: "pipe",
});
const serverLog = createWriteStream(join(values.output, "server.log")),
serverLogFinished = finished(serverLog);
serverLogFinished.catch(() => {});
transport.stderr?.pipe(serverLog);
const report = {
status: "running",
lane: values.lane,
artifact: expected.expected.artifact,
calls: [],
phases: {},
question: `Return every exported function named ${expected.names.join(", ")} with its original export-table index, function atom index, arity and code label. Preserve complete source Evidence.`,
client: values["sdk-cache"]
? "Production stdio SDK, cached catalog definitions and default validator"
: "Production stdio SDK, exact discovered toolDefinition and default validator",
runtime_identity_sha256: hash(identityBytes),
oracle_sha256: hash(oracleBytes),
driver_sha256: hash(await readFile(new URL(import.meta.url))),
};
const verifyRuntime = async () => {
assert.equal(hash(await readFile(values.identity)), hash(identityBytes));
assert.equal(hash(await readFile(values.oracle)), hash(oracleBytes));
for (const r of [...identity.compiled_files, ...identity.entrypoints])
assert.equal(
hash(await readFile(join(values.runtime, r.path))),
r.sha256,
r.path,
);
for (const r of identity.dependency_files ?? [])
assert.equal(hash(await readFile(r.path)), r.sha256, r.path);
for (const r of identity.driver_files)
assert.equal(
hash(await readFile(new URL(r.path, import.meta.url))),
r.sha256,
r.path,
);
assert.equal(hash(await readFile(process.execPath)), identity.node_sha256);
assert.equal(
hash(await readFile(new URL(import.meta.url))),
report.driver_sha256,
);
};
const transcript = [];
let definitions;
const caller = createRetainedQueryCaller(
{
callTool(params, options) {
return client.callTool(
params,
values["sdk-cache"]
? options
: {
...options,
toolDefinition: definitions.get(params.name),
},
);
},
},
(receipt, event) => {
report.calls.push(receipt);
transcript.push(event);
},
);
const observe = async (name, input, phase) => {
const result = await caller(name, input, phase);
assert.notEqual(result.isError, true, JSON.stringify(result));
assert.ok(result.structuredContent);
return result.structuredContent;
};
const verifySource = (source) => {
for (const [key, value] of Object.entries(expected.expected))
assert.deepEqual(source[key], value, key);
assert.equal(source.coverage.status, "complete");
};
try {
await verifyRuntime();
const started = performance.now();
await client.connect(transport);
report.server_instructions = client.getInstructions() ?? "";
const catalog = await client.listTools();
definitions = new Map(catalog.tools.map((t) => [t.name, t]));
report.catalog_tools = catalog.tools.length;
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
report.phases.discovery_ms = performance.now() - started;
const selected = values.lane === "selected";
const selectedView = (offset) => ({
kind: "beam",
collection: "exports",
names: expected.names,
fields: ["function_index", "function", "arity", "label"],
offset,
limit: pageSize,
});
const response = await observe(
"inspect_erlang_beam",
{
path: expected.expected.artifact.path,
...(selected ? { detail: "summary", view: selectedView(0) } : {}),
},
"inspect",
);
report.parent_evidence_id = selected
? response.normalized_result.parent_evidence_id
: response.evidence_id;
let answer;
if (!selected) {
verifySource(response.normalized_result);
answer = (response.normalized_result.exports ?? [])
.map((r, i) => ({ source_index: i, ...r }))
.filter((r) => expected.names.includes(r.function));
} else {
answer = [];
let offset = 0,
page = response.normalized_result;
for (;;) {
assert.equal(page.kind, "beam");
assert.equal(page.page.collection, "exports");
assert.deepEqual(page.beam.artifact, expected.expected.artifact);
assert.deepEqual(page.beam.coverage, expected.expected.coverage);
assert.equal(page.beam.counts.exports, expected.expected.exports.length);
assert.equal(page.coverage.total, expected.answer.length);
answer.push(...page.page.items);
if (page.coverage.next_offset === null) break;
assert.ok(page.coverage.next_offset > offset);
offset = page.coverage.next_offset;
page = (
await observe(
"inspect_analysis_view",
{
source: {
kind: "retained-evidence",
evidence_id: report.parent_evidence_id,
},
view: selectedView(offset),
},
"selected-page",
)
).normalized_result;
}
}
assert.deepEqual(answer, expected.answer);
report.answer_count = answer.length;
report.answer_sha256 = hash(
JSON.stringify(
answer.map((r) =>
Object.fromEntries(
Object.entries(r).sort(([a], [b]) => a.localeCompare(b)),
),
),
),
);
await writeFile(join(values.output, "answer.json"), JSON.stringify(answer));
await observe(
"export_evidence_bundle",
{ path: join(values.output, "evidence.json") },
"export",
);
const bundle = JSON.parse(
await readFile(join(values.output, "evidence.json")),
);
const parent = bundle.records.find(
(r) => r.evidence_id === report.parent_evidence_id,
);
assert.ok(parent);
assert.equal(parent.operation, "inspect_erlang_beam");
verifySource(parent.normalized_result);
report.source_report_sha256 = hash(JSON.stringify(parent.normalized_result));
report.complete_source_export_verified = true;
report.status = "passed";
} catch (cause) {
report.status = "failed";
report.error = String(cause);
} finally {
try {
await verifyRuntime();
} catch (cause) {
report.status = "failed";
report.runtime_identity_failure = String(cause);
}
await finalizeRetainedQueryReceipt({
client,
transport,
serverLog,
serverLogFinished,
account,
report,
output: values.output,
transcript,
});
if (
report.status !== "passed" ||
report.cleanup_failures.length ||
report.receipt_failures.length
)
process.exitCode = 1;
}
@@ -1,218 +0,0 @@
# EVM descriptor and real-template checkpoint
Continues [#2016](https://github.com/morluto/rea/issues/2016) after the native
checkpoint merged. The internal descriptor binds the existing exact EVM
contract, CLI name and typed service lifetime. Existing complete inputs, output,
provider selection and cleanup remain the baseline. Wiring alone does not establish
token or latency savings.
The descriptor contains only the named contract, existing CLI name and existing
factory. The legacy `createEvmInterfaceService` export remains usable. The MCP
host selects one complete service instance, preserves injection identity, passes
that same instance to registration and retains it for cleanup/retry. No bound
service wrapper, generic registry, automatic dependency acquisition or new pack
selection is introduced.
Five actual in-memory SDK boundary scenarios verify converted schemas, duplicate
registration rejection before use, malformed inputs, explicit unavailable or
incompatible-output failures with a healthy WASM peer, cleanup retry, and request
cancellation followed by recovery. Their injected ports are recording seams, not
real EVMole/WABT claims. Nine service and 26 provider-boundary cases also pass;
two unsupported-host scenarios are skipped on this Linux host. The existing seven
WASM MCP scenarios pass. Full `npm run check` passes; its approximately 3 GiB
sampled peak and `memory.high` events are excluded from performance claims.
Candidate real-artifact/catalog and model observations are recorded below.
## Public inputs and independent checks
[corpus.json](corpus.json) retains the original WETH historical example and adds
Uniswap v3-core 1.0.1 Factory/Pool compiler runtime templates from the official npm
package. Archive integrity, exact regular member, compiler JSON, original carrier
and decoded-byte digests are pinned. The carrier is the original `deployedBytecode`
field plus one trailing LF. No placeholder is rewritten and no contract is executed.
These are compiler templates, not live deployed instances. Their published Hardhat
artifacts omit immutable-reference maps; that metadata remains explicitly unknown.
The recorded empty link-reference maps do not establish absence of immutables.
WETH remains a published historical RPC example, with unknown live authenticity.
Actual control CLI observations infer 7 Factory and 26 Pool functions. The
[independent verifier](verify-evm-template.mjs) hashes each canonical published ABI
signature with caller-supplied Ethereum Keccak and checks the complete ABI-dispatch
selector multiset against the original REA response. Both sets match. This does not
establish inferred argument/mutability equivalence, deployment authenticity or
runtime semantics; all producer unknowns remain visible.
## Latest-main control
Production control: `1770f38cfc6cbe87491d0de8b8730c305c5cf8ea`, Node 24.18.0,
npm 11.16.0, matching locked dependencies and unchanged EVMole 0.9.3.
The frozen `dist/`/`bridge/` file-table SHA-256 is
`7fd6c2c37b4e166df158ec29d48d31f3712b9b087d9807ee019ee997d06ea144`.
Measurement driver SHA-256 is
`f767c0c0232be1e11918e70e3197a6919680d16688076bef28d4320662f712f7`.
Each public SDK run validates the selected named advertised schema and verifies
original carrier and decoded identities, inference/unknown scope and unchanged
input bytes. All three runs return the same complete 2,420,709-byte catalog.
| Question | Inferred functions | SDK tool-result JSON bytes | First workflow seconds |
| ------------------------ | -----------------: | -------------------------: | ---------------------: |
| WETH historical carrier | 11 | 26,760 | 11.432 |
| Uniswap Factory template | 7 | 110,424 | 3.628 |
| Uniswap Pool template | 26 | 110,998 | 11.211 |
These are initial observations, not paired performance conclusions or model-token
counts. Each workflow includes connection, complete catalog discovery and one
interface query. Full bytecode remains inline. Candidate/control repetitions must
use the same source driver, inputs, provider settings and Node/worker limits.
Preparation against earlier frozen main `b1bf4a70d44191348ca20a91a52fb663da4daed0`
also passed all 59 cases of the existing real CLI/MCP EVM verifier with privately
supplied solc 0.8.30 and strace 6.1. Those cases cover compiler configurations,
raw/hex identity, malformed/unsupported carriers, missing limiter and resource
constraints. Its measured scope took 233.1 seconds, sampled peak 1,192,730,624
bytes and last-observed CPU 181.018 seconds. This is a control, not candidate proof.
All heavy runs are serial under one CPU, 4 GiB hard memory/3 GiB early-stop, no swap
and bounded tasks. Final cgroup counters disappear at scope teardown and remain
unverified; last-observed zero limit events are not substituted for final counters.
Raw replies, tools, compiler artifacts and detailed receipts remain outside Git.
## Reproduction
Build and freeze each pinned runtime using the repository toolchain, then run
`measure.mjs --artifact uniswap-v3.factory.hex` or `uniswap-v3.pool.hex` with the
exact derived carrier, `--schemas selected`, explicit runtime/revision and a fresh
private output directory. Existing `weth.hex` answer fields and invocation remain
unchanged. The same common workflow verifies all three offline carriers.
Run `verify-evm-template.mjs` with `--artifact` selecting the corpus ID,
`--artifact-json` selecting its original published compiler JSON, `--evidence`
selecting the recorded CLI Evidence or SDK structured reply, `--keccak-module`
selecting the absolute `js-sha3/src/sha3.js` module from the private solc toolchain,
and `--output` selecting a new private receipt. It acquires no dependencies and
does not overwrite an existing receipt.
The [template model question](questions/evm-uniswap.txt) requires the same complete
inferred interface facts and explicit unknowns. An ABI-count match is not used as
a model grade.
## Candidate compatibility and real-provider verification
Candidate: `152984da771450048ee9d60db8a410994a125878`, frozen runtime file-table
SHA-256 `725563640c27eb50ddb6f977d2dfbe1bdaddf7e05c02897318bceaa36fa602d7`.
Two repetitions for each EVM input produce six passing paired fact/settings
comparisons with the same driver and explicit limits. Each report preserves both
original Evidence reference maps. The complete actual SDK catalog is byte-identical
on every pair, including order, names, descriptions, schemas and annotations;
catalog SHA-256 is
`8664e42fd5225645cfd910f440f731c67ac198d3d53aa85b5c485374c74cf603`.
Result byte counts remain unchanged. Candidate Factory/Pool selectors also match
the independent published ABI oracle.
| Question | Control seconds, repetitions 1 / 2 | Candidate seconds, repetitions 1 / 2 |
| -------- | ---------------------------------: | -----------------------------------: |
| Factory | 3.628 / 3.650 | 3.708 / 3.605 |
| Pool | 11.211 / 3.591 | 3.749 / 3.581 |
| WETH | 11.432 / 3.694 | 3.572 / 3.677 |
First controls preceded wiring; each second pair ran candidate then control.
Every measurement starts a new MCP process; host cache/page state is not fully
controlled. Slower initial controls do not establish a descriptor speedup.
Second-pair latency and last-observed CPU are similar. Sampled EVM SDK family peaks
range from 510,255,104 to 527,622,144 bytes. No scope abort occurred; final counters
remain unknown, so these measurements are descriptive.
The candidate passes all 59 existing real CLI/MCP verifier cases with solc 0.8.30
and unchanged EVMole 0.9.3. Actual syscall traces report target execution and network
sockets absent for those scenarios, and the verifier reports no remaining
descendants. The scope took 177.456 seconds, sampled peak 1,237,020,672 bytes and
last-observed CPU 144.913 seconds. Different totals from the earlier control are
not attributed to unchanged provider algorithms.
Post-descriptor public MCP regression uses the original pinned shipped corpus:
| Domain / selected question | Actual provider / scope | Calls | Seconds |
| ------------------------------------------- | -------------------------------------------------- | ----: | ------: |
| SQLite WASM counts / exports | WABT 1.0.42; selected summary and retained read | 2 | 5.516 |
| ShareX managed metadata / native boundaries | Static PE/CLI, no assembly execution | 2 | 3.710 |
| Obsidian manifest / inventory | Complete inventory retained, no AST claim | 4 | 5.376 |
| NewPipe package / MainActivity methods | Real headless JADX 0.7.1 | 3 | 21.669 |
| OpenWrt regions | Real firmware signature scan, no device execution | 1 | 3.581 |
| jq native entry | Ghidra 12.1.4, default analysis, ephemeral Program | 4 | 270.986 |
All six workflows pass named SDK output validation and original digest checks;
input bytes remain unchanged. Maximum sampled family peak is 1,284,648,960 bytes.
They establish those available Linux workflows, not paired performance or
Windows/macOS, IDA/Hopper, Android-device, DOS COM, browser or JavaScript AST
behavior. Browser/AST were not repeated for this narrow migration.
## Real model complete-output calibration
Six serial Codex CLI 0.161.0 trials request `gpt-5.6-terra`, high reasoning, with
isolated temporary profiles pointing to the frozen candidate. Resolved server-side
model identity is not inferred. Only the EVM inspection tool is enabled, matching
the earlier WETH calibration; this is not full-catalog tool selection verification.
Every trial makes one successful call and no failed call.
| Trial | Total input | Uncached input | Output | Seconds |
| --------- | ----------: | -------------: | -----: | ------: |
| Factory 1 | 66,158 | 36,974 | 1,329 | 43.483 |
| Pool 1 | 65,600 | 36,416 | 2,307 | 69.546 |
| WETH 1 | 54,700 | 25,516 | 1,144 | 56.372 |
| Factory 2 | 65,964 | 36,780 | 1,453 | 26.412 |
| Pool 2 | 65,753 | 36,569 | 1,939 | 74.706 |
| WETH 2 | 54,502 | 25,318 | 1,208 | 35.137 |
Every trial reports 29,184 cached input tokens. Selected fact/reference/lifecycle
grades all pass: original/decoded identities, every inferred selector, offset,
dispatch, argument/mutability string including null, explicit unknowns, no runtime
execution, observed Evidence references, unchanged bytes and removed profiles.
Raw transcripts and grades remain private. Sampled peaks are
594,853,888–776,335,360 bytes; last-observed CPU is 6.181–7.034 seconds. Strict
overall grades remain unverified because final resource counters are unknown;
they are not relabeled as wholly passing resource-qualified runs.
This calibrates the current complete path after compatible wiring, not a paired
model performance experiment. The next token experiment should answer the same
interface question through an explicitly selected precise view, retain full
bytecode/original Evidence under the existing lifetime, and preserve complete
defaults. Discover support before sending new options to an older server. Include
discovery, follow-up, failures and retries in the complete task cost.
## Static contract admission verification
The follow-up from main `c462a4ebd3a10e59253bcf38353a0914b176738b` tests the
pilot's actual static boundary, without adding a dynamic module registry or
changing production code. Complete-project TypeScript checking now verifies
that the inferred descriptor type rejects a WASM operation contract, a WASM
CLI command, a WASM service factory, an unrelated input schema and an unrelated
output schema. These are negative assignability assertions: their guarantee
comes from the compiler, not from executing an incompatible descriptor.
The existing named contract and CLI identity are also checked at runtime;
separate factory calls produce distinct service/provider owners. Invalid target
input remains an application error without launching a worker.
The actual public in-memory MCP SDK test additionally spies on the provider:
duplicate EVM registration throws the named `already registered` error before
any provider call. Converted input/output JSON Schemas validate before the
first valid operation. Invalid client arguments are rejected before execution;
incompatible provider reply data is validated **after** execution and is not
presented as a pre-use contract guarantee. Existing healthy-peer, cancellation,
retained summary/default and cleanup-retry scenarios remain covered.
Full project typecheck passes with the installed compiler's supported single
thread scheduling; it includes all source/test files. The two exact focused
files pass 12 runtime cases. The resource scopes sample 2,057,961,472 bytes for
typecheck and 2,426,290,176 bytes for required build plus focused tests, below the
unchanged 3 GiB early-stop/4 GiB hard ceiling. Final transient-cgroup counters
remain unknown. No new real-provider or performance improvement is claimed by
these compile/SDK checks; the earlier pinned real EVMole/WABT/lifecycle evidence
retains its original scope and source identity.
This proves the one statically composed descriptor's named/schema/factory
compatibility and duplicate-registration guard. It does not implement runtime
admission of arbitrary JavaScript descriptors, third-party plugin ABI/version
negotiation or a general plugin SDK. Those remain deferred by the original plan.
Existing callers, full defaults, provider inputs, owned lifetimes and output
identities are unchanged.
@@ -1,208 +0,0 @@
# EVM interface delivery and migration checkpoint
Continues [#2016](https://github.com/morluto/rea/issues/2016) after the static
descriptor merged in [#2261](https://github.com/morluto/rea/pull/2261). Opt-in
`inspect_evm_interface` MCP `detail: "summary"` returns every inferred interface
candidate without bytecode hex or duplicate stdout. Complete Evidence remains
retained under its original identity. Delivery bytes fall; the measured model
cohort does **not** establish a stable token or task-time improvement.
## Frozen scope
Control production is `ad08510a113d7a9b8d92fb1e385757880e469869`; candidate
production is `32c7548b9ae5c8cda348653d177b1747127c74ba`. Their compiled file-table
SHA-256 values are respectively
`f68e85030cb05082520ea9f0d7650298ae316ce7bba59a8942e0cc1318ae9850` and
`978b3a014e3b2a1e1c6620364dcd793fa7cb40f19892d44cfb0312739a8c14a3`.
The SDK driver SHA-256 is
`a95d8d8e9f72c0b2b885c61907ed13054ebf7d1642f6f13b0126163840b1221a`.
Driver commit `03bff83e70e09a917341c1fe3ebffbfdc67785c5` fixes only reading the
public `get_evidence_bundle.result.records` envelope; production is unchanged.
Both runtimes use Node 24.18.0, matching locked dependencies, SDK client/server
2.3.1 and unchanged bundled EVMole 0.9.3.
Inputs are the digest-pinned WETH historical hex carrier and Uniswap v3-core
1.0.1 Factory/Pool compiler runtime templates in [corpus.json](corpus.json).
Original carrier and decoded-byte identities remain distinct. Templates are
not authenticated deployed instances; constructor-dependent values and absent
immutable-reference metadata stay unknown. No chain, target execution or
signature lookup is introduced.
## Public SDK pairs
Two pairs per artifact run serially: complete control then selected candidate,
followed by candidate then control. Each starts a fresh MCP process, discovers
the catalog and validates the actual named advertised schema. Host caches are
not independently controlled. All six fact/settings comparisons pass. Both
answer projections omit unrequested hex; original complete replies remain intact.
| Input | Complete / summary result bytes | Reduction | Control seconds, 1 / 2 | Candidate seconds, 1 / 2 |
| ------- | ------------------------------: | --------: | ---------------------: | -----------------------: |
| Factory | 110,424 / 14,442 | 86.92% | 3.670 / 3.766 | 3.664 / 3.867 |
| Pool | 110,998 / 19,922 | 82.05% | 4.110 / 3.740 | 3.977 / 3.870 |
| WETH | 26,760 / 15,378 | 42.53% | 3.671 / 3.710 | 3.930 / 3.673 |
These measurements support smaller delivery, not faster interpretation. Analysis
still runs and the full parent still incurs producer/retention costs. Catalog
count and order stay at 184 tools. The complete catalog grows from 2,422,229 to
2,429,938 compact JSON bytes (+7,709); all name/description/input definitions
grow from 512,655 to 514,074 bytes. The EVM input definition grows from 1,443 to
2,029 bytes. Only `inspect_analysis_view` and `inspect_evm_interface` definitions
change. These discovery costs are visible and are not converted into token counts.
Nine separate measurements verify new-server complete defaults, old-server
summary fallback and retained reads for all three inputs. The four-call retained
proofs retrieve the complete canonical parent plus pages and cost 160,110,
177,106 and 79,160 bytes for Factory, Pool and WETH respectively. They are
verification workflows, not examples of one-call delivery savings.
## Real Codex pairs and answer qualification
Twelve serial Codex CLI 0.161.0 trials use the existing identical questions for
each pair, the same artifact path, one enabled EVM tool, temporary MCP profiles
and requested `gpt-5.6-terra` high. The requested alias is not proof of the
resolved model identity. Trials include schema discovery, reasoning, one actual
successful EVM call and final answer. No call fails or retries. Profiles are
removed and input bytes remain unchanged. This fixed one-tool profile does not
test selection from the full catalog.
| Input / repetition | Uncached input, control → candidate | Seconds, control → candidate | Candidate delivery |
| ------------------ | ----------------------------------: | ---------------------------: | ------------------ |
| Factory / 1 | 22,901 → 25,668 | 52.485 → 48.768 | summary |
| Factory / 2 | 37,181 → 15,270 | 53.404 → 45.345 | summary |
| Pool / 1 | 36,652 → 26,657 | 57.764 → 66.642 | summary |
| Pool / 2 | 28,305 → 36,259 | 62.068 → 71.756 | complete |
| WETH / 1 | 21,128 → 25,187 | 44.869 → 46.809 | summary |
| WETH / 2 | 17,417 → 17,529 | 30.364 → 39.943 | summary |
Keep all trials, including the candidate that explicitly chose complete delivery.
Five of six candidates select summary, but uncached input improves in only two
pairs and time improves in only two. Aggregate control → candidate observations
are 362,496 → 357,770 total input; 198,912 → 211,200 cached input;
163,584 → 146,570 uncached input; 8,953 → 10,397 output; and
300.954 → 319.263 seconds. Cache state and model/service variation are not
separated; the aggregate does not establish a repeatable saving for an individual
question or a qualified performance recommendation.
Every answer passes exact core-field checks: both identities/byte counts, every
function selector/offset/dispatch/argument/mutability including null, inference,
unknown completeness/kind/hardfork/authenticity, and absent runtime execution.
All reported IDs occur in actual responses. However, the existing grader requires
the final limitation list to equal the producer list verbatim and in order.
Only three trials pass that additional check; nine fail. The questions request
preserving limitations but do not explicitly request a verbatim ordered list.
Four failures contain all original strings with extra correct scope statements
or different ordering. The other five paraphrase or omit source caveats, including
worker or upstream-metadata limitations. Do not silently waive those omissions
or turn these into twelve fully acceptable answers. The strict reports and
original transcripts remain intact; core-field success is a narrower result.
Model scope sampled peaks range from 562.5 to 710.5 MiB, with last-observed CPU
5.945–6.878 seconds. CPU includes the local process family, not remote model
computation. All heavy jobs use one CPU, a 4 GiB hard limit, 3 GiB early-stop,
no swap and 192 tasks. No scope abort occurs in this cohort. Final cgroup
counters are unavailable after teardown: final OOM/task/limit-event assertions
remain **unverified**. Last-observed counters are not substituted for final ones.
Consequently neither strict answer acceptance nor resource-qualified performance
success is claimed for the whole cohort.
## Existing-user compatibility
Omitted and explicit-complete MCP requests, the existing CLI and provider inputs
preserve their full behavior. `detail` is removed before the unchanged provider
input; original full Evidence parameters, raw result and identity are retained.
Summary receives its own identity linked to that parent. All candidate facts,
original report limitations, stderr/truncation and explicit unknowns stay inline.
Complete-within-view candidate coverage is not global discovery completeness.
Function pages retain original order; text pages use UTF-16 code units, with
hex characters distinguished from decoded-byte function offsets.
Actual new default replies for all three artifacts validate against the frozen
old advertised input/output schemas, and structured/text payloads agree. Older
servers reject neither request nor unsupported fields because the experimental
client discovers support before selecting summary and otherwise calls complete.
This is an explicit client fallback, not universal server forward compatibility.
Three authentic old-runtime records and old-created canonical bundles pass new
MCP import, retained summary, CLI inline summary and export. Original records,
IDs and bundle source bytes stay deeply equal. The reproducible verifier below
also checks function/hex pages with the analysis limiter explicitly unavailable,
so retained reads cannot silently start successful producer analysis. This
proves the pinned old/new Linux workflows, not every historical client release.
The initial retained experiment failed because its driver read the bundle's
wrong envelope. Correcting that consumer produced passing retained proofs; the
failed receipt stays archived and is not recorded as a REA defect. Separately,
one static-check attempt hit the unchanged 3 GiB early-stop. The bounded retry
passes with Node heap 512 MiB/native checker memory controls; its roughly 3 GiB
peak and soft-limit events are excluded from efficiency comparisons.
## Reproduction and checks
Use [the shared instructions](README.md), explicit frozen revisions/runtimes and
fresh private output directories. Run `measure.mjs` for `weth.hex`,
`uniswap-v3.factory.hex` and `uniswap-v3.pool.hex`, with `--schemas selected` and
paired `--delivery complete` / `summary`; run `compare.mjs` on equivalent pairs.
Add `--verify-retained` only for the separate four-call proof. Use the unchanged
[WETH](questions/evm.txt) or [template](questions/evm-uniswap.txt) question and the
same one-tool profile for `agent-trial.py`, preserving all usage and grader results.
Export complete Evidence with the old runtime's public ledger workflow, then run:
```sh
node experiments/analysis-efficiency/verify-evm-retention.mjs \
--runtime-root ABS_FROZEN_NEW_RUNTIME --revision NEW_SHA \
--bundle ABS_OLD_BUNDLE_JSON --output NEW_PRIVATE_ABS_DIR
```
The script acquires no dependencies and preserves the old bundle. Revision is
caller-supplied; retain the frozen file-table receipt alongside it. Raw JSON,
transcripts, profiles, bytecode, compiler inputs and generated artifacts stay
outside Git. This checkpoint adds source verification and curated observations.
The production change passes 64 focused EVM/WASM/generic-view/domain/service
checks and full static/docs gates. Recording provider seams establish exact
schemas, old defaults, inference/null preservation, historical identity failures,
peer availability, cancellation and cleanup retry; they are not real-provider
claims. Real-provider and shared-domain checks are recorded separately.
The same frozen candidate also passes six actual public MCP artifact workflows:
SQLite WASM with WABT summary, ShareX managed metadata, Obsidian inventory,
NewPipe with JADX, an OpenWrt region inspection, and jq entry analysis with
Ghidra. These are shared-contract regressions; they do not establish AST analysis
of all Obsidian code, full firmware reconstruction, browser coverage, or new token
savings in those domains. Native Windows PE and DOS coverage remains at the
[earlier native checkpoint](native-entry-views.md), not a new Ghidra/Windows claim.
The source retention verifier passes for all three authentic old EVM bundles,
including exact summaries, function/hex pages, CLI inline views and unchanged
exported originals with the limiter explicitly unavailable.
The frozen candidate passes all 59 existing real EVM CLI/MCP verifier cases with
caller-supplied solc 0.8.30 and strace 6.1, including malformed/unsupported
carriers, unavailable limiter and capacity failures. Its scope takes 96.076
seconds with 1,345,052,672 sampled peak bytes and 95.130 last-observed CPU seconds.
This is regression evidence, not an interpreter speedup comparison; final
resource counters remain unverified. Fresh fast static and generated-document
checks also pass after adding the migration verifier and this report.
## Latest-main integration
The branch rebases without conflict onto main
`0043d9b134c0a353e8bf85c0bde4492275f1f720`. Production integration commit
`3766749316c345ccb594b1b748747b1af9582b34` preserves the same EVM projection,
provider and delivery semantics; intervening main adds unrelated capabilities.
The above performance cohort stays pinned to its original frozen runtimes and
is not relabeled as a measurement of the larger integrated catalog.
At `cda794fb9d4a5ca2bd661853a899d579fb8a47b2`, all 30 affected EVM/WASM/generic-view
MCP tests pass again, as do full `npm run check` and `npm run docs:check`. A fresh
real Factory SDK selected-summary/retained-parent/page workflow and the original
Pool bundle's SDK/CLI migration verifier also pass against the rebuilt integrated
runtime. The latter still has an explicitly unavailable analysis limiter.
The initial integrated static-check scope stops at 3,221,446,656 sampled bytes,
just above the unchanged 3 GiB early-stop. Running the same typecheck through
its single Turbo task with less launcher overhead passes at 3,219,353,600 sampled
bytes; the subsequent full gate uses that valid cached typecheck and passes.
No memory threshold or check requirement is relaxed. Preserve both receipts;
these near-limit validation jobs are excluded from efficiency conclusions.
@@ -1,159 +0,0 @@
# EVM descriptor lifecycle acceptance
Continues [#2016](https://github.com/morluto/rea/issues/2016) after
[#2349](https://github.com/morluto/rea/pull/2349) merged. The
[public verifier](verify-evm-lifecycle.mjs) exercises actual compiled CLI and
stdio MCP workflows with unchanged EVMole 0.9.3/WASM. This increment adds
verification only; production contracts, factories, algorithms and defaults
remain unchanged. It does not introduce a plugin SDK or database cache.
## Runtime and public artifacts
The verified runtime is explicitly frozen from
`67f817e3653efad35f7044e68fbd6f163b86864b`, built with Node 24.18.0 and the
matching locked dependencies. Its `dist/` and `bridge/` file-table SHA-256 is
`cf03167870ac9def929cfaab536d6bd791089031660d1b35bff20672ac07cbbf`.
The table is checked again after each workflow; source and target bytes must
remain unchanged before a success receipt is published.
Inputs are the digest-pinned [corpus](corpus.json): published historical WETH
and official Uniswap v3-core 1.0.1 Factory/Pool compiler runtime templates.
They are not independently authenticated live deployments. Their deployment,
hardfork, immutable-reference and inferred-interface limitations remain as
recorded in [the descriptor checkpoint](evm-descriptor.md). The healthy peer
is the shipped SQLite WASM 3.53.4-build1 module, inspected by real WABT 1.0.42.
No contract, native target, chain/RPC query or transaction is executed.
## Actual cancellation and retained cleanup
The verifier supplies a private test launcher through the existing explicit
`REA_EVM_PRLIMIT_COMMAND` setting. It passes the original arguments to real
`/usr/bin/prlimit`, runs the unchanged Node/EVMole worker, then briefly gates
launcher completion after that worker exits and writes its complete reply.
This controls a real lifecycle boundary without replacing the analysis engine.
It adds process overhead and is not a performance comparison.
Before cancellation or permission changes, production inspection primitives
verify the live launcher's run token, parent/group relationship and native
start identity. The complete worker reply must match the original selected
bytecode and producer facts. Filesystem changes apply only to the verified,
non-root user's temporary runtime directory.
Three scenarios are verified:
- **Late cancellation:** the real worker has completed, while the public SDK
request remains pending. SDK cancellation reports `REQUEST_TIMEOUT` locally;
token sweeps and directory removal independently confirm provider cleanup.
A subsequent complete request returns identical analysis facts.
- **Successful analysis followed by cleanup failure:** removing write permission
produces actual `EACCES`. MCP reports `cleanup_incomplete` and retains the
original resource path. A retry before permission repair fails on that same
resource. Real SQLite WASM analysis still succeeds. Restoring permission lets
the provider retire the old resource and perform a fresh complete EVM query.
- **Format failure followed by cleanup failure:** a deliberately malformed
odd-nibble carrier reaches the unchanged worker. The cleanup error preserves
its original `invalid_input` category and exact producer diagnostic rather
than overwriting the failure. The same repair/retry and healthy-peer checks
succeed.
The existing CLI `inspect-evm-interface PATH hex --json` produces the same
complete normalized facts, parameters and subject as MCP. Fresh Evidence IDs
are not equated. Final verifier lineage is checked after client/transport close;
an empty descendant list is a point-in-time observation, not proof that the
workflow never launched children. Cleanup callbacks always run and preserve
verification and cleanup failures together.
This proves cancellation after actual analysis, not interruption inside
EVMole's synchronous WASM computation. Server-shutdown cleanup retry still has
the separately labeled injected-owner SDK tests from the descriptor checkpoint.
These Linux x64 observations do not establish Windows/macOS provider behavior.
Earlier real CLI/MCP malformed/unavailable/capacity checks remain linked in the
descriptor checkpoint; they are not silently rerun or attributed to this script.
## Measurements and compatibility scope
Initial three-carrier verifier source SHA-256 is
`2fec069b53324ef82070d323b26dd613b6da0caa7f18100adbeb39b045cf98dc`.
All three carriers pass all three scenarios and CLI parity with this same source.
| Carrier | Seconds | Sampled peak family bytes | Last-observed CPU seconds |
| ------- | ------: | ------------------------: | ------------------------: |
| WETH | 25.595 | 1,116,884,992 | 23.926 |
| Factory | 23.979 | 1,132,994,560 | 23.419 |
| Pool | 24.589 | 1,141,415,936 | 24.043 |
Runs are serial under the existing one-CPU quota, 4 GiB hard memory, unchanged
3 GiB early-stop, no swap and 192-task ceiling. Resource receipts, complete
producer replies, derived malformed input, launcher and runtime copies stay in
the private lab outside Git. Final cgroup counters disappear at scope teardown
and remain unknown; sampled zero events do not prove final no-OOM/no-limit
results. This is compatibility and recovery evidence, not a model-token,
whole-task latency or CPU saving claim.
Latest main `81cca2e0e3ebd98c21f41d7ad63d6a00e4cbc3f9` could not build during
setup: its shared partial-error handler passes three arguments to an interface
requiring four. Independently completed main CI at
`fdd8f826023c568937d237c2f97d855de283a9b8` reports the same TypeScript failure
([run 38309578208](https://github.com/morluto/rea/actions/runs/38309578208)).
The unrelated regression is recorded in the maintainer-requested private bug
handoff; this change does not repair it. The maintainer independently corrected
the producing caller in main `8fc2043eb8bcedbf34dce064f977e455438635d6`.
The frozen-runtime observations above remain pinned to their original revision;
synchronizing the branch does not relabel them as latest-main verification.
After rebasing, a fresh build and all 10 focused EVM MCP boundary tests pass.
The same unchanged source verifier also passes all three real Factory scenarios,
CLI parity, healthy SQLite WASM checks and final cleanup on the newly built
runtime `b3eadf147221a2fa997ac26667a183ca5bccb1f4` (parent is corrected main
`8fc2043eb8bcedbf34dce064f977e455438635d6`). Its `dist/`/`bridge/` file-table
SHA-256 is `7a4cec0fa4f7e929356b50a6b13472f80bd6f526221e427125370cca3ef98d00`.
This related rerun takes 24.285 seconds, samples 1,146,167,296 peak family bytes
and records 23.662 last-observed CPU seconds. The older failed build and earlier
cohort receipts are preserved; wider domain/model matrices are not repeated.
Static composition continues to bind one named canonical contract and one
typed factory. Existing SDK checks reject duplicate registration before use
and reject malformed inputs before invoking the provider. Producer contract
violations are detected when the producer returns a result; they cannot be
validated in advance without that result. A dynamic third-party module ABI
preflight is not implemented or claimed. The corresponding broader acceptance
item remains open rather than creating the deferred generic plugin registry.
## Final launcher review
A missing gate marker while gating is disabled cannot establish absence of a
new launch. The fixture now appends a launch record before starting the actual
worker; both permission-failure retries must leave that log byte-identical.
This independently verifies that retained cleanup failure stops a new EVM
launch, while preserving the original resource and healthy WASM peer.
Final verifier source SHA-256 is
`fc9a50e2e54e4c8eede4b8a19283e11ddaa1faadea52eebc390b55b5a1902668`.
All three related real Factory scenarios, CLI parity and final cleanup pass
again on the corrected-main runtime: 23.579 seconds, 1,163,685,888 peak sampled
family bytes and 23.110 last-observed CPU seconds. The earlier three-carrier
receipts remain pinned to their earlier source; wider artifacts/models are
not repeated for this review fix.
## Reproduction
Build and freeze the explicit source revision as documented in the measurement
README. Supply an object of explicit string-valued provider settings, including
the caller-installed WABT paths. As a non-root Linux x64 user, run under the
existing resource wrapper:
```bash
python3 experiments/analysis-efficiency/resource-run.py -- \
node experiments/analysis-efficiency/verify-evm-lifecycle.mjs \
--runtime-root /absolute/frozen-runtime \
--revision 67f817e3653efad35f7044e68fbd6f163b86864b \
--path /absolute/UniswapV3Factory.runtime.hex \
--peer-path /absolute/sqlite3.wasm \
--providers /absolute/explicit-providers.json \
--output /absolute/new-private-receipt-directory
```
The verifier checks the carrier and SQLite module against the committed corpus,
creates an owner-only receipt directory without overwriting existing receipts,
and acquires no dependencies. Preserve failed attempts separately. A shorter
reply or a successful cleanup verifier does not establish an agent-cost gain.
@@ -1,22 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.rea.components" android:versionCode="1" android:versionName="1">
<uses-sdk android:minSdkVersion="21" android:targetSdkVersion="30"/>
<application android:permission="com.rea.components.PRIVATE" android:enabled="true" android:process=":application">
<activity android:name=".Main" android:exported="true" android:enabled="false" android:process=":activity">
<intent-filter android:priority="7">
<action android:name="android.intent.action.VIEW"/>
<category android:name="android.intent.category.DEFAULT"/>
<data android:scheme="https" android:host="example.test" android:pathPrefix="/open"/>
</intent-filter>
<intent-filter android:priority="@integer/filter_priority">
<action android:name="android.intent.action.EDIT"/>
<data android:scheme="content" android:mimeType="text/plain"/>
</intent-filter>
<meta-data android:name="public.configuration" android:value="retained-in-manifest"/>
</activity>
<receiver android:name=".SmsReceiver" android:exported="false" android:enabled="@bool/receiver_enabled" android:process=":sms" android:permission="android.permission.BROADCAST_SMS">
<intent-filter android:priority="-10"><action android:name="android.provider.Telephony.SMS_RECEIVED"/></intent-filter>
</receiver>
<provider android:name=".DemoProvider" android:enabled="true" android:exported="false" android:authorities="com.rea.components.provider" android:grantUriPermissions="true"/>
</application>
</manifest>
@@ -1,15 +0,0 @@
import com.android.tools.smali.smali.Smali;
import com.android.tools.smali.smali.SmaliOptions;
import java.util.List;
/** Assemble the local fixture without Smali CLI's optional JCommander dependency. */
class BuildDex {
public static void main(String[] args) throws Exception {
SmaliOptions options = new SmaliOptions();
options.apiLevel = 30;
options.jobs = 1;
options.outputDexFile = args[1];
if (!Smali.assemble(options, List.of(args[0])))
throw new IllegalStateException("Manifest fixture assembly failed");
}
}
@@ -1,3 +0,0 @@
# A DEX container for a static manifest fixture; no Android app is executed.
.class public Lcom/rea/components/ManifestFixture;
.super Ljava/lang/Object;
@@ -1,43 +0,0 @@
# Static compiled-manifest fixture
These source-owned declarations cover URI data/filter association, exact resource
references, metadata names, component process/enablement and provider URI grants.
`ManifestFixture.smali` supplies a DEX container; it does not implement or execute
the declared Android components. The verification uses existing AAPT2 36.1.0,
Android 36 `android.jar`, JDK 21 and the digest-pinned JADX headless MCP 0.7.1 JAR.
No Android device, emulator or app execution is involved.
From the repository root, choose existing `AAPT2`, `ANDROID_JAR`, `JAVA_HOME` and
`REA_JADX_MCP_JAR` paths, then build into a fresh private directory:
```sh
fixture=experiments/analysis-efficiency/fixtures/android-components
work=$(mktemp -d)
"$AAPT2" compile --dir "$fixture/res" -o "$work/resources.zip"
"$AAPT2" link --manifest "$fixture/AndroidManifest.xml" -I "$ANDROID_JAR" \
-o "$work/manifest.apk" "$work/resources.zip"
"$JAVA_HOME/bin/java" -Xmx512m -XX:ActiveProcessorCount=2 \
-cp "$REA_JADX_MCP_JAR" "$fixture/BuildDex.java" \
"$fixture/ManifestFixture.smali" "$work/classes.dex"
python3 - "$work" <<'PY'
from pathlib import Path
import sys, zipfile
root = Path(sys.argv[1])
with zipfile.ZipFile(root / "manifest.apk", "a") as apk:
apk.write(root / "classes.dex", "classes.dex")
PY
"$AAPT2" dump xmltree --file AndroidManifest.xml "$work/manifest.apk" > "$work/aapt2-xmltree.txt"
"$AAPT2" dump resources "$work/manifest.apk" > "$work/aapt2-resources.txt"
REA_JADX_HEAP_MIB=1024 REA_JADX_ACTIVE_PROCESSOR_COUNT=2 \
node scripts/rea.mjs inspect-android-package "$work/manifest.apk" \
--detail summary --view '{"kind":"android","facet":"components"}' \
--evidence_output "$work/evidence.json" --json > "$work/cli.json"
python3 "$fixture/verify-fixture.py" "$work"
```
`verify-fixture.py` cross-checks concrete compiled XML facts with the independent
AAPT2 dump, structured CLI facts, parent APK identity and every retained manifest
source location. It deliberately keeps resource declarations unresolved even
though the resource table contains values. `manifest-oracle.py` is a separate
stdlib ElementTree oracle used by the public MCP task driver. Generated binaries,
Evidence and dumps stay outside Git.
@@ -1,70 +0,0 @@
"""Independent stdlib XML oracle over retained manifests in this verification corpus."""
import json
import sys
import xml.etree.ElementTree as ET
ANDROID = "{http://schemas.android.com/apk/res/android}"
KINDS = {"activity", "activity-alias", "service", "receiver", "provider"}
DATA_ATTRIBUTES = {
"scheme": "scheme", "host": "host", "port": "port", "path": "path",
"path_prefix": "pathPrefix", "path_pattern": "pathPattern",
"path_advanced_pattern": "pathAdvancedPattern", "path_suffix": "pathSuffix",
"mime_type": "mimeType",
}
def filter_facts(element):
def names(tag):
return [value.get(ANDROID + "name") for value in element.findall(tag)
if value.get(ANDROID + "name") is not None]
return {
"priority": element.get(ANDROID + "priority"),
"actions": names("action"), "categories": names("category"),
"data": [{key: value.get(ANDROID + attribute)
for key, attribute in DATA_ATTRIBUTES.items()}
for value in element.findall("data")],
}
def main():
with open(sys.argv[1], encoding="utf-8") as source:
bundle = json.load(source)
parent = next(record for record in bundle["records"] if record["evidence_id"] == sys.argv[2])
root = ET.fromstring(parent["normalized_result"]["manifest"]["text"])
package = root.get("package")
application = root.find("application")
assert application is not None and package, "Verification corpus requires a known package/application"
inventory = []
ordinal = 0
for component in application:
if component.tag not in KINDS:
continue
declared = component.get(ANDROID + "name")
if declared is None:
continue
filters = [filter_facts(value) for value in component.findall("intent-filter")]
enabled = component.get(ANDROID + "enabled")
permission = component.get(ANDROID + "permission")
if permission is None and component.tag != "activity-alias":
permission = application.get(ANDROID + "permission")
name = (package + declared if declared.startswith(".") else
package + "." + declared if "." not in declared else declared)
inventory.append({
"source_index": ordinal, "kind": component.tag,
"declared_name": declared, "name": name,
"enabled": True if enabled == "true" else False if enabled == "false" else None,
"enabled_declaration": enabled, "process": component.get(ANDROID + "process"),
"permission": permission or None, "intent_filters": filters,
"meta_data": [{"name": value.get(ANDROID + "name")}
for value in component.findall("meta-data")],
})
ordinal += 1
selected = [row for row in inventory
if any(sys.argv[3] in value["actions"] for value in row["intent_filters"])]
print(json.dumps({"components": selected, "inventory": inventory, "source_components": ordinal},
ensure_ascii=False, separators=(",", ":")))
if __name__ == "__main__":
main()
@@ -1,4 +0,0 @@
<resources>
<bool name="receiver_enabled">false</bool>
<integer name="filter_priority">80</integer>
</resources>
@@ -1,58 +0,0 @@
"""Check real AAPT2/JADX/CLI facts against the source-built manifest corpus."""
import hashlib
import json
from pathlib import Path
import sys
root = Path(sys.argv[1])
reply = json.loads((root / "cli.json").read_text())["summary"]
view = reply["normalized_result"]
bundle = json.loads((root / "evidence.json").read_text())
parent = next(record for record in bundle["records"]
if record["evidence_id"] == view["parent_evidence_id"])
sha256 = hashlib.sha256((root / "manifest.apk").read_bytes()).hexdigest()
assert parent["subject"]["digest"]["sha256"] == view["artifact"]["sha256"] == sha256
assert reply["evidence_links"] == [parent["evidence_id"]]
assert view["source_status"] == "complete" and view["source_components"] == 3
assert view["coverage"]["exhausted"] and view["coverage"]["total"] == 3
activity, receiver, provider = view["items"]
assert [row["source_index"] for row in view["items"]] == [0, 1, 2]
assert all(row["declaration_coverage"] == "complete" for row in view["items"])
assert activity["enabled"] is False and activity["process"] == ":activity"
first, second = activity["intent_filters"]
assert first["actions"] == ["android.intent.action.VIEW"] and first["priority"] == "7"
assert first["data"][0]["scheme"] == "https"
assert first["data"][0]["host"] == "example.test" and first["data"][0]["path_prefix"] == "/open"
assert second["actions"] == ["android.intent.action.EDIT"]
assert second["priority"] == "@integer/filter_priority"
assert second["data"][0]["scheme"] == "content" and second["data"][0]["mime_type"] == "text/plain"
assert activity["meta_data"][0]["name"] == "public.configuration"
assert "value" not in activity["meta_data"][0]
assert receiver["enabled"] is None and receiver["enabled_declaration"] == "@bool/receiver_enabled"
assert receiver["process"] == ":sms" and receiver["permission"] == "android.permission.BROADCAST_SMS"
assert receiver["intent_filters"][0]["priority"] == "-10"
assert provider["enabled"] is True and provider["grant_uri_permissions"] is True
assert provider["grant_uri_permissions_declaration"] == "true" and provider["process"] is None
manifest = parent["normalized_result"]["manifest"]["text"]
assert manifest.isascii() and "retained-in-manifest" in manifest
for row in view["items"]:
nodes = [(row, row["kind"])]
for element in row["intent_filters"]:
nodes.append((element, "intent-filter"))
nodes.extend((data, "data") for data in element["data"])
nodes.extend((metadata, "meta-data") for metadata in row["meta_data"])
for node, tag in nodes:
location = node["location"]
offset = location["offset"]
assert manifest[offset:].startswith("<" + tag)
assert manifest[:offset].count("\n") + 1 == location["line"]
assert offset - manifest.rfind("\n", 0, offset) == location["column"]
tree = (root / "aapt2-xmltree.txt").read_text()
for fragment in [':scheme(0x01010027)="https"', ':host(0x01010028)="example.test"',
':pathPrefix(0x0101002b)="/open"', ':priority(0x0101001c)=7',
':priority(0x0101001c)=-10', ':priority(0x0101001c)=@0x7f020000',
':enabled(0x0101000e)=@0x7f010000', ':grantUriPermissions(0x0101001b)=true']:
assert fragment in tree, fragment
print(json.dumps({"compiled_manifest_cli": "passed", "sha256": sha256,
"components": 3, "filter_grouping": True, "locations": True,
"resource_values_remain_declarations": True, "aapt2_oracle": True}))
@@ -1,30 +0,0 @@
<!doctype html>
<meta charset="utf-8" />
<title>Immediate saved-response follow-up fixture</title>
<button id="run">Exchange and follow up</button>
<output id="result">pending</output>
<output id="sensitive"></output>
<script>
document.querySelector("#run").onclick = async () => {
const source = await (
await fetch("/escaped-values.json", { cache: "no-store" })
).json();
document.querySelector("#sensitive").textContent = source.token;
console.log(source.token);
const response = await fetch(
"/api/collections/rea_items/records?token=" +
encodeURIComponent(source.token),
{
method: "POST",
headers: {
"Content-Type": "application/json",
"X-Followup": source.token,
},
body: JSON.stringify({ token: source.token }),
},
);
const result = document.querySelector("#result");
result.textContent = String(response.status);
result.dataset.ready = "yes";
};
</script>
@@ -1,39 +0,0 @@
<!doctype html><meta charset="utf-8" /><title>
PocketBase saved value verifier
</title>
<label>Email<input id="email" /></label
><label>Password<input type="password" id="password" /></label
><button id="login">Sign in</button><button id="create">Create record</button>
<pre id="result"></pre>
<script>
document.querySelector("#login").onclick = async () => {
const r = await fetch("/api/collections/_superusers/auth-with-password", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
identity: document.querySelector("#email").value,
password: document.querySelector("#password").value,
}),
});
const data = await r.json();
localStorage.setItem("rea_auth", JSON.stringify(data));
document.querySelector("#result").textContent = r.ok
? "Signed in"
: "Sign in failed";
if (r.ok) document.querySelector("#result").dataset.ready = "yes";
};
document.querySelector("#create").onclick = async () => {
const token = JSON.parse(localStorage.getItem("rea_auth") || "{}").token;
const r = await fetch("/api/collections/rea_items/records", {
method: "POST",
headers: {
"Content-Type": "application/json",
...(token ? { Authorization: token } : {}),
},
body: JSON.stringify({ title: "rea-ui-record" }),
});
document.querySelector("#result").textContent = JSON.stringify(
await r.json(),
);
};
</script>
@@ -1,15 +0,0 @@
<!doctype html>
<meta charset="utf-8" />
<title>Declared JSON payload privacy fixture</title>
<output id="result">pending</output>
<script>
fetch("/api/collections/rea_items/records", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: localStorage.getItem("whole_json"),
}).then((response) => {
const result = document.querySelector("#result");
result.textContent = String(response.status);
result.dataset.ready = "yes";
});
</script>
@@ -1,160 +0,0 @@
# Dart AOT summary and retained string pools
Continues [#2016](https://github.com/morluto/rea/issues/2016) and
[#2455](https://github.com/morluto/rea/pull/2455). The selected analyst question
asks for shipped LocalSend target/ABI/library identity, every nullable snapshot
observation, string-pool lower bounds and returned counts, and original coverage
and limitations. It does not require individual observed strings.
## Compatibility and ownership
MCP `inspect_dart_aot` accepts explicit `detail: "summary"`. Omitted and complete
requests preserve the complete result and original provider inputs. The CLI
remains complete. ABI omission and explicit ABI selection retain their original
meaning; the provider never receives the delivery field. Missing retention,
invalid detail and pre-cancellation fail before provider work.
The existing application workflow retains the complete original Evidence, then
returns a distinct derived summary linked to its parent ID. It preserves target
and libapp digests, every snapshot section and nullable fact, original pool and
overall coverage, lower-bound counts and limitations. `returned_counts` reports
the lengths of the producer-returned arrays; it does not upgrade lower bounds
to a complete snapshot inventory. The analysis and original source retention
are unchanged. No new engine, persistent cache or plugin SDK is introduced.
Read exact retained strings using `inspect_analysis_view`, the parent ID, and
`view: {kind: "dart-aot", collection: "package_uris", offset: 0, limit: 64}`.
Other collections are `dart_uris` and `class_like_tokens`. Follow the returned
next offset. Pages preserve source order, ABI/library identity and partial pool
coverage without opening the APK again. Page exhaustion describes retained
strings, not the entire snapshot. Strings remain byte-pattern observations,
rather than recovered typed class/function semantics. Missing historical
subjects remain explicitly unknown; conflicting recorded target digests fail.
## Frozen runtime and real SDK pairs
The official LocalSend 1.18.2 arm64 APK acquisition, size and SHA-256 are pinned
in `corpus.json`. No APK installation, target execution or version-dataset
lookup occurred. Source runtime: `811034a7283c9751a3ecbff33e8b2f28bec55793`;
compiled dist/bridge file-table SHA-256:
`dabbb63536c5862be839ecdc0f3eaad05b29c2a3816f57041528d30f754311be`.
The previous runtime is `28c84e78cf2fbfde4aceed345b773c59eb889f68`, digest
`5332dda13981f043e146b3601ebec1f380d9f698280ca53d75786b638a912e11`.
Its production sources were verified byte-equal to main f254 at acquisition.
Subsequent test-only changes and main's SAZ addition do not relabel these pins.
Node is 24.18.0 and the SDK client/server are 2.3.1. Each workflow starts a fresh
server and validates the selected live advertised schema. Artifact bytes,
provider environment/search-path fingerprints, driver and locked dependencies
stay equal. Host page caches are not flushed. Heavy tasks run serially under
the unchanged CPU, memory, task and host guards.
| Trial | AOT reply bytes | SDK workflow seconds | Payload CPU seconds | Kernel peak bytes |
| -------------- | --------------: | -------------------: | ------------------: | ----------------: |
| sdk-complete-1 | 151,341 | 5.139 | 6.819 | 691,441,664 |
| sdk-summary-1 | 12,037 | 5.219 | 6.608 | 639,430,656 |
| sdk-summary-2 | 12,037 | 5.064 | 6.405 | 640,868,352 |
| sdk-complete-2 | 151,341 | 5.128 | 6.468 | 633,614,336 |
Both pairs use one successful public tool call and pass settings/fact comparison,
including all snapshot fields, coverage, original limitations and all counts.
Reply bytes fall 92.05%. Whole SDK time stays approximately 5.1 seconds; these
trials do not establish engine speed, CPU or memory savings. Payload CPU includes
driver/catalog work beyond the SDK workflow timer. Catalog JSON grows from
2,506,213 bytes on the previous runtime to 2,514,896 (about 0.35%).
A fresh old-runtime complete request passes comparison against candidate
complete, including identical 151,341-byte replies. This is scoped compatibility
evidence, not a guarantee for every historical release or third-party client.
## Real old Evidence continuation
Fifteen successful public MCP calls import genuine old-runtime AOT Evidence,
project retained summary, reconstruct all 500 package URIs, 216 Dart URIs and
2,000 class-like tokens, compare inline portable summary, and export the original
record. The APK is absent during all these operations. Original string order,
lower bounds, partial coverage, nullable snapshot facts, subject and parent ID
remain intact; the original exported record is deeply equal to its source.
The first private verification attempt used a nonexistent receipt filename and
failed before import or artifact relocation. That driver error is retained;
the corrected attempt selects the fresh old-runtime record. It is not a REA
defect or a successful migration trial.
## Actual Codex task outcomes
Four serial trials use the same frozen candidate in complete/summary/summary/
complete order. Requested configuration is Codex CLI 0.161.0, `gpt-5.6-terra`
high, one exact enabled tool (`inspect_dart_aot`), and identical optional owned
context capture/discovery guidance. This requested alias does not certify the
server-resolved model. Temporary configuration points to the modified frozen
REA MCP; all copied credential profiles are removed and target bytes remain
unchanged. The source question is fixed; equal treatment instructions choose
detail and explain the summary carrier and parent reference.
| Trial | Input | Cached | Uncached | Output | Seconds | MCP attempts | Failed attempts | Facts |
| ---------------- | -----: | -----: | -------: | -----: | ------: | -----------: | --------------: | ----- |
| model-complete-1 | 74,641 | 50,176 | 24,465 | 2,765 | 116.898 | 3 | 1 | pass |
| model-summary-1 | 38,253 | 22,016 | 16,237 | 1,449 | 88.904 | 1 | 0 | pass |
| model-summary-2 | 38,363 | 22,016 | 16,347 | 1,515 | 55.190 | 1 | 0 | pass |
| model-complete-2 | 66,585 | 38,144 | 28,441 | 2,217 | 79.180 | 2 | 0 | pass |
Both pairs pass checks for every selected producer fact, exact nullable
observations, target digest, observed response Evidence IDs, original limitations,
artifact preservation and profile cleanup. The grader compares the reported
facts with producer observations; it is not an independent Dart deserializer.
Total input falls 48.75% / 42.38%, uncached input 33.63% / 42.52%, reported output
47.59% / 31.66%, and whole-task time 23.95% / 30.30%. Usage is the final aggregate
reported by the actual CLI, one usage record per trial; cached tokens are not
subtracted from total input. Reported output includes reasoning output (1,262,
321, 383 and 709 tokens respectively).
The first complete trial initially supplied `artifact_path` instead of the
advertised `path`, corrected it, then repeated a successful complete call.
The second complete trial made two successful calls. Both summary trials made
one successful call. All attempts, discovery and correction costs remain
included. The paired whole-task improvement therefore includes actual model
decisions and the convenience of explicit returned counts; it does not isolate
serialization as the sole cause or establish engine acceleration. Two pairs on
one app/question/model configuration support this scope, not a universal gain.
Payload CPU seconds / kernel peak bytes are respectively 12.556 / 876,937,216;
9.213 / 785,272,832; 9.108 / 793,939,968; and 13.164 / 803,209,216. Model payload
observations verify owned cleanup and zero phase limit events, with no guard
aborts. Whole-scope final counters remain unknown, so every strict all-selected
resource grade remains false even though the separate factual grade passes.
## Resource and verification scope
SDK and migration payload-exit observations verify owned cleanup and zero
observed memory/pids limit events, with peaks below 692 MB. Missing whole-scope
terminal counters remain unknown; the strict whole-resource grader stays false.
One-CPU quotas are experiment controls rather than CPU-efficiency claims.
Production-provider fixture tests cover explicit ABI, old complete defaults,
actual SDK-converted schemas, exact pages after APK removal, portable records,
nullable/partial facts, malformed inputs, target digest conflicts and early
refusals. CLI/MCP acceptance covers original report equality, duplicate-member
refusals and unsupported snapshot kinds. Fixture successes are distinguished
from the real LocalSend SDK/migration workflows. Other Flutter apps, architectures,
SDK layouts, typed class recovery and runtime behavior remain unverified here.
Raw replies, binaries, model traces and temporary credentials stay outside Git.
## Integration with the newer main
The rebased verification runtime is `63984ce0898bd7997e6d9b138e6d96d0a24f2b09`
on main `a2d8b94dd50dab8d966239dd63ab84718aa8b4b4`, with compiled file-table
SHA-256 `19c8ccdd3e99bb46a5c54f4edd8c6b2d2eb4f3a4870b3c129061d38145382b68`.
Targeted Flutter/SAZ/shared-view integration passes 35 tests; one opposite
runtime-capacity branch is skipped because this host enforces decoder memory.
Documentation checks and two README conformance tests pass.
Fresh real complete/summary SDK checks on this runtime match the earlier selected
facts and settings, with unchanged 151,341 / 12,037-byte AOT replies. Fifteen
old-Evidence migration calls again preserve every original string and record
with the source APK absent. The real one-shot CLI complete result also matches
the previous-runtime normalized report, subject, parameters and original ID.
An initial CLI-only controller check lacked an owned resource receipt; it is
retained and excluded from resource/performance claims, then repeated in the
bounded owned scope. These checks establish integration, rather than relabeling
the earlier paired model/SDK measurements as the rebased runtime.
@@ -1,81 +0,0 @@
"""Count complete actual-client requests, not only smaller tool-reply bytes."""
import argparse
import hashlib
import json
from pathlib import Path
import tiktoken
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--lab', type=Path, required=True)
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
enc = tiktoken.get_encoding('cl100k_base')
def tokens(value):
return len(enc.encode(json.dumps(value, ensure_ascii=False, separators=(',', ':'), allow_nan=False), disallowed_special=()))
def resources(root):
receipt = json.loads(Path(str(root) + '.log').read_text().splitlines()[-1])
payload = receipt['payload_resource_observation']
assert receipt['abort'] is None and receipt['payload_resource_observation_verified']
assert payload['payload_exit_code'] == 0 and payload['payload_cleanup_verified']
for events in payload['resource_events'].values():
assert not any(events.values())
return {'wall_seconds': receipt['elapsed_seconds'], 'cpu_seconds': payload['cpu_stat']['usage_usec'] / 1e6,
'family_peak_mib': payload['memory_peak_bytes'] / 1048576,
'payload_cleanup_verified': True, 'final_whole_scope_counters_verified': receipt['resource_counters_verified']}
wire, sdk = [], []
for root in sorted(args.lab.glob('wire-v2-*')):
if not root.is_dir():
continue
report = json.loads((root / 'report.json').read_text())
assert report['status'] in ['passed', 'expected-incomplete-task'] and not report['cleanup_failures']
assert report['appserver_exit_code'] == 0 and report['owned_scope_drain']['remaining_processes'] == []
requests = json.loads((root / 'model-requests.json').read_text())
details = []
for captured in requests:
body = captured['body']
namespace = [n for n in body.get('tools', []) if n.get('type') == 'namespace' and n.get('name') == 'mcp__rea_audit']
assert len(namespace) == 1 and len(namespace[0]['tools']) == 211, 'Unexpected compaction/catalog: audit before comparing'
details.append({'whole_json_tokens_proxy': tokens(body), 'rea_namespace_tokens_proxy': tokens(namespace[0]), 'rea_functions': 211})
wire.append({'case': root.name, 'task_completed': report['task_completed'], 'mode': report['mode'],
'budget': report['observations']['tool_output_budget'], 'requests': len(details), 'compactions': report['compaction_requests'],
'whole_json_tokens_proxy': sum(r['whole_json_tokens_proxy'] for r in details), 'request_details': details, **resources(root)})
for root in sorted(args.lab.glob('sdk-*')):
if not root.is_dir():
continue
report = json.loads((root / 'report.json').read_text())
assert report['status'] == 'passed' and not report['cleanup_failures'] and not report['receipt_failures']
transcript = json.loads((root / 'transcript.json').read_text())
catalog = json.loads((root / 'catalog.json').read_text())
assert len(catalog['tools']) == 211 and report['original_target_exited']
sdk.append({'case': root.name, 'detail': report['detail'], 'capture': report['capture'], 'tool_calls': len(report['calls']),
'exported_bundle_bytes': report['exported_bundle_bytes'], 'answer': report['answer'],
'status_calls': [c for c in report['calls'] if c['name'] == 'frida_session_status'],
'one_catalog_plus_all_sdk_calls_tokens_proxy': tokens(catalog) + sum(tokens({'name': t['name'], 'input': t['input'], 'reply': t['reply']}) for t in transcript), **resources(root)})
assert len(sdk) == len(wire) == 4, 'A partial cohort cannot qualify a comparison'
for capture in ['small', 'libc']:
pair = [r for r in sdk if r['capture'] == capture]
assert len(pair) == 2 and {r['detail'] for r in pair} == {'complete', 'summary'}
assert {k: v for k, v in pair[0]['answer'].items() if k != 'pid'} == {k: v for k, v in pair[1]['answer'].items() if k != 'pid'}
reports = [json.loads((args.lab / r['case'] / 'report.json').read_text()) for r in pair]
assert reports[0]['load']['source_sha256'] == reports[1]['load']['source_sha256']
completed = [r for r in wire if r['budget'] == 300000]
assert len(completed) == 2 and all(r['task_completed'] for r in completed)
answers = [json.loads((args.lab / r['case'] / 'report.json').read_text())['observations']['answers'] for r in completed]
comparable_answers = [[{k: v for k, v in a.items() if k != 'parent_evidence_id'} for a in run] for run in answers]
assert comparable_answers[0] == comparable_answers[1]
result = {
'scope': 'Real local Frida binding with controlled sleep target and caller-authored libc export inventory. Actual Codex0.163 app-server uses a deterministic local SSE fixture: no AI inference, model usage, quality or billing claim. Whole compact JSON token estimates include all 211 schemas and every request replay. Incomplete default baseline is excluded from completed-task gains. SDK exchange estimates are not model-input measurements. Fresh processes have unflushed OS caches; both status reads query live session observations. Timings include the eight-second target lifetime and cleanup; single pairs establish no general speedup. Payload-exit cgroup counters are verified; final whole-scope counters may be unavailable.',
'tokenizer': {'package': tiktoken.__version__, 'encoding': enc.name},
'runtime_pins': {v: {k: x for k, x in json.loads((args.lab / (v + '-manifest.json')).read_text()).items() if k != 'files'} for v in ['baseline', 'candidate']},
'native': json.loads((args.lab / 'native-prerequisite-manifest.json').read_text()),
'script_pins': {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in Path(__file__).parent.glob('frida*') if p.suffix in ['.py', '.mjs']},
'wire': wire, 'sdk': sdk,
}
args.output.write_text(json.dumps(result, indent=2) + '\n')
print(json.dumps({'wire': [{k: v for k, v in r.items() if k != 'request_details'} for r in wire], 'sdk': [{k: v for k, v in r.items() if k not in ['status_calls', 'answer']} for r in sdk]}, indent=2))
@@ -1,267 +0,0 @@
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { pathToFileURL } from "node:url";
import { finished } from "node:stream/promises";
import { parseArgs } from "node:util";
import { createRetainedQueryCaller } from "./retained-query-calls.mjs";
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
const { values } = parseArgs({
options: {
runtime: { type: "string" },
output: { type: "string" },
detail: { type: "string", default: "complete" },
capture: { type: "string", default: "libc" },
},
});
const { runtime, output, detail, capture } = values;
assert.ok(runtime && output);
assert.ok(["complete", "summary"].includes(detail));
assert.ok(["libc", "small"].includes(capture));
const require = createRequire(join(runtime, "package.json"));
const { Client } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client"))
);
const { StdioClientTransport } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio"))
);
const { parseLinuxProcessStat } = await import(
pathToFileURL(join(runtime, "dist/process/LinuxProcessStat.js"))
);
await mkdir(output, { mode: 0o700 });
const account = await mkdtemp(join(tmpdir(), "rea-frida-real-"));
const serverLog = createWriteStream(join(output, "server.log"));
const serverLogFinished = finished(serverLog);
serverLogFinished.catch(() => {});
const client = new Client({ name: "rea-frida-audit", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(runtime, "scripts/rea.mjs"), "mcp"],
env: {
PATH: process.env.PATH,
HOME: account,
NODE_OPTIONS: "--max-old-space-size=2048",
UV_THREADPOOL_SIZE: "1",
REA_LOG_LEVEL: "silent",
},
stderr: "pipe",
});
transport.stderr?.pipe(serverLog, { end: false });
const report = {
question:
"Retain a complete caller-requested libc export inventory, then check only session identity/state and message capture coverage twice, preserving complete Evidence.",
detail,
capture,
calls: [],
status: "pending",
};
const transcript = [];
const caller = createRetainedQueryCaller(client, (receipt, event) => {
report.calls.push(receipt);
transcript.push(event);
});
const call = async (name, input) => {
const r = await caller(name, input, name);
assert.ok(!r.isError, JSON.stringify(r));
return r.structuredContent;
};
const delay = (ms) => new Promise((r) => setTimeout(r, ms));
let session;
let birth;
const stat = async () => {
try {
return parseLinuxProcessStat(
await readFile(`/proc/${session.pid}/stat`, "utf8"),
);
} catch (e) {
if (e.code === "ENOENT") return null;
throw e;
}
};
try {
await client.connect(transport);
const catalog = await client.listTools();
await writeFile(join(output, "catalog.json"), JSON.stringify(catalog));
report.catalog_tools = catalog.tools.length;
const defs = new Map(catalog.tools.map((t) => [t.name, t]));
const orig = client.callTool.bind(client);
client.callTool = (input, options) =>
orig(input, {
...options,
timeout: 30000,
toolDefinition: defs.get(input.name),
});
report.devices = await call("list_frida_devices", {});
session = await call("start_frida_session", {
mode: "spawn",
device_id: "local",
program: "/usr/bin/sleep",
argv: ["8"],
});
report.session = session;
birth = await stat();
assert.ok(birth);
report.target_birth = birth;
report.same_owned_cgroup =
(await readFile(`/proc/${session.pid}/cgroup`, "utf8")) ===
(await readFile("/proc/self/cgroup", "utf8"));
assert.ok(report.same_owned_cgroup);
const source = `send({kind:'ready',pid:Process.id});
setTimeout(() => {
if (${JSON.stringify(capture)} === 'small') { send({kind:'complete',total:0}); return; }
const module = Process.enumerateModules().find(m => /(^|\\/)libc\\.so\\.6$/.test(m.path));
if (!module) { send({kind:'missing-libc'}); return; }
const exports = module.enumerateExports().map(e => ({type:e.type,name:e.name,address:e.address.toString()}));
send({kind:'module',pid:Process.id,path:module.path,base:module.base.toString(),size:module.size,total:exports.length});
for (let offset = 0; offset < exports.length; offset += 64) send({kind:'exports',offset,items:exports.slice(offset,offset+64)});
send({kind:'complete',total:exports.length});
}, 300);`;
await writeFile(join(output, "selected-script.js"), source);
report.source_sha256 = createHash("sha256").update(source).digest("hex");
const loaded = await call("load_frida_script", {
session_id: session.session_id,
source_kind: "inline",
source,
});
assert.equal(loaded.normalized_result.source_sha256, report.source_sha256);
report.load = loaded.normalized_result;
await call("resume_frida_session", { session_id: session.session_id });
await delay(750);
const snapshots = [];
for (let n = 0; n < 2; n++)
snapshots.push(
await call("frida_session_status", {
session_id: session.session_id,
...(detail === "summary" ? { detail } : {}),
}),
);
await call("export_evidence_bundle", { path: join(output, "evidence.json") });
const bundle = JSON.parse(
await readFile(join(output, "evidence.json"), "utf8"),
);
for (const snapshot of snapshots) {
assert.equal(snapshot.raw_result, null);
const parent =
detail === "summary"
? bundle.records.find(
(e) =>
e.evidence_id === snapshot.normalized_result.parent_evidence_id,
)
: snapshot;
assert.ok(parent);
assert.equal(parent.authority, "external-service");
assert.equal(parent.environment, null);
assert.equal(parent.raw_result, null);
assert.equal(parent.operation, "frida_session_status");
assert.deepEqual(parent.parameters, { session_id: session.session_id });
const r = parent.normalized_result;
assert.equal(r.pid, session.pid);
assert.equal(r.state, "running");
assert.equal(r.messages_truncated, false);
const payloads = r.messages
.filter((m) => m.type === "send")
.map((m) => m.payload);
const ready = payloads.find((p) => p?.kind === "ready");
const complete = payloads.find((p) => p?.kind === "complete");
assert.equal(ready?.pid, session.pid);
assert.ok(complete);
let count = 0;
let header;
if (capture === "libc") {
header = payloads.find((p) => p?.kind === "module");
assert.ok(header);
assert.equal(header.pid, session.pid);
for (const chunk of payloads.filter((p) => p?.kind === "exports")) {
assert.equal(chunk.offset, count);
count += chunk.items.length;
}
assert.equal(count, header.total);
assert.ok(count > 1000);
const maps = await readFile(`/proc/${session.pid}/maps`, "utf8");
assert.ok(maps.includes(header.path));
}
assert.equal(count, complete.total);
if (detail === "summary") {
const view = snapshot.normalized_result;
assert.equal(view.parent_operation, parent.operation);
assert.equal(view.parent_digest, parent.evidence_id.slice(3));
assert.equal(view.artifact, null);
assert.deepEqual(view.source_parameters, parent.parameters);
assert.deepEqual(view.source_provider, parent.provider);
const { messages, ...metadata } = r;
assert.deepEqual(view.summary, {
...metadata,
messages_retained: messages.length,
});
assert.ok(view.unknowns.length > 0);
assert.ok(view.limitations.some((m) => m.includes("operating system")));
assert.deepEqual(
bundle.records.find((e) => e.evidence_id === snapshot.evidence_id),
snapshot,
);
}
report.answer = {
pid: r.pid,
state: r.state,
scripts: r.scripts.length,
messages: r.messages.length,
messages_truncated: r.messages_truncated,
exports: count,
module_path: header?.path ?? null,
};
}
report.native_oracle =
"Caller-authored script digest and contiguous chunk coverage; PID/birthtime/cgroup and libc path independently checked in procfs. Export addresses and native export completeness not independently qualified.";
report.exported_bundle_bytes = Buffer.byteLength(
await readFile(join(output, "evidence.json")),
);
await call("close_frida_session", { session_id: session.session_id });
session.closed = true;
const deadline = performance.now() + 12000;
while (true) {
const current = await stat();
if (
current === null ||
current.startTime !== birth.startTime ||
["zombie", "dead"].includes(current.lifecycle)
) {
report.original_target_exited = true;
break;
}
assert.ok(
performance.now() < deadline,
"Original target exit not observed",
);
await delay(50);
}
report.status = "passed";
} catch (error) {
report.status = "failed";
report.failure = String(error?.stack ?? error);
process.exitCode = 1;
} finally {
if (session && !session.closed)
try {
await call("close_frida_session", { session_id: session.session_id });
} catch (error) {
report.session_cleanup_failure = String(error);
process.exitCode = 1;
}
if (
!(await finalizeRetainedQueryReceipt({
report,
client,
transport,
account,
serverLog,
serverLogFinished,
output,
transcript,
}))
)
process.exitCode = 1;
}
@@ -1,468 +0,0 @@
{
"scope": "Real local Frida binding with controlled sleep target and caller-authored libc export inventory. Actual Codex0.163 app-server uses a deterministic local SSE fixture: no AI inference, model usage, quality or billing claim. Whole compact JSON token estimates include all 211 schemas and every request replay. Incomplete default baseline is excluded from completed-task gains. SDK exchange estimates are not model-input measurements. Fresh processes have unflushed OS caches; both status reads query live session observations. Timings include the eight-second target lifetime and cleanup; single pairs establish no general speedup. Payload-exit cgroup counters are verified; final whole-scope counters may be unavailable.",
"tokenizer": {
"package": "0.12.0",
"encoding": "cl100k_base"
},
"runtime_pins": {
"baseline": {
"source_base": "17f581ff75d2b6cf643a997275e3e296f6c95760",
"patch_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"archive_sha256": "cd16ca3e36633459bffc10bb90d1fe4ff5887fdd403729765b9a812969013807"
},
"candidate": {
"source_base": "32a4690059907f09fb29e284ad492d94c1722278",
"patch_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"archive_sha256": "4ac402f1de93f76f71b7ccdb8b8e36d8c8d0a87b1184812f06393bfc6fadccba"
}
},
"native": {
"host": "Linux-6.1.0-53-cloud-amd64-x86_64-with-glibc2.36",
"toolchain": {
"node": "24.18.0",
"npm": "11.16.0"
},
"frida_binding_package_version": "17.23.2",
"files": [
{
"path": "/usr/bin/sleep",
"bytes": 43888,
"sha256": "4add4bb89d8ca0e3b1bd861130ddd7ae0fd9617a8055de0a38c8d2ca1ac95723"
},
{
"path": "/usr/lib/x86_64-linux-gnu/libc.so.6",
"bytes": 1926232,
"sha256": "6b4a45352fd0c540a9c7c718f35ce8c8e46a4e482f9d3885a910c32d1a0e1421"
},
{
"path": "/home/yann/yann/rea-perf-next-audit-20261020/node_modules/frida/package.json",
"bytes": 2252,
"sha256": "7efb59d2050b7b6304e1617f463c4eb7b2f7cb7e3e80e677976825a76ee31e8d"
},
{
"path": "/home/yann/yann/rea-perf-next-audit-20261020/node_modules/frida/build/frida_binding.node",
"bytes": 112775824,
"sha256": "5b2db5ae57c058ad0765e627d9d9c9c98e4e90c09746c6f727b7a80769eed7a5"
}
],
"qualifications": {
"Inspector": "real REA MCP target, concurrent native metadata only oracle, 776 rows matched hash/length; not global completeness or execution proof",
"Frida": "local native binding and controlled sleep spawn; caller script digest and procfs PID/birthtime/cgroup/libc path verified; native export address/global completeness oracle unavailable",
"ADB": "default executable unavailable in prior public call; no real device qualified"
}
},
"script_pins": {
"frida_status_delivery.py": "ed19bcd003165c7b522c8cdd4799bcf987178e4b79fc8d5c2442012f3bf60b8d",
"frida-status-costs.py": "2f3196bcc3198a265f7f4842effc8dce31669723b42f07367f065afee0073906",
"frida-status-queries.mjs": "c5919ae8bba046909858e92a945b4472645bbacbfcf88dd75c89ba959a986980",
"frida-status-wire.py": "b4d20e64bf77b7815bd43b52569b3e4756983b254ab44c50ddae874d1964ef69"
},
"wire": [
{
"case": "wire-v2-baseline-0",
"task_completed": false,
"mode": "complete",
"budget": "client default",
"requests": 8,
"compactions": 0,
"whole_json_tokens_proxy": 540065,
"request_details": [
{
"whole_json_tokens_proxy": 62729,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 62917,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63599,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63739,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 68261,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 72785,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 72926,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 73109,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
}
],
"wall_seconds": 15.993,
"cpu_seconds": 8.211179,
"family_peak_mib": 830.6484375,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "wire-v2-baseline-300000",
"task_completed": true,
"mode": "complete",
"budget": 300000,
"requests": 8,
"compactions": 0,
"whole_json_tokens_proxy": 1049997,
"request_details": [
{
"whole_json_tokens_proxy": 62715,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 62903,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63577,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63714,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 141108,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 218504,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 218646,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 218830,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
}
],
"wall_seconds": 15.418,
"cpu_seconds": 8.026944,
"family_peak_mib": 824.28125,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "wire-v2-candidate-0",
"task_completed": true,
"mode": "summary",
"budget": "client default",
"requests": 8,
"compactions": 0,
"whole_json_tokens_proxy": 515539,
"request_details": [
{
"whole_json_tokens_proxy": 62791,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 62978,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63661,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63803,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 64749,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 65697,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 65837,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 66023,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
}
],
"wall_seconds": 15.889,
"cpu_seconds": 7.939387,
"family_peak_mib": 823.69921875,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "wire-v2-candidate-300000",
"task_completed": true,
"mode": "summary",
"budget": 300000,
"requests": 8,
"compactions": 0,
"whole_json_tokens_proxy": 515713,
"request_details": [
{
"whole_json_tokens_proxy": 62804,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 62990,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63677,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63815,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 64772,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 65731,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 65870,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 66054,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
}
],
"wall_seconds": 15.439,
"cpu_seconds": 8.371664,
"family_peak_mib": 813.6328125,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
}
],
"sdk": [
{
"case": "sdk-baseline-libc",
"detail": "complete",
"capture": "libc",
"tool_calls": 8,
"exported_bundle_bytes": 214214,
"answer": {
"pid": 517427,
"state": "running",
"scripts": 1,
"messages": 50,
"messages_truncated": false,
"exports": 2963,
"module_path": "/usr/lib/x86_64-linux-gnu/libc.so.6"
},
"status_calls": [
{
"name": "frida_session_status",
"input": {
"session_id": "ef8982d3-dbf2-43cb-8256-3a4f4bc83cc3"
},
"phase": "frida_session_status",
"elapsed_ms": 197.58100300000024,
"reply_bytes": 462831,
"is_error": false
},
{
"name": "frida_session_status",
"input": {
"session_id": "ef8982d3-dbf2-43cb-8256-3a4f4bc83cc3"
},
"phase": "frida_session_status",
"elapsed_ms": 108.50572199999988,
"reply_bytes": 462831,
"is_error": false
}
],
"one_catalog_plus_all_sdk_calls_tokens_proxy": 1186065,
"wall_seconds": 13.759,
"cpu_seconds": 6.097489,
"family_peak_mib": 578.5625,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "sdk-baseline-small",
"detail": "complete",
"capture": "small",
"tool_calls": 8,
"exported_bundle_bytes": 2335,
"answer": {
"pid": 516970,
"state": "running",
"scripts": 1,
"messages": 2,
"messages_truncated": false,
"exports": 0,
"module_path": null
},
"status_calls": [
{
"name": "frida_session_status",
"input": {
"session_id": "3f1f778f-ca18-48ba-a7db-b58afbf4710d"
},
"phase": "frida_session_status",
"elapsed_ms": 79.25232200000028,
"reply_bytes": 2643,
"is_error": false
},
{
"name": "frida_session_status",
"input": {
"session_id": "3f1f778f-ca18-48ba-a7db-b58afbf4710d"
},
"phase": "frida_session_status",
"elapsed_ms": 6.851230000000214,
"reply_bytes": 2643,
"is_error": false
}
],
"one_catalog_plus_all_sdk_calls_tokens_proxy": 882323,
"wall_seconds": 13.039,
"cpu_seconds": 4.953337,
"family_peak_mib": 577.140625,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "sdk-candidate-libc",
"detail": "summary",
"capture": "libc",
"tool_calls": 8,
"exported_bundle_bytes": 216931,
"answer": {
"pid": 517901,
"state": "running",
"scripts": 1,
"messages": 50,
"messages_truncated": false,
"exports": 2963,
"module_path": "/usr/lib/x86_64-linux-gnu/libc.so.6"
},
"status_calls": [
{
"name": "frida_session_status",
"input": {
"session_id": "9af7f188-05f8-4559-b570-98556196090e",
"detail": "summary"
},
"phase": "frida_session_status",
"elapsed_ms": 159.16798299999937,
"reply_bytes": 5597,
"is_error": false
},
{
"name": "frida_session_status",
"input": {
"session_id": "9af7f188-05f8-4559-b570-98556196090e",
"detail": "summary"
},
"phase": "frida_session_status",
"elapsed_ms": 61.96135099999992,
"reply_bytes": 5597,
"is_error": false
}
],
"one_catalog_plus_all_sdk_calls_tokens_proxy": 885626,
"wall_seconds": 12.843,
"cpu_seconds": 5.012703,
"family_peak_mib": 579.0,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "sdk-candidate-small",
"detail": "summary",
"capture": "small",
"tool_calls": 8,
"exported_bundle_bytes": 5051,
"answer": {
"pid": 517162,
"state": "running",
"scripts": 1,
"messages": 2,
"messages_truncated": false,
"exports": 0,
"module_path": null
},
"status_calls": [
{
"name": "frida_session_status",
"input": {
"session_id": "1bcda0f5-e3cd-45ee-b82c-5886d8312bc6",
"detail": "summary"
},
"phase": "frida_session_status",
"elapsed_ms": 107.78244100000029,
"reply_bytes": 5595,
"is_error": false
},
{
"name": "frida_session_status",
"input": {
"session_id": "1bcda0f5-e3cd-45ee-b82c-5886d8312bc6",
"detail": "summary"
},
"phase": "frida_session_status",
"elapsed_ms": 5.996030000000246,
"reply_bytes": 5595,
"is_error": false
}
],
"one_catalog_plus_all_sdk_calls_tokens_proxy": 885676,
"wall_seconds": 13.533,
"cpu_seconds": 5.495048,
"family_peak_mib": 576.48046875,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
}
]
}
@@ -1,163 +0,0 @@
# Frida status delivery and runtime-observation audit
Issue: [#5640](https://github.com/morluto/rea/issues/5640). Paired runtime base:
`17f581ff75d2b6cf643a997275e3e296f6c95760`. Final candidate source: `32a4690059907f09fb29e284ad492d94c1722278` (clean tree).
Exact package, patch, native input,
script and resource pins are in [the measurement record](frida-status-views.json).
The analyst requests a complete libc export inventory as retained evidence,
then checks the session's identity/state, script identity, queued message count
and truncation twice. The caller-authored script enumerates actual native module
exports in a controlled `/usr/bin/sleep` spawn, in chunks of 64. Both workflows
retain all captured messages and export the complete Evidence bundle.
Start, status and one-shot contracts share one canonical session envelope.
Message JSON is validated at the Evidence owner; the summary validates the
envelope without reinterpreting opaque payloads.
The owning application service now accepts optional `detail=summary` for
`frida_session_status`. It reads the provider on every call, retains the original
complete Evidence, then uses the existing shared analysis-view projector.
The summary omits arbitrary message payloads and preserves all other status
fields, the retained message count, truncation, exact parent identity/digest,
provider and source parameters. Default and explicit complete delivery keep the
original Evidence identity and payload. Historical CLI/file projections use the
same projector. Source script decoding/representation Evidence and session
resource ownership are unchanged.
## Completed tasks
The native Frida Node binding package 17.23.2 captured 2,963 libc exports in 50 queued
messages. Independent checks establish the test-owned process birthtime, cgroup,
PID and libc mapping; the selected script digest and contiguous chunks are
checked against the complete retained parent. This does not independently
validate every native export address or establish global event coverage.
| Public SDK task | Complete | Summary |
| ------------------------------------------ | -------: | ------: |
| Large-history status reply, bytes per call | 462,831 | 5,597 |
| Small-history status reply, bytes per call | 2,643 | 5,595 |
| Large complete bundle, bytes | 214,214 | 216,931 |
| Small complete bundle, bytes | 2,335 | 5,051 |
| Large task elapsed, seconds | 13.759 | 12.843 |
| Large task CPU, seconds | 6.097 | 5.013 |
| Large family peak, MiB | 578.6 | 579.0 |
| Small task elapsed, seconds | 13.039 | 13.533 |
| Small task CPU, seconds | 4.953 | 5.495 |
| Small family peak, MiB | 577.1 | 576.5 |
Each SDK task discovers all 211 tools and executes eight public calls, including
two status reads and complete export. Small history contains two readiness/
completion messages. Fresh processes retain unflushed OS caches. The second
status read is a repeated live observation, not a cache hit. Every task includes
the controlled target's eight-second lifetime and verified natural exit.
Single pairs and this fixed wait establish no general latency/CPU advantage.
Large status calls took 159.2/62.0 ms versus 197.6/108.5 ms. The first small
summary status was slower (107.8 versus 79.3 ms).
## Actual client context
Codex app-server 0.163.0 performs the native Frida workflow through the actual
REA MCP package. A deterministic local Responses/SSE fixture selects the calls;
there is no AI inference, paid model, model-quality or billing claim. The capture
contains all 211 tool input schemas on each request and all history replays. `tiktoken`
0.12.0 `cl100k_base` counts compact JSON as a token estimate, not actual model
usage. Both completed paths use the same 300,000 tool-output budget.
| Completed client task | Complete | Summary |
| -------------------------------------- | --------: | ------: |
| Requests / compactions | 8 / 0 | 8 / 0 |
| Whole request JSON token estimate, sum | 1,049,997 | 515,713 |
| Elapsed seconds | 15.418 | 15.439 |
| CPU seconds | 8.027 | 8.372 |
| Family peak MiB | 824.3 | 813.6 |
The completed task uses **50.9% fewer estimated request tokens**, including
schemas and continuations. Client elapsed and CPU time increased in this pair. At the client's
default output budget, complete status JSON is truncated and the task is
incomplete; summary status is intact and completes. That failed delivery is
excluded from completed-task gains. Three acceptance tests exercise invalid
JSON, incorrect counts/state/source linkage, missing chunks and wrong sessions.
The complete catalog plus all SDK exchanges also shrinks for the large case
(1,186,065 to 885,626 estimated tokens), but grows for the small case (882,323 to
885,676). These SDK exchange estimates include advertised output schemas and
are **not** actual model-input measurements. Summary delivery is optional
because its provenance wrapper costs more than a small complete observation.
## Resource accounting and limits
Runtime cases use one CPU, a 4 GiB cgroup ceiling, a 3 GiB early stop, no swap,
192 tasks, and host-memory admission/early-stop guards. Payload exit, descendant
cleanup and zero resource-limit/OOM events are verified while the scope observer
remains alive. Final whole-scope counters disappear after teardown and remain
unknown. Runtime family peaks stay below 1 GiB. Build/typecheck use separately
guarded larger limits; they are not runtime performance measurements.
Full queued messages still undergo validation, Evidence creation and retention;
this change primarily reduces delivery and repeated context. The extra derived
record increases retained bundle size. Long-lived heap retention, many-script
sessions, remote devices, binary messages and saturated capture budgets are not
performance-qualified by these native cases. Boundary tests preserve malformed
input, truncation, cancellation and cleanup semantics; simulated providers do
not qualify real remote behavior.
## Other providers in this audit
Inspector was measured against a test-owned real REA MCP process while it
handled a verified GitHub CLI Go build-information request. A concurrent native
CDP metadata subscription matched all 776 retained scripts by hash and reported
length: 565 file locations and 211 built-ins. The reply was 608,719 bytes;
capture retained 996/996 events with no reported drop/truncation. The qualification
run took 20.956 seconds, 18.076 CPU seconds and 987.4 MiB peak family memory,
including the independent oracle. This is not a paired performance cohort,
global completeness proof, source-byte validation or evidence of script
execution. Only `Runtime.enable` and `Debugger.enable` were sent. A prior
post-capture oracle missed 25 historical rows and was rejected; it does not
establish an Inspector regression. Existing raw-null delivery, grouped location
validation and eight-worker projection were already present and were preserved.
ADB's configured default executable is unavailable on this host, and no real
device is qualified. This audit makes no ADB device-performance claim. Inspector
selection/caching and Frida queue/filter redesign remain unimplemented without
an equivalent completed-task measurement that justifies them.
The existing bundled Frida verifier also returned functional success, but its
1,500 ms fixture outlived the verifier's 500 ms observation. Its payload-exit
cleanup receipt was therefore unverified and is excluded from qualified
resource/performance results. That unchanged verifier has no process birthtime
stamp to safely wait on an arbitrary later PID. The paired harness instead owns
the paused spawn and records birthtime/cgroup before resume, then waits for the
original target's natural exit. No PID-only cleanup or target-kill policy was
introduced.
The shared-service import also exposed an existing module-mock load-order
assumption in the Evidence-file cancellation test. That one mocked test now
runs in the existing isolated `process-global` lane, with an assertion that the
mocked read actually ran. Other file/service tests keep the shared module graph.
The final exact ten-file gate passes 225 tests with file parallelism disabled
under one CPU. An earlier parallel gate hit one existing 30-second MCP lifecycle
timeout; its serial retry passed without increasing a timeout or changing the
production lifecycle contract.
## Reproduction
Use the pinned Node/npm toolchain and native Frida dependency. Freeze separate
baseline/candidate built packages and install the same locked dependencies.
Run each case independently with the resource observer:
```sh
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
node experiments/analysis-efficiency/frida-status-queries.mjs \
--runtime /absolute/built-package --output /absolute/new-case \
--capture libc --detail summary
```
Repeat with `--detail complete` and `--capture small`. For actual-client captures,
`frida-status-wire.py --help` names the runtime, pinned Codex and Node executables,
output directory and selected script file (`selected-script.js` from the SDK
case). Use both `--mode complete`/`summary` and `--budget 300000`/`0`.
`frida-status-costs.py` checks task/resource receipts and counts complete requests.
The first wire attempt used ambiguous suffix matching for load/unload names;
the fixture now selects exact operation names. Failed attempts remain outside
the performance cohort. No production behavior changed to accommodate them.
@@ -1,299 +0,0 @@
"""Actual Codex wire capture with a deterministic local SSE backend; no AI inference."""
import argparse
import http.server
import threading
import asyncio
import hashlib
import json
import os
from pathlib import Path
import re
import time
import traceback
import subprocess
from frida_status_delivery import verify_frida_delivery
async def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--codex", type=Path, required=True)
parser.add_argument("--node", type=Path, required=True)
parser.add_argument("--input-profile", choices=["full", "compact"], default="full")
parser.add_argument("--output-profile", choices=["full", "off"], default="full")
parser.add_argument("--script", type=Path, required=True)
parser.add_argument("--expected-tools", type=int, default=211)
parser.add_argument("--mode", choices=["complete", "summary"], required=True)
parser.add_argument("--budget", type=int, default=30000)
args = parser.parse_args()
source=args.script.read_text()
assert args.runtime.is_absolute() and args.output.is_absolute() and args.script.is_absolute()
def delivered(pieces):
assert isinstance(pieces,str) and pieces.startswith("Wall time: "), pieces[:300]
_,sep,payload=pieces.partition("Output:\n");assert sep
try: value=json.loads(payload)
except json.JSONDecodeError: return {"_invalid_json_payload":payload}
return value["result"] if "result" in value and "normalized_result" not in value else value
def outputs_of(body):
return {item["call_id"]:delivered(item["output"]) for item in body.get("input",[]) if item.get("type")=="function_call_output"}
steps=7
args.output.mkdir(mode=0o700)
audit_home = args.output / "account"
codex_state = audit_home / ".codex"
mcp_home = audit_home / "mcp"
workspace = audit_home / "workspace"
for directory in [codex_state, mcp_home, workspace]:
directory.mkdir(parents=True)
captured = []
known_outputs = {}
state = {"normal_requests":0,"compactions":0}
class Capture(http.server.BaseHTTPRequestHandler):
def log_message(self, *_):
pass
def do_POST(self):
size = int(self.headers["Content-Length"])
assert 0 < size <= 16 * 1024 * 1024
assert self.headers.get("Content-Encoding", "identity") == "identity"
body = json.loads(self.rfile.read(size))
captured.append({"path": self.path, "body": body})
(args.output / "model-requests.json").write_text(json.dumps(captured) + "\n")
http_index = len(captured)
known_outputs.update(outputs_of(body))
names = [t.get("name", "") for ns in body.get("tools", []) if ns.get("type") == "namespace" and ns.get("name") == "mcp__rea_audit" for t in ns.get("tools", [])]
if not names:
last=body.get('input',[])[-1]
assert 'CONTEXT CHECKPOINT COMPACTION' in json.dumps(last)
state['compactions']+=1;assert state['compactions']<=1
checkpoint={'fixture':'deterministic observed-ID checkpoint, no AI inference','task':'Check Frida session state, queued message count and truncation twice; retain/export complete observations and close instrumentation.','session_id':known_outputs['call_local_1']['session_id'],'delivered_call_ids':list(known_outputs),'remaining':'Continue scripted tools; source message bodies remain in retained Evidence.'}
text=json.dumps(checkpoint)
item={"type":"message","id":f"msg_checkpoint_{state['compactions']}","role":"assistant","status":"completed","content":[{"type":"output_text","text":text,"annotations":[]}]}
captured[-1]['kind']='deterministic-compaction'
else:
state['normal_requests']+=1;index=state['normal_requests'];assert index<=steps+1
captured[-1]['kind']='tool-planning-or-final'
if index<=steps:
if index==1:
operation='start_frida_session';arguments={'mode':'spawn','device_id':'local','program':'/usr/bin/sleep','argv':['8']}
else:
session=known_outputs['call_local_1']['session_id']
if index==2:
operation='load_frida_script';arguments={'session_id':session,'source_kind':'inline','source':source}
elif index==3:
operation='resume_frida_session';arguments={'session_id':session}
elif index in [4,5]:
if index==4: time.sleep(0.75)
operation='frida_session_status';arguments={'session_id':session,**({'detail':'summary'} if args.mode=='summary' else {})}
elif index==6:
operation='close_frida_session';arguments={'session_id':session}
else:
operation='export_evidence_bundle';arguments={'path':str(args.output/'evidence.json')}
selected=[name for name in names if name == operation];assert len(selected)==1
item={"type":"function_call","id":f"fc_local_{index}","call_id":f"call_local_{index}","name":selected[0],"namespace":"mcp__rea_audit","arguments":json.dumps(arguments),"status":"completed"}
else:
item={"type":"message","id":"msg_local_done","role":"assistant","status":"completed","content":[{"type":"output_text","text":"Scripted local Frida state and capture-coverage task complete.","annotations":[]}]}
response_id=f"resp_local_{http_index}"
base = {"id": response_id, "object": "response", "model": body.get("model"), "status": "completed", "output": [item]}
events = [
{"type": "response.created", "response": {**base, "status": "in_progress", "output": []}},
{"type": "response.output_item.added", "output_index": 0, "item": {**item, "status": "in_progress"}},
{"type": "response.output_item.done", "output_index": 0, "item": item},
{"type": "response.completed", "response": base},
]
wire = "".join("event: " + e["type"] + "\ndata: " + json.dumps(e) + "\n\n" for e in events).encode()
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.send_header("Content-Length", str(len(wire)))
self.end_headers()
self.wfile.write(wire)
endpoint = http.server.HTTPServer(("127.0.0.1", 0), Capture)
endpoint_thread = threading.Thread(target=endpoint.serve_forever, daemon=True)
endpoint_thread.start()
task_path = f"{args.node.parent}:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
mcp_env = {"HOME": str(mcp_home), "PATH": task_path,
"NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1",
"REA_LOG_LEVEL": "silent", "REA_MCP_INPUT_SCHEMA_PROFILE": args.input_profile,
"REA_MCP_OUTPUT_SCHEMA_PROFILE": args.output_profile}
config = 'model_provider = "rea_capture"\nmodel = "rea-wire-probe"\n[analytics]\nenabled = false\n[mcp_servers.rea_audit]\n'
config += f'command = {json.dumps(str(args.node))}\n'
config += f'args = {json.dumps([str(args.runtime / "scripts/rea.mjs"), "mcp"])}\n'
config += 'startup_timeout_sec = 60\ntool_timeout_sec = 60\n'
config += '[mcp_servers.rea_audit.env]\n'
config += ''.join(f'{key} = {json.dumps(value)}\n' for key, value in mcp_env.items())
for name in ['start_frida_session','load_frida_script','resume_frida_session','frida_session_status','close_frida_session','export_evidence_bundle']:
config += f'[mcp_servers.rea_audit.tools.{name}]\napproval_mode = "approve"\n'
if args.budget: config += f'output_token_limit = {args.budget}\n'
config += f'\n[model_providers.rea_capture]\nname = "Local scripted capture"\nbase_url = "http://127.0.0.1:{endpoint.server_port}/v1"\nwire_api = "responses"\nrequires_openai_auth = false\nsupports_websockets = false\nrequest_max_retries = 0\nstream_max_retries = 0\n'
(codex_state / "config.toml").write_text(config)
(args.output / "declared-config.toml").write_text(config)
environment = {"HOME": str(audit_home), "CODEX_HOME": str(codex_state), "PATH": task_path,
"NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1"}
report = {"question": "Retain a caller-requested native libc export inventory, check session identity/state, queued message count and truncation twice, close instrumentation and export complete Evidence.",
"client": "Codex app-server 0.163.0", "model_turns_started": 1, "backend": "local deterministic SSE fixture; no AI model or paid service",
"actual_model_usage": None, "input_profile": args.input_profile,
"output_profile": args.output_profile, "calls": [], "observations": [],
"unexpected_server_requests": [], "cleanup_failures": []}
client_version = subprocess.check_output([str(args.codex), "--version"], env=environment, cwd=workspace, text=True, timeout=10).strip()
assert client_version == "codex-cli 0.163.0", client_version
transcript = []
pending = {}
completed_turn = asyncio.Event()
next_id = 0
stderr = (args.output / "server.log").open("wb")
process = await asyncio.create_subprocess_exec(str(args.codex), "app-server", "--stdio",
stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE, stderr=stderr,
cwd=workspace, env=environment, limit=16 * 1024 * 1024)
async def send(message):
encoded = (json.dumps(message, separators=(",", ":")) + "\n").encode()
transcript.append({"direction": "client", "bytes": len(encoded), "message": message})
process.stdin.write(encoded)
await process.stdin.drain()
async def read_messages():
try:
while line := await process.stdout.readline():
value = json.loads(line)
transcript.append({"direction": "server", "bytes": len(line), "message": value})
if value.get("method") == "turn/completed":
report["turn"] = value["params"]["turn"]
completed_turn.set()
if "id" in value and "method" not in value:
future = pending.pop(value["id"], None)
if future and not future.done():
future.set_result(value)
elif "id" in value:
report["unexpected_server_requests"].append(value["method"])
await send({"id": value["id"], "error": {"code": -32601, "message": "Unexpected request in scripted metadata probe"}})
raise RuntimeError("app-server stdout closed")
except Exception as error:
for future in pending.values():
if not future.done():
future.set_exception(error)
reader = asyncio.create_task(read_messages())
async def request(method, params):
nonlocal next_id
assert method in ["initialize", "thread/start", "mcpServerStatus/list", "mcpServer/tool/call", "turn/start"]
next_id += 1
message = {"id": next_id, "method": method, "params": params}
future = asyncio.get_running_loop().create_future()
pending[next_id] = future
start = time.monotonic()
await send(message)
reply = await asyncio.wait_for(future, 90)
report["calls"].append({"method": method, "elapsed_ms": (time.monotonic() - start) * 1000,
"error": reply.get("error")})
if "error" in reply:
raise RuntimeError(json.dumps(reply["error"])[:4000])
return reply["result"]
try:
await request("initialize", {"clientInfo": {"name": "rea_host_probe", "title": "REA host compatibility probe", "version": "1"},
"capabilities": {"experimentalApi": True, "explicitGatewayOauth": True}})
await send({"method": "initialized"})
created = await request("thread/start", {"cwd": str(workspace), "ephemeral": True,
"approvalPolicy": "never", "sandbox": "read-only"})
thread_id = created["thread"]["id"]
assert not created["thread"].get("turns", [])
inventory = await request("mcpServerStatus/list", {"threadId": thread_id,
"serverName": "rea_audit", "detail": "toolsAndAuthOnly", "limit": 10})
assert inventory.get("nextCursor") is None
assert len(inventory["data"]) == 1
server = inventory["data"][0]
assert server["name"] == "rea_audit" and server.get("toolsError") is None
tools = list(server["tools"].values())
names = [tool["name"] for tool in tools]
assert len(names) == len(set(names)) == args.expected_tools
assert "frida_session_status" in names
output_schemas = sum(tool.get("outputSchema") is not None for tool in tools)
assert output_schemas == (args.expected_tools if args.output_profile == "full" else 0)
raw = json.dumps(inventory, separators=(",", ":"), ensure_ascii=False).encode()
(args.output / "catalog.json").write_bytes(raw)
report["catalog"] = {"tools": len(tools), "output_schemas": output_schemas,
"bytes": len(raw), "sha256": hashlib.sha256(raw).hexdigest(),
"server_info": server.get("serverInfo"), "runtime_status": server.get("runtimeStatus")}
await request("turn/start", {"threadId":thread_id,"input":[{"type":"text","text":"Retain a complete libc export inventory using the selected caller-authored script. Check session state and queued message capture count/truncation twice, close Frida instrumentation and export complete Evidence. The local fixture scripts the tools; no AI inference."}]})
await asyncio.wait_for(completed_turn.wait(), 90)
assert report["turn"]["status"] == "completed", report["turn"]
assert state["normal_requests"]==steps+1
assert len(captured)==steps+1+state["compactions"]
outputs=known_outputs
assert len(outputs)==steps
bundle=json.loads((args.output/'evidence.json').read_text())
delivery_complete=True
answers=[]
for index in [4,5]:
delivered_result=outputs[f'call_local_{index}']
answer=verify_frida_delivery(delivered_result,bundle,args.mode,args.budget,outputs['call_local_1'])
if answer is None: delivery_complete=False
else: answers.append(answer)
loaded=next(e for e in bundle['records'] if e['operation']=='load_frida_script')
assert loaded['normalized_result']['source_sha256']==hashlib.sha256(source.encode()).hexdigest()
report['task_completed']=delivery_complete
report['observations']={'answers':answers,'exported':True,'exported_bundle_bytes':(args.output/'evidence.json').stat().st_size,'tool_output_budget':args.budget or 'client default','source_sha256':hashlib.sha256(source.encode()).hexdigest()}
report["captured_requests"]=len(captured)
report["compaction_requests"]=state["compactions"]
report["compaction_backend"]="Deterministic checkpoint of observed identifiers and retained-body location; no AI inference or quality claim."
report["mode"]=args.mode
report["request_tool_counts"] = [len(c["body"].get("tools", [])) for c in captured]
assert not report["unexpected_server_requests"]
report["status"] = "passed" if report["task_completed"] else "expected-incomplete-task"
except Exception:
report["status"] = "failed"
report["failure"] = traceback.format_exc()[-6000:]
finally:
process.stdin.close()
try:
await asyncio.wait_for(process.wait(), 15)
except asyncio.TimeoutError:
report["cleanup_failures"].append("app-server did not exit after stdin close; terminated owned process")
process.terminate()
try:
await asyncio.wait_for(process.wait(), 5)
except asyncio.TimeoutError:
process.kill()
await process.wait()
await reader
# app-server can acknowledge exit before its MCP child finishes EOF
# cleanup. Wait on the allocated resource scope; never target a PID.
try:
groups = [line[3:] for line in Path("/proc/self/cgroup").read_text().splitlines()
if line.startswith("0::")]
assert len(groups) == 1
group = Path("/sys/fs/cgroup") / groups[0].lstrip("/")
assert re.fullmatch(r"rea-efficiency-[0-9a-f]{32}\.scope", group.name)
observer = os.getppid()
observer_args = Path(f"/proc/{observer}/cmdline").read_bytes().split(b"\0")
assert any(Path(os.fsdecode(value)).name == "resource_payload.py" for value in observer_args if value)
expected = {os.getpid(), observer}
until = time.monotonic() + 10
while True:
actors = {int(value) for value in (group / "cgroup.procs").read_text().splitlines()}
assert expected <= actors
remaining = sorted(actors - expected)
if not remaining or time.monotonic() >= until:
break
await asyncio.sleep(0.05)
report["owned_scope_drain"] = {"remaining_processes": remaining}
if remaining:
report["cleanup_failures"].append("owned MCP child did not settle after app-server exit")
except (OSError, ValueError, AssertionError) as error:
report["owned_scope_drain"] = {"status": "unavailable", "reason": type(error).__name__}
report["cleanup_failures"].append("allocated resource-scope cleanup could not be verified")
endpoint.shutdown()
endpoint.server_close()
endpoint_thread.join(5)
stderr.close()
report["appserver_exit_code"] = process.returncode
if process.returncode != 0:
report["cleanup_failures"].append(f"app-server exit code: {process.returncode}")
(args.output / "transcript.json").write_text(json.dumps(transcript) + "\n")
(args.output / "report.json").write_text(json.dumps(report, indent=2) + "\n")
print(json.dumps({key: report.get(key) for key in ["status", "failure", "catalog", "observations", "cleanup_failures", "appserver_exit_code"]}))
return 0 if report["status"] in ["passed","expected-incomplete-task"] and not report["cleanup_failures"] else 1
raise SystemExit(asyncio.run(main()))
@@ -1,719 +0,0 @@
{
"issue": 5337,
"initial_base_revision": "4e716b06078d725a53234503cbefbb340731e809",
"integration_base_revision": "3fca2e0ca6e54ba700c286d9a85a9600dee6a19d",
"environment": {
"platform": "Linux x86_64",
"node": "24.18.0",
"npm": "11.16.0",
"mcp_sdk": "2.3.1",
"ghidra": "12.1.4",
"java": "Eclipse Adoptium 21.0.12.1+1-LTS",
"configured_java_heap": "768M",
"configured_active_processors": 1,
"query_cpu_quota_percent": 100,
"query_cpu_affinity": [15],
"query_hard_memory_bytes": 4294967296,
"query_early_stop_bytes": 3221225472,
"query_task_limit": 192,
"swap_bytes": 0,
"os_caches": "warm and unflushed; fresh servers, homes and Java analysis",
"initial_startup_and_rpc_deadline_ms": 300000,
"integrated_startup_and_rpc_deadline_ms": 600000,
"product_default_deadlines_changed": false
},
"method": {
"task": "Full observed procedure inventory, overview/load image and first internal non-thunk function facts; repeat inventory/pseudocode, export Evidence and close.",
"calls": 15,
"catalog_discoveries": 1,
"fact_comparison_excludes_only": ["evidence_id", "raw_result"],
"bytes": "Node JSON.stringify UTF-8 of complete SDK catalog, tool name/argument objects and reply envelopes; initialization, wire framing and model rendering unmeasured.",
"actual_model_usage": null,
"tokenizer_estimates": null,
"memory": "Kernel cgroup family peak including file cache at payload exit; post-GC retained heap unmeasured.",
"counter_scope": "Verified payload exit while the observer remains alive; final whole-scope counters after removal unavailable.",
"latency_claim": "No whole-task latency, CPU or peak-memory improvement established.",
"candidate_control_overhead": "Baseline fact-file parsing/comparison is included on the candidate side; its isolated cost is unmeasured."
},
"runtime_tables": {
"baseline": {
"files": 1841,
"sha256_sorted_path_tab_digest_lines": "549bb479ed41e9c55505c169e726552b632f0e5341e63194380b8d0ded344658",
"revision": "4e716b06078d725a53234503cbefbb340731e809"
},
"candidate": {
"files": 1841,
"sha256_sorted_path_tab_digest_lines": "0ebceab15311cc5ccdcfb92a0891022fc37736f2e7d01e2c1d22e88920df1ca8",
"base_revision": "4e716b06078d725a53234503cbefbb340731e809",
"source_diff_sha256": "7c460367e77daf2fad507800f1346dcd9e74553999fb049232b33b27684c8f49"
},
"integrated-baseline": {
"files": 1847,
"sha256_sorted_path_tab_digest_lines": "0d1672378dd3989663f7ef9ead3a91f56283dd06868f4574cae4d062dc83b041",
"base_revision": "3fca2e0ca6e54ba700c286d9a85a9600dee6a19d"
},
"integrated-candidate": {
"files": 1847,
"sha256_sorted_path_tab_digest_lines": "d4348d25b4b18e371ac63375accd8f14e72197588934f66b8ab2bb7049ddb7d3",
"base_revision": "3fca2e0ca6e54ba700c286d9a85a9600dee6a19d",
"source_diff_sha256": "7c460367e77daf2fad507800f1346dcd9e74553999fb049232b33b27684c8f49",
"driver_sha256": "b0463f8da110dcbae45769045a690dc5accf74791fa53f5086a60e8383f074de"
}
},
"driver_versions": {
"earlier": "b5bf4b378102de16f871c5f13e7302cbdcecbe0a14cc90b30ab80031920aaa06",
"integrated": "b0463f8da110dcbae45769045a690dc5accf74791fa53f5086a60e8383f074de"
},
"completed_tasks": [
{
"name": "baseline-true-4",
"expected_projection": "duplicate",
"artifact_sha256": "c79bf44242829108e323378531f4ac839513ca1fba45efd6583643526e1e9fd2",
"artifact_bytes": 35664,
"procedure_count": 231,
"selected_procedure": "0x102000",
"facts_sha256": "5b826454c00709fb26777e3b6663aa69ce879f51597c54d6ec3021fc1984eb74",
"call_count": 15,
"catalog": {
"elapsed_ms": 1132.755321,
"bytes": 3255374,
"sha256": "8a646b95cc5607ee57e5a3c0ef480b77e4cec7e5168498e72f511a8af40daf61"
},
"rpc_argument_bytes": 908,
"rpc_reply_bytes": 473344,
"catalog_arguments_replies_bytes": 3729626,
"normalized_value_bytes_once_per_selected_reply": 94357,
"raw_value_bytes_once_per_selected_reply": 90839,
"bundle_bytes": 177922,
"phases": {
"open_binary": {
"elapsed_ms": 913.6441110000001,
"reply_bytes": 499
},
"list_documents": {
"elapsed_ms": 27770.679519,
"reply_bytes": 6029
},
"list_procedures": {
"elapsed_ms": 156.53601400000116,
"reply_bytes": 125181
},
"binary_overview": {
"elapsed_ms": 101.32018699999753,
"reply_bytes": 17387
},
"inspect_native_load_image": {
"elapsed_ms": 265.31070799999725,
"reply_bytes": 101123
},
"procedure_info": {
"elapsed_ms": 80.39036699999997,
"reply_bytes": 7643
},
"procedure_pseudo_code": {
"elapsed_ms": 81.57453700000042,
"reply_bytes": 6549
},
"procedure_assembly": {
"elapsed_ms": 53.14257799999905,
"reply_bytes": 6715
},
"procedure_callers": {
"elapsed_ms": 22.954539999998815,
"reply_bytes": 6709
},
"procedure_callees": {
"elapsed_ms": 8.837930000001506,
"reply_bytes": 6753
},
"analyze_function": {
"elapsed_ms": 294.8276279999991,
"reply_bytes": 56545
},
"warm-inventory": {
"elapsed_ms": 49.338327999998,
"reply_bytes": 125181
},
"warm-pseudocode": {
"elapsed_ms": 16.670878999997512,
"reply_bytes": 6549
},
"export_evidence_bundle": {
"elapsed_ms": 51.058947999998054,
"reply_bytes": 312
},
"close_binary": {
"elapsed_ms": 494.6449689999936,
"reply_bytes": 169
}
},
"scope_seconds": 34.369,
"payload_cpu_seconds": 33.150037,
"payload_family_peak_bytes": 993923072,
"payload_cleanup_verified": true,
"zero_limit_events_at_payload_exit": true,
"final_whole_scope_counters_verified": false
},
{
"name": "candidate-true-1",
"expected_projection": "compact",
"artifact_sha256": "c79bf44242829108e323378531f4ac839513ca1fba45efd6583643526e1e9fd2",
"artifact_bytes": 35664,
"procedure_count": 231,
"selected_procedure": "0x102000",
"facts_sha256": "5b826454c00709fb26777e3b6663aa69ce879f51597c54d6ec3021fc1984eb74",
"call_count": 15,
"catalog": {
"elapsed_ms": 1208.093488,
"bytes": 3255374,
"sha256": "8a646b95cc5607ee57e5a3c0ef480b77e4cec7e5168498e72f511a8af40daf61"
},
"rpc_argument_bytes": 909,
"rpc_reply_bytes": 326642,
"catalog_arguments_replies_bytes": 3582925,
"normalized_value_bytes_once_per_selected_reply": 94357,
"raw_value_bytes_once_per_selected_reply": 22114,
"bundle_bytes": 137135,
"phases": {
"open_binary": {
"elapsed_ms": 941.0834749999995,
"reply_bytes": 499
},
"list_documents": {
"elapsed_ms": 27897.447016,
"reply_bytes": 5981
},
"list_procedures": {
"elapsed_ms": 136.00869700000112,
"reply_bytes": 65589
},
"binary_overview": {
"elapsed_ms": 124.65224699999817,
"reply_bytes": 17387
},
"inspect_native_load_image": {
"elapsed_ms": 249.8859930000035,
"reply_bytes": 101123
},
"procedure_info": {
"elapsed_ms": 95.36772800000472,
"reply_bytes": 6671
},
"procedure_pseudo_code": {
"elapsed_ms": 84.8552570000029,
"reply_bytes": 6355
},
"procedure_assembly": {
"elapsed_ms": 97.34392800000205,
"reply_bytes": 6305
},
"procedure_callers": {
"elapsed_ms": 8.018889999999374,
"reply_bytes": 6177
},
"procedure_callees": {
"elapsed_ms": 6.999988999996276,
"reply_bytes": 6199
},
"analyze_function": {
"elapsed_ms": 271.9528430000064,
"reply_bytes": 31929
},
"warm-inventory": {
"elapsed_ms": 30.13376900000003,
"reply_bytes": 65589
},
"warm-pseudocode": {
"elapsed_ms": 7.71154899999965,
"reply_bytes": 6355
},
"export_evidence_bundle": {
"elapsed_ms": 71.93626799999765,
"reply_bytes": 314
},
"close_binary": {
"elapsed_ms": 620.6287440000015,
"reply_bytes": 169
}
},
"scope_seconds": 34.901,
"payload_cpu_seconds": 33.834228,
"payload_family_peak_bytes": 1025228800,
"payload_cleanup_verified": true,
"zero_limit_events_at_payload_exit": true,
"final_whole_scope_counters_verified": false
},
{
"name": "baseline-jq-1",
"expected_projection": "duplicate",
"artifact_sha256": "b1c22172dd303f3be49e935aa56aa48a8b7a46e0bc838b4997d3bb451495870f",
"artifact_bytes": 2267912,
"procedure_count": 2276,
"selected_procedure": "0x401000",
"facts_sha256": "f7b95167c93f2b9129b5a4332590ef6bbb9d7b646d554ba4c396db1055ef0d24",
"call_count": 15,
"catalog": {
"elapsed_ms": 2761.8931860000002,
"bytes": 3255374,
"sha256": "8a646b95cc5607ee57e5a3c0ef480b77e4cec7e5168498e72f511a8af40daf61"
},
"rpc_argument_bytes": 965,
"rpc_reply_bytes": 2372592,
"catalog_arguments_replies_bytes": 5628931,
"normalized_value_bytes_once_per_selected_reply": 536023,
"raw_value_bytes_once_per_selected_reply": 533635,
"bundle_bytes": 609212,
"phases": {
"open_binary": {
"elapsed_ms": 1856.9621389999993,
"reply_bytes": 617
},
"list_documents": {
"elapsed_ms": 290349.418768,
"reply_bytes": 6167
},
"list_procedures": {
"elapsed_ms": 624.2473109999555,
"reply_bytes": 1092959
},
"binary_overview": {
"elapsed_ms": 214.38793299999088,
"reply_bytes": 15727
},
"inspect_native_load_image": {
"elapsed_ms": 432.1906869999948,
"reply_bytes": 75633
},
"procedure_info": {
"elapsed_ms": 106.62601700000232,
"reply_bytes": 7813
},
"procedure_pseudo_code": {
"elapsed_ms": 165.65909500000998,
"reply_bytes": 6869
},
"procedure_assembly": {
"elapsed_ms": 98.534897000005,
"reply_bytes": 6855
},
"procedure_callers": {
"elapsed_ms": 23.43981900002109,
"reply_bytes": 6891
},
"procedure_callees": {
"elapsed_ms": 9.403080000018235,
"reply_bytes": 7479
},
"analyze_function": {
"elapsed_ms": 327.3197410000139,
"reply_bytes": 45277
},
"warm-inventory": {
"elapsed_ms": 396.67419799999334,
"reply_bytes": 1092959
},
"warm-pseudocode": {
"elapsed_ms": 6.517379999975674,
"reply_bytes": 6869
},
"export_evidence_bundle": {
"elapsed_ms": 56.13109899999108,
"reply_bytes": 308
},
"close_binary": {
"elapsed_ms": 550.6349530000007,
"reply_bytes": 169
}
},
"scope_seconds": 304.543,
"payload_cpu_seconds": 294.743485,
"payload_family_peak_bytes": 1159921664,
"payload_cleanup_verified": true,
"zero_limit_events_at_payload_exit": true,
"final_whole_scope_counters_verified": false
},
{
"name": "integrated-baseline-jq",
"expected_projection": "duplicate",
"artifact_sha256": "b1c22172dd303f3be49e935aa56aa48a8b7a46e0bc838b4997d3bb451495870f",
"artifact_bytes": 2267912,
"procedure_count": 2279,
"selected_procedure": "0x401000",
"facts_sha256": "87722aab25011f1b047eb2bf8302cdbc5a4256c1b7f17d7bd68404a7d191e070",
"call_count": 15,
"catalog": {
"elapsed_ms": 1411.786849,
"bytes": 3263492,
"sha256": "c754f774156866d70be754e050335d0e4d3a7a6cbb156746cf1f7b5ffa42b26b"
},
"rpc_argument_bytes": 974,
"rpc_reply_bytes": 2375466,
"catalog_arguments_replies_bytes": 5639932,
"normalized_value_bytes_once_per_selected_reply": 536689,
"raw_value_bytes_once_per_selected_reply": 534301,
"bundle_bytes": 609878,
"phases": {
"open_binary": {
"elapsed_ms": 983.5489280000002,
"reply_bytes": 617
},
"list_documents": {
"elapsed_ms": 270159.690556,
"reply_bytes": 6167
},
"list_procedures": {
"elapsed_ms": 717.9032449999941,
"reply_bytes": 1094387
},
"binary_overview": {
"elapsed_ms": 203.3595150000183,
"reply_bytes": 15727
},
"inspect_native_load_image": {
"elapsed_ms": 362.37314199999673,
"reply_bytes": 75633
},
"procedure_info": {
"elapsed_ms": 92.72515800001565,
"reply_bytes": 7813
},
"procedure_pseudo_code": {
"elapsed_ms": 85.43632900004741,
"reply_bytes": 6869
},
"procedure_assembly": {
"elapsed_ms": 78.87645799998427,
"reply_bytes": 6855
},
"procedure_callers": {
"elapsed_ms": 32.66119900002377,
"reply_bytes": 6891
},
"procedure_callees": {
"elapsed_ms": 8.895469999988563,
"reply_bytes": 7479
},
"analyze_function": {
"elapsed_ms": 343.2701520000119,
"reply_bytes": 45277
},
"warm-inventory": {
"elapsed_ms": 338.9290720000281,
"reply_bytes": 1094387
},
"warm-pseudocode": {
"elapsed_ms": 9.391289999999572,
"reply_bytes": 6869
},
"export_evidence_bundle": {
"elapsed_ms": 82.86341799999354,
"reply_bytes": 326
},
"close_binary": {
"elapsed_ms": 679.7842850000015,
"reply_bytes": 169
}
},
"scope_seconds": 279.699,
"payload_cpu_seconds": 277.696079,
"payload_family_peak_bytes": 1111494656,
"payload_cleanup_verified": true,
"zero_limit_events_at_payload_exit": true,
"final_whole_scope_counters_verified": false
},
{
"name": "integrated-candidate-jq",
"expected_projection": "compact",
"artifact_sha256": "b1c22172dd303f3be49e935aa56aa48a8b7a46e0bc838b4997d3bb451495870f",
"artifact_bytes": 2267912,
"procedure_count": 2279,
"selected_procedure": "0x401000",
"facts_sha256": "87722aab25011f1b047eb2bf8302cdbc5a4256c1b7f17d7bd68404a7d191e070",
"call_count": 15,
"catalog": {
"elapsed_ms": 1213.2912340000003,
"bytes": 3263492,
"sha256": "c754f774156866d70be754e050335d0e4d3a7a6cbb156746cf1f7b5ffa42b26b"
},
"rpc_argument_bytes": 975,
"rpc_reply_bytes": 1264884,
"catalog_arguments_replies_bytes": 4529351,
"normalized_value_bytes_once_per_selected_reply": 536689,
"raw_value_bytes_once_per_selected_reply": 16143,
"bundle_bytes": 345626,
"phases": {
"open_binary": {
"elapsed_ms": 927.8219500000005,
"reply_bytes": 617
},
"list_documents": {
"elapsed_ms": 391135.781249,
"reply_bytes": 6119
},
"list_procedures": {
"elapsed_ms": 646.0353969999705,
"reply_bytes": 550261
},
"binary_overview": {
"elapsed_ms": 198.11136500001885,
"reply_bytes": 15727
},
"inspect_native_load_image": {
"elapsed_ms": 456.49483100004727,
"reply_bytes": 75633
},
"procedure_info": {
"elapsed_ms": 99.12928700004704,
"reply_bytes": 6825
},
"procedure_pseudo_code": {
"elapsed_ms": 114.82446700002765,
"reply_bytes": 6584
},
"procedure_assembly": {
"elapsed_ms": 80.73368800000753,
"reply_bytes": 6444
},
"procedure_callers": {
"elapsed_ms": 10.588890000013635,
"reply_bytes": 6337
},
"procedure_callees": {
"elapsed_ms": 19.5354899999802,
"reply_bytes": 6631
},
"analyze_function": {
"elapsed_ms": 352.812751999998,
"reply_bytes": 26364
},
"warm-inventory": {
"elapsed_ms": 197.59054599999217,
"reply_bytes": 550261
},
"warm-pseudocode": {
"elapsed_ms": 8.122649999975692,
"reply_bytes": 6584
},
"export_evidence_bundle": {
"elapsed_ms": 61.80356899998151,
"reply_bytes": 328
},
"close_binary": {
"elapsed_ms": 614.0456879999838,
"reply_bytes": 169
}
},
"scope_seconds": 399.097,
"payload_cpu_seconds": 352.659548,
"payload_family_peak_bytes": 1088741376,
"payload_cleanup_verified": true,
"zero_limit_events_at_payload_exit": true,
"final_whole_scope_counters_verified": false
}
],
"incomplete_attempts": [
{
"name": "baseline-true-1",
"reason": "Harness selected subject.sha256 instead of subject.digest.sha256.",
"reported_status": "failed",
"payload_exit_code": 1,
"scope_seconds": 41.401,
"payload_cpu_seconds": 39.027824,
"payload_family_peak_bytes": 1132675072,
"observed_procedures": null,
"payload_cleanup_verified": true,
"resource_events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"is_completed_performance_sample": false
},
{
"name": "baseline-true-2",
"reason": "Harness assumed the composed overview provider was Ghidra rather than rea-workflow.",
"reported_status": "failed",
"payload_exit_code": 1,
"scope_seconds": 37.538,
"payload_cpu_seconds": 36.599406,
"payload_family_peak_bytes": 1001476096,
"observed_procedures": 231,
"payload_cleanup_verified": true,
"resource_events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"is_completed_performance_sample": false
},
{
"name": "baseline-true-3",
"reason": "Facts and cleanup completed, but the harness incorrectly read a status property from the boolean finalizer and exited 1.",
"reported_status": "passed",
"payload_exit_code": 1,
"scope_seconds": 35.796,
"payload_cpu_seconds": 34.736678,
"payload_family_peak_bytes": 1001455616,
"observed_procedures": 231,
"payload_cleanup_verified": true,
"resource_events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"is_completed_performance_sample": false
},
{
"name": "candidate-jq-1",
"reason": "SDK five-minute first-query deadline expired while awaiting list_documents; cleanup completed.",
"reported_status": "failed",
"payload_exit_code": 1,
"scope_seconds": 316.605,
"payload_cpu_seconds": 293.28205,
"payload_family_peak_bytes": 1065631744,
"observed_procedures": null,
"payload_cleanup_verified": true,
"resource_events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"is_completed_performance_sample": false
},
{
"name": "integrated-baseline-jq-control",
"reason": "Unmodified-main repeat differs by three observed procedure rows; strict comparison rejected it before export. Cause unestablished.",
"reported_status": "failed",
"payload_exit_code": 1,
"scope_seconds": 276.557,
"payload_cpu_seconds": 274.502165,
"payload_family_peak_bytes": 1200726016,
"observed_procedures": 2276,
"payload_cleanup_verified": true,
"resource_events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"is_completed_performance_sample": false
}
],
"capacity_control": {
"baseline_revision": "3fca2e0ca6e54ba700c286d9a85a9600dee6a19d",
"records": 30000,
"baseline": "regression fails with resource_constraint for list_procedures at the unchanged 10485760-byte stdio budget",
"candidate": "all 30000 records delivered exactly; raw_result null",
"real_provider": false,
"production_boundary": "GhidraProvider -> BinarySession -> public MCP"
},
"executed_validation": [
{
"check": "npm run check",
"base": "4e716b060",
"result": "6 tasks and architecture guards passed"
},
{
"check": "npm run check",
"base": "3fca2e0ca",
"result": "6 tasks and architecture guards passed; before adding the capacity regression"
},
{
"check": "Focused Ghidra inventory/function/provider and four MCP files",
"base": "4e716b060",
"result": "246 assertions passed across 7 files"
},
{
"check": "Capacity regression alone",
"base": "3fca2e0ca",
"result": "candidate passes; unchanged production baseline fails with resource_constraint; six other tests filtered out"
},
{
"check": "Existing real Ghidra large-result helper through a private stdio harness",
"result": "passed",
"successful_calls": 15,
"expected_rejections": 4,
"runtime_empty_after_provider_close": true,
"scope": "oversized lookup/read/annotation/dossier retention and recovery, complete CLI readback, unchanged source bytes"
},
{
"check": "node scripts/verify-real-ghidra-address-context.mjs",
"result": "passed with real Ghidra/CLI/MCP and verified empty descendant set",
"scope": "PIE load identity, file-offset decompilation/roles, address round trips, failure guidance/recovery, unchanged source and complete cleanup",
"fixture_sha256": "093f8ba3aef80f3d7c0e8f22557e9a8af29505e286835578282b8e32ff804a02"
},
{
"check": "Final focused Ghidra source and MCP regression selection",
"base": "3fca2e0ca",
"result": "247 assertions across 7 files passed"
},
{
"check": "npm run docs:check",
"result": "generated metadata/documentation checks and 5 README conformance assertions passed"
}
],
"unmodified_main_control": {
"same_target_digest_and_analysis_profile": true,
"expected_procedures": 2279,
"observed_procedures": 2276,
"missing_relative_to_first_run": ["0x4f9930", "0x5260b0", "0x529640"],
"cause": "unestablished; no engine/default-analysis repair included",
"interpretation": "The matched pair is an observation, not a guarantee of cross-session procedure discovery or a latency distribution."
},
"limitations": [
"No actual model-token or tokenizer measurement.",
"Single completed pairs, not latency distributions.",
"Unmodified-main procedure-inventory repeat failed the strict oracle; retained as an incomplete control.",
"First three successful query receipts used the earlier driver; archived warm results were rechecked when explicit repeat assertions were added.",
"Real Windows/macOS Ghidra, other engine versions and real NativeAOT overlays remain unqualified.",
"Initial cached baseline build had an unverified payload-exit cleanup observation; a later systemd inspection found its scope inactive. It is not a query measurement."
],
"resource_stopped_validation": [
{
"check": "Final focused regression selection, first attempt",
"abort": "host_available_memory_early_stop",
"peak_family_bytes_sampled": 837332992,
"outcome": "guard-stopped without final test results; not counted as passing",
"retry_budget": "4 GiB hard/3 GiB early, based on the earlier focused run peak of about 1.3 GiB"
}
]
}
@@ -1,165 +0,0 @@
# Ghidra Evidence projection
Continues [#5337](https://github.com/morluto/rea/issues/5337). The bridge
adapter supplied a complete `rawResult` for every successful operation, even
when strict parsing preserved the same value in `normalized_result`. Function
observation envelopes repeated their value and already-projected qualifications.
The generic Evidence contract already permits `raw_result: null` in this case.
## Owning change
The Ghidra inventory and function parsers now declare whether they retain an
independent upstream representation. Execution consumes those parsed facts and
named metadata, replacing the positional helper argument that required every
caller to supply another complete report. CLI and MCP use the same projection.
Ordinary inventories and function observations omit the redundant copy.
Limitations, relationship coverage and locations remain canonical Evidence
fields. The following representations remain available:
- The overview's upstream segment permissions, sections and provenance.
- Load-image observations before local attestation and enrichment.
- File-offset selection roles, which generic locations alone do not express.
- The database type reply before adding a locally derived NativeAOT overlay.
No tool, schema, query default, provider protocol, cache, target execution or
annotation authority is added. Existing source Evidence is not rewritten.
New records have the identities appropriate to their new representation;
artifact and procedure identities remain unchanged. Consumers should read
function results from `normalized_result`, rather than `raw_result.value`.
## Real inputs and completed question
Both ELF x86-64 targets are read without executing them:
| Target | Bytes | SHA-256 | Provenance |
| ---------------- | --------: | ------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `true` | 35,664 | `c79bf44242829108e323378531f4ac839513ca1fba45efd6583643526e1e9fd2` | Installed Debian coreutils 9.1-1; package metadata is a local observation. |
| `jq-linux-amd64` | 2,267,912 | `b1c22172dd303f3be49e935aa56aa48a8b7a46e0bc838b4997d3bb451495870f` | [Official jq 1.8.2 asset](https://github.com/jqlang/jq/releases/tag/jq-1.8.2); matches both the corpus and the release API's asset digest/size. |
Each fresh production stdio MCP session discovers the complete catalog, opens
the target, reads documents and the complete observed procedure inventory,
selects the first internal non-thunk procedure, reads overview/load image and
six function operations, repeats inventory/pseudocode, exports Evidence and
closes. This is one catalog discovery and 15 RPCs, including cleanup.
It qualifies this inventory/function-read question, rather than application
feature reconstruction or complete native-code discovery.
The live named SDK contracts validate replies. Each selected record binds to
the original target digest and Ghidra profile. The candidate's complete selected
fact vector equals the baseline after excluding only `evidence_id` and
`raw_result`: subject, parameters, normalized facts, provider/profile, authority,
limitations, relationships, locations and Evidence links remain equal. Exported
records match returned Evidence, repeated facts agree, and target hashes remain
unchanged. This is representation equivalence, not an independent qualification
of every Ghidra interpretation.
## Measurements and limits
Linux x86-64, Node 24.18.0/npm 11.16.0, MCP SDK 2.3.1, Ghidra 12.1.4 and
Eclipse Adoptium 21.0.12.1+1-LTS. Sessions and Java analysis are fresh; OS caches
are warm and unflushed. Configured Java heap is 768 MiB with one active processor.
The whole process family uses one logical CPU, 4 GiB hard/3 GiB early-stop memory,
no swap and 192 tasks. No completed query task has an OOM/limit event or cleanup
failure at verified payload exit. Final whole-scope counters after scope removal
remain unavailable. Post-GC retained heap is unmeasured.
The small pair uses main `4e716b06078d725a53234503cbefbb340731e809` and its
candidate. Main advanced during the experiment; the large pair uses
`3fca2e0ca6e54ba700c286d9a85a9600dee6a19d` on both sides. Existing timing
samples keep their original runtime fingerprints. The [curated receipt](ghidra-evidence-projection.json)
retains individual observations and incomplete attempts; raw data stays outside Git.
| Recording / metric | Before | After |
| ----------------------------------------------- | ---------: | ---------: |
| `true`: observed procedures | 231 | 231 |
| `true`: all RPC result bytes | 473,344 | 326,642 |
| `true`: catalog + request arguments + results | 3,729,626 | 3,582,925 |
| `true`: complete scope wall | 34.369 s | 34.901 s |
| `true`: payload-family CPU | 33.150 s | 33.834 s |
| `true`: kernel family peak including file cache | 947.9 MiB | 977.7 MiB |
| `jq`: observed procedures | 2,279 | 2,279 |
| `jq`: all RPC result bytes | 2,375,466 | 1,264,884 |
| `jq`: catalog + request arguments + results | 5,639,932 | 4,529,351 |
| `jq`: complete scope wall | 279.699 s | 399.097 s |
| `jq`: payload-family CPU | 277.696 s | 352.660 s |
| `jq`: kernel family peak including file cache | 1060.0 MiB | 1038.3 MiB |
RPC result bytes include the SDK's text and structured representations. Request
objects and the complete catalog are included in the combined byte count;
initialization, wire framing and model rendering are unmeasured. Actual model
tokens and tokenizer estimates are unmeasured. The observed byte reduction
does not establish a token saving or better agent investigation outcome.
The large result envelopes shrink by 46.8%; combined serialization shrinks by
19.7%, with unchanged catalog and call count. Small combined serialization
shrinks by 3.9%. These are single pairs, not distributions. No whole-task latency,
CPU or peak-memory improvement is established. The large candidate's first
`list_documents` costs 391.136 s versus 270.160 s before; this phase includes
startup, analysis wait and client work, not isolated engine time. Changed
projection code executes after the bridge returns its observation, but the
cause of the preparation-time variation is unestablished. One warm inventory
read costs 338.93 → 197.59 ms; this is not a general latency qualification.
A subsequent unchanged-main repeat with the same target/profile returned 2,276
procedures instead of 2,279, omitting `0x4f9930`, `0x5260b0` and `0x529640`
relative to its earlier inventory. The strict oracle rejected that control
before export. Its cause is unestablished; it is retained as an incomplete
attempt, not added to performance averages or treated as a projection defect.
## Verification scope
The production-boundary capacity regression supplies 30,000 valid simulated
Ghidra procedure rows through `GhidraProvider`, `BinarySession` and MCP. On
unchanged production main it fails with `resource_constraint` at the existing
10 MiB response budget. The candidate delivers every row exactly, without a
budget increase or truncation. This qualifies response admission, not a real
30,000-function Ghidra analysis or a universal maximum procedure count.
Source/parser tests cover strict rejection, sidecars, distinct file-offset
mapping roles, overview details, NativeAOT enrichment and unchanged observations.
Executed checks and real-provider verification are recorded in the receipt.
NativeAOT overlay tests use producer fixtures; real Windows/macOS and other
engine versions remain unqualified here.
The existing real large-result helper passed through a private stdio harness:
oversized lookup/read/annotation/dossier replies retained complete Evidence,
subsequent queries recovered, complete CLI readback matched the original bytes
with `raw_result: null`, and generated target bytes remained unchanged.
The public `verify-real-ghidra-address-context.mjs` also passed with real
Ghidra/CLI/MCP, including file-offset selection, address round trips, recovery,
unchanged source and verified cleanup. These capacity/address fixtures use a
separate 6 GiB family cap; they are correctness checks, not latency pairs.
Three early `true` harness attempts used an incorrect subject-digest selector,
assumed the composed overview's provider was Ghidra, or misread the existing
finalizer's boolean result. The last completed its facts but exited nonzero;
none is a completed performance sample. A first `jq` candidate hit the SDK's
five-minute deadline while awaiting readiness, then cleaned up successfully.
The final driver's startup/RPC deadlines are ten minutes on both sides; product
defaults are unchanged. Earlier successful warm facts were rechecked from their
archived production replies when explicit warm-result assertions were added.
## Reproduce
Build and retain separate baseline/candidate runtimes. Use the same absolute
target path and caller-supplied Ghidra/JDK for both:
```sh
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
node experiments/analysis-efficiency/ghidra-evidence-projection.mjs \
--runtime /absolute/baseline --target /absolute/application \
--ghidra /absolute/ghidra --java /absolute/jdk \
--expect duplicate --output /absolute/new-before
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
node experiments/analysis-efficiency/ghidra-evidence-projection.mjs \
--runtime /absolute/candidate --target /absolute/application \
--ghidra /absolute/ghidra --java /absolute/jdk \
--expect compact --control /absolute/new-before/facts.json \
--output /absolute/new-after
```
The candidate's independent baseline comparison is included in harness costs;
its isolated verification overhead is unmeasured. Static checks use a separate
8 GiB memory cap. All heavy operations run serially.
@@ -1,216 +0,0 @@
/** Real Ghidra inventory/function Evidence; raw inputs and receipts stay outside Git. */
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { finished } from "node:stream/promises";
import { parseArgs } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { createRetainedQueryCaller } from "./retained-query-calls.mjs";
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
const { values } = parseArgs({
options: Object.fromEntries(
["runtime", "target", "output", "ghidra", "java", "expect", "control"].map(
(name) => [name, { type: "string" }],
),
),
});
for (const name of ["runtime", "target", "output", "ghidra", "java"])
assert.ok(values[name] && values[name] === resolve(values[name]), name);
assert.ok(["duplicate", "compact"].includes(values.expect));
const hash = (value) => createHash("sha256").update(value).digest("hex");
const output = values.output;
await mkdir(output, { mode: 0o700 });
const bytes = await readFile(values.target);
const home = await mkdtemp(join(tmpdir(), "rea-ghidra-evidence-"));
const client = new Client({ name: "ghidra-evidence-projection", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
env: {
HOME: home,
PATH: process.env.PATH,
NODE_OPTIONS: "--max-old-space-size=2048",
GHIDRA_INSTALL_DIR: values.ghidra,
JAVA_HOME: values.java,
GHIDRA_HEADLESS_MAXMEM: "768M",
JAVA_TOOL_OPTIONS: "-XX:ActiveProcessorCount=1",
REA_GHIDRA_STARTUP_TIMEOUT_MS: "600000",
REA_LOG_LEVEL: "silent",
},
stderr: "pipe",
});
const serverLog = createWriteStream(join(output, "server.log"));
const serverLogFinished = finished(serverLog);
serverLogFinished.catch(() => {});
transport.stderr?.pipe(serverLog);
const report = {
artifact: { path: values.target, bytes: bytes.length, sha256: hash(bytes) },
expected_projection: values.expect,
calls: [],
};
const transcript = [];
const evidence = [];
const call = createRetainedQueryCaller(client, (receipt, event) => {
report.calls.push(receipt);
transcript.push(event);
});
const result = async (name, input, phase = name) => {
const reply = await call(name, input, phase);
assert.notEqual(reply.isError, true, JSON.stringify(reply));
assert.ok(reply.structuredContent);
return reply.structuredContent;
};
const inspect = async (name, input, phase = name) => {
const value = await result(name, input, phase);
assert.equal(value.subject.digest.sha256, report.artifact.sha256);
assert.equal(
value.provider.id,
name === "binary_overview" ? "rea-workflow" : "ghidra",
);
assert.equal(
value.provider.version,
name === "binary_overview" ? "1" : "12.1.4",
);
assert.ok(value.analysis_profile);
evidence.push({ phase, value });
return value;
};
const started = performance.now();
let opened = false;
try {
await client.connect(transport);
const discovery = performance.now();
const catalog = await client.listTools();
assert.equal(catalog.nextCursor, undefined);
report.catalog = {
elapsed_ms: performance.now() - discovery,
bytes: Buffer.byteLength(JSON.stringify(catalog)),
sha256: hash(JSON.stringify(catalog)),
};
await writeFile(join(output, "catalog.json"), JSON.stringify(catalog));
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
const original = client.callTool.bind(client);
client.callTool = (input, options) =>
original(input, {
...options,
timeout: 600000,
toolDefinition: definitions.get(input.name),
});
await result("open_binary", { path: values.target, provider_id: "ghidra" });
opened = true;
const documents = await inspect("list_documents", {});
assert.equal(documents.normalized_result.length, 1);
const procedures = await inspect("list_procedures", {});
assert.ok(procedures.normalized_result.length > 0);
const selected = procedures.normalized_result.find(
(row) => row.procedure.external === false && row.procedure.thunk === false,
);
assert.ok(selected);
report.procedure = selected.address;
report.procedure_count = procedures.normalized_result.length;
await inspect("binary_overview", {});
await inspect("inspect_native_load_image", {});
for (const name of [
"procedure_info",
"procedure_pseudo_code",
"procedure_assembly",
"procedure_callers",
"procedure_callees",
"analyze_function",
])
await inspect(name, { procedure: selected.address });
const warmInventory = await inspect("list_procedures", {}, "warm-inventory");
assert.deepEqual(
warmInventory.normalized_result,
procedures.normalized_result,
);
const warmPseudocode = await inspect(
"procedure_pseudo_code",
{ procedure: selected.address },
"warm-pseudocode",
);
assert.deepEqual(
warmPseudocode.normalized_result,
evidence.find(({ phase }) => phase === "procedure_pseudo_code").value
.normalized_result,
);
for (const { phase, value } of evidence) {
if (phase === "binary_overview") {
assert.equal(value.raw_result, null);
continue;
}
const distinct = phase === "inspect_native_load_image";
if (values.expect === "compact" && !distinct)
assert.equal(value.raw_result, null, phase);
else assert.notEqual(value.raw_result, null, phase);
if (values.expect === "duplicate" && !distinct) {
const raw = value.raw_result;
const wrapped =
raw !== null &&
!Array.isArray(raw) &&
Object.hasOwn(raw, "value") &&
Object.hasOwn(raw, "limitations");
assert.deepEqual(
wrapped ? raw.value : raw,
value.normalized_result,
phase,
);
}
}
const facts = evidence.map(({ phase, value }) => {
const { evidence_id: _id, raw_result: _raw, ...facts } = value;
return { phase, facts };
});
report.facts_sha256 = hash(JSON.stringify(facts));
await writeFile(join(output, "facts.json"), JSON.stringify(facts));
if (values.control)
assert.deepEqual(facts, JSON.parse(await readFile(values.control, "utf8")));
const path = join(output, "evidence.json");
await result("export_evidence_bundle", { path });
const bundle = JSON.parse(await readFile(path, "utf8"));
for (const { value } of evidence)
assert.deepEqual(
bundle.records.find((record) => record.evidence_id === value.evidence_id),
value,
);
assert.equal(hash(await readFile(values.target)), report.artifact.sha256);
report.status = "passed";
} catch (cause) {
report.status = "failed";
report.failure = String(cause?.stack ?? cause);
process.exitCode = 1;
} finally {
if (opened) {
try {
await result("close_binary", {});
} catch (cause) {
report.status = "failed";
report.close_failure = String(cause?.stack ?? cause);
}
}
report.elapsed_before_cleanup_ms = performance.now() - started;
try {
await writeFile(
join(output, "transcript.json"),
JSON.stringify(transcript),
);
} catch (cause) {
report.status = "failed";
report.transcript_failure = String(cause);
}
const finalized = await finalizeRetainedQueryReceipt({
report,
client,
transport,
account: home,
serverLog,
serverLogFinished,
output,
});
if (report.status !== "passed" || !finalized) process.exitCode = 1;
}
@@ -1,142 +0,0 @@
# Existing Ghidra project reuse: real workflow verification
The task counts the analyzed functions, reads one 32-record inventory window,
reports the overview, decompiles two known local non-thunk functions, repeats the
inventory query, exports complete Evidence and verifies it offline. Function
bodies and the complete inventory are compared by stable function identity.
The agent receives inventory summaries and the selected page; the complete
inventory remains accessible through retained Evidence and its exported bundle.
## Executed scope
Linux x64, Ghidra 12.1.4, JDK 21.0.12.1, Node 24.22.1. Measurements follow main
`e3066c170` with the feature core at `9746ec7d9`; receipt revisions distinguish
that driver from subsequent `5d237eb0c` / `314b0e038` refinements. Subsequent core
change only improves the setting named for an invalid derived program selection.
The final source suites and actual CLI/MCP cases use the updated dependency lock.
- `/usr/bin/ls`: 652 function records; complete facts SHA-256
`d2eb1ea6586b8d549663f98089290dbc020bcc9f9c867545f8ba132aa1a13195`.
- `/usr/lib/x86_64-linux-gnu/libcrypto.so.3`: 14,338 function records; artifact SHA-256
`aa01e614af634bc7c8b678c9250b5c4f613c61989bacf0a5d6aed6570e7dcabd`;
complete facts SHA-256 `033eccbb258317b7d05a4d4a48f049f5d2649b276852623cef3c1fb93e9a029a`.
Both pairs return identical full function records and the same nonempty pseudocode
for each selected address. Evidence IDs/profile identities differ intentionally:
existing-project observations commit the caller project content and unknown
original analysis history. Successful reuse verifies source project content
unchanged after closing; original binary digests remain unchanged too.
## Matched completed tasks
Each row includes SDK initialization, one discovery, every call, full bundle
export/authentication, connection cleanup and receipt publication. Both columns
in a target pair use compact output schemas and the same families:
`direct,enhanced,native,application,session`.
| Task | Functions | Wall s | Family CPU s | Peak family GiB | First / warm inventory s | Wire tokens, estimate |
| ---------------- | --------: | -----: | -----------: | --------------: | -----------------------: | --------------------: |
| current-ls-fresh | 652 | 63.62 | 84.87 | 1.276 | 55.83 / 0.416 | 85,151 |
| current-ls-reuse | 652 | 31.69 | 46.61 | 0.938 | 20.58 / 0.151 | 85,448 |
| crypto-fresh | 14,338 | 186.42 | 287.01 | 1.857 | 176.71 / 1.535 | 84,162 |
| crypto-reuse | 14,338 | 25.45 | 40.23 | 1.258 | 17.29 / 1.256 | 84,516 |
The first query starts a new provider process; the warm query reuses that session.
OS caches were not flushed. These are sequential samples on a shared host, not a
general speed guarantee. Earlier completed small-target calibrations vary
substantially (fresh 34.8–56.1 seconds; reuse 19.2 seconds), so elapsed time alone
should not be treated as a stable ratio.
CPU is summed process-family user/system time. Memory sums family RSS sampled
every 100 ms; shared pages can be double-counted and short peaks missed. Every
run used 2-CPU affinity, an 8-GiB family RSS guard and a wall deadline. Fresh and
reuse used the same heap request per target: 1,536 MiB for ls, 3,072 MiB for crypto.
No measured workflow triggered OOM, a resource guard or a cleanup failure.
Token estimates encode every captured JSON-RPC message, including initialize,
full discovery schemas, requests, result text/structured content and notifications,
with `cl100k_base`. They exclude framing newlines and are not actual model usage.
The model adapter, model tokenizer, hidden prompts and billing were not measured.
Project provenance adds a small response cost; this feature primarily avoids
repeated import/analysis rather than reducing tokens. Complete Evidence files
are verified locally and are not inserted into the agent transcript.
## Preparation cost and scope refinement
The explicit pre-import step saves a new project outside REA; its time is not
hidden in a warm result. With the same 2-CPU bound, ls preparation cost 36.17 s
wall, 59.54 CPU s and 0.845 GiB peak family RSS. Crypto preparation cost 231.76 s,
251.08 CPU s and 1.615 GiB peak. Both used default analysis followed by the same
REA no-return repair that a fresh REA import uses, enabling an equivalent-data
comparison. Ordinary caller projects may have different analysis histories.
Preparation plus the first investigation can cost more than one fresh import;
the benefit is repeated sessions or separately paid analysis that exceeds startup.
The task does not actually need the native family. A narrower executed ls pair
uses `direct,enhanced,application,session`, advertises 97 tools, preserves the same
complete facts and costs 67,523 / 67,820 estimated wire tokens for fresh / reuse,
versus 85,151 / 85,448 above. Its observed wall/CPU/peak values are 37.35 s / 62.63 s /
1.089 GiB fresh and 20.17 s / 32.21 s / 0.886 GiB reuse. These separate samples
confirm the smaller catalog works; they do not establish a general speedup from
scoping. Use the narrow scope when reproducing the task.
## Negative cases and verification limits
The actual production MCP path rejected `/usr/bin/true` against the ls project
with the recorded-import digest mismatch. A separate owned source clone changed
after open was rejected by the first query with its changed-input reason, then
cleaned up; the test restores its own intentional marker edit. Successful files,
source content and cleanup outcomes are verified separately. The actual compiled
CLI `decompile` selected the same project/profile and returned the same pseudocode
as MCP for `0x10a420`.
Final focused validation: 35 files, 958 passed / 4 platform skips; typecheck/lint,
build, 9 documentation cases and full formatting passed. No real macOS, Windows,
PE or Mach-O reuse claim is made. Windows and special fresh-import profiles remain
explicitly unsupported for this mode. Very large project trees and interrupted
cross-process database writes were not measured.
Initial driver calibrations caught external-first inventory ordering and
Evidence-ID ordering across different profiles. The driver now selects known
local functions from a bounded middle page and compares pseudocode by function
identity. Those failed calibration receipts remain in the maintainer workspace;
they are not counted as completed task successes.
## Reproduce
Build with the pinned toolchain and locked dependencies. Prepare a new owned
project using the documented [pre-import route](../../docs/installation.md#reuse-an-existing-ghidra-project).
The measurement used the same no-return repair during preparation for data parity.
Run the committed driver first without `--project`, then with it and `--control`:
```sh
node experiments/analysis-efficiency/ghidra-project-reuse.mjs \
--runtime /absolute/rea --target /usr/bin/ls --output /absolute/new-receipt \
--profile compact --families direct,enhanced,application,session \
--ghidra "$GHIDRA_INSTALL_DIR" --java "$JAVA_HOME" \
--project /absolute/projects/ls-analysis.gpr --program ls \
--control /absolute/fresh-receipt
```
Use `--heap 3072M` for the crypto case. `--expected mismatch` and `--expected stale`
exercise negative cases; stale intentionally edits/restores an owned test marker,
so use a disposable source copy. The driver retains full wire messages, catalog,
report, exported Evidence and normalized comparison facts in its output directory.
Resource receipts were collected with an external bounded process-family sampler;
its method and limits are described above. Actual model-token usage remains unmeasured.
## Review boundary regressions
After the measurement runs, review coverage adds resolved-executable defaults for
`.app` inputs (caller bundle paths remain provenance), early unavailable Windows
existing-project discovery, exact cancellation reasons from both checkpoints and
owned file reads, and selected-pair overlap checks that allow a project directly
under `/tmp`. The ambiguous Windows properties fixture cases were removed; the
test now covers its actual two distinct release-directory inputs.
A real Ghidra test copy saved seed and disabled-analyzer reports with REA's actual
pre-scripts. Reopening it through production MCP preserved both as one immutable
historical Evidence record, linked it from function observations, returned the
same function/pseudocode facts, and left source content unchanged. Fresh-import
checks still validate fresh requested work. No real macOS/Windows reuse claim is
implied by simulated platform boundary tests.
@@ -1,377 +0,0 @@
/** Equivalent bounded Ghidra investigations with fresh import or an existing project. */
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { finished } from "node:stream/promises";
import { pathToFileURL } from "node:url";
import { parseArgs, promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
const { values } = parseArgs({
options: Object.fromEntries(
[
"runtime",
"output",
"profile",
"target",
"receive-bytes",
"families",
"ghidra",
"java",
"project",
"program",
"control",
"expected",
"heap",
"history",
].map((key) => [key, { type: "string" }]),
),
});
for (const key of ["runtime", "output", "target", "ghidra", "java"]) {
assert.ok(values[key], `${key} is required`);
values[key] = resolve(values[key]);
}
assert.ok(["full", "compact"].includes(values.profile));
const receiveBytes = Number(values["receive-bytes"] ?? 10 * 1024 * 1024);
assert.ok(Number.isSafeInteger(receiveBytes) && receiveBytes > 0);
await mkdir(values.output);
const report = {
profile: values.profile,
families: values.families ?? null,
receive_bytes: receiveBytes,
runtime: values.runtime,
target: values.target,
project: values.project ?? null,
expected: values.expected ?? "success",
calls: [],
};
const transcript = [];
let account, client, transport, serverLog, serverLogFinished;
let phase = "initialize";
const began = performance.now();
try {
account = await mkdtemp(join(tmpdir(), "rea-output-profile-"));
report.target_sha256 = createHash("sha256")
.update(await readFile(values.target))
.digest("hex");
report.revision = (
await promisify(execFile)("git", [
"-C",
values.runtime,
"rev-parse",
"HEAD",
])
).stdout.trim();
transport = new StdioClientTransport({
command: process.execPath,
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
cwd: values.runtime,
maxBufferSize: receiveBytes,
env: {
PATH: process.env.PATH,
GHIDRA_INSTALL_DIR: values.ghidra,
JAVA_HOME: values.java,
GHIDRA_HEADLESS_MAXMEM: values.heap ?? "1536M",
JAVA_TOOL_OPTIONS: "-XX:ActiveProcessorCount=2",
REA_GHIDRA_STARTUP_TIMEOUT_MS: "600000",
...(values.project === undefined
? {}
: { REA_GHIDRA_PROJECT_PATH: values.project }),
...(values.program === undefined
? {}
: { REA_GHIDRA_PROGRAM_NAME: values.program }),
HOME: account,
USERPROFILE: account,
NODE_OPTIONS: process.env.NODE_OPTIONS ?? "--max-old-space-size=3072",
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
REA_MCP_OUTPUT_SCHEMA_PROFILE: values.profile,
...(values.families === undefined
? {}
: { REA_MCP_TOOL_FAMILIES: values.families }),
},
stderr: "pipe",
});
const start = transport.start.bind(transport);
transport.start = async () => {
const onerror = transport.onerror;
transport.onerror = (cause) => {
(report.transport_errors ??= []).push(String(cause));
onerror?.(cause);
};
const receive = transport.onmessage;
transport.onmessage = (message) => {
transcript.push({ direction: "response", phase, message });
receive?.(message);
};
await start();
};
const send = transport.send.bind(transport);
transport.send = async (message) => {
transcript.push({ direction: "request", phase, message });
await send(message);
};
serverLog = createWriteStream(join(values.output, "server.log"));
serverLogFinished = finished(serverLog);
serverLogFinished.catch(() => {});
transport.stderr.pipe(serverLog);
client = new Client({
name: "rea-output-profile-investigation",
version: "1",
});
await client.connect(transport);
phase = "discovery-cold";
const discoveryBegan = performance.now();
const catalog = await client.listTools({}, { timeout: 15000 });
report.discovery_ms = performance.now() - discoveryBegan;
report.catalog_bytes = Buffer.byteLength(JSON.stringify(catalog));
report.tools = catalog.tools.length;
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
const call = async (name, args, label) => {
phase = label;
const at = performance.now();
const result = await client.callTool(
{ name, arguments: args },
{ timeout: 900000, toolDefinition: definitions.get(name) },
);
assert.notEqual(
result.isError,
true,
`${name}: ${JSON.stringify(result).slice(0, 500)}`,
);
report.calls.push({ name, phase, elapsed_ms: performance.now() - at });
return result.structuredContent;
};
let sourceCommitment, projectResources, resolveProject;
if (values.project !== undefined) {
({ resolveGhidraProject: resolveProject } = await import(
pathToFileURL(
join(values.runtime, "dist/ghidra/GhidraProjectSnapshot.js"),
).href
));
const { ArtifactResourceScope } = await import(
pathToFileURL(
join(values.runtime, "dist/artifacts/ArtifactResourceScope.js"),
).href
);
projectResources = new ArtifactResourceScope();
try {
sourceCommitment = await resolveProject(
{
path: values.project,
...(values.program === undefined
? {}
: { programName: values.program }),
},
{ path: values.target },
projectResources,
);
} finally {
await projectResources.close();
}
report.project_sha256 = sourceCommitment.sha256;
}
await call(
"open_binary",
{ path: values.target, provider_id: "ghidra" },
"open",
);
if (["mismatch", "stale"].includes(values.expected)) {
phase = "mismatched-project";
const marker =
values.expected === "stale" ? await readFile(values.project) : undefined;
if (marker !== undefined)
await writeFile(
values.project,
Buffer.concat([marker, Buffer.from("\n")]),
);
let rejected;
try {
rejected = await client.callTool(
{ name: "list_procedures", arguments: { detail: "summary" } },
{ timeout: 900000, toolDefinition: definitions.get("list_procedures") },
);
} finally {
if (marker !== undefined) await writeFile(values.project, marker);
}
assert.equal(rejected.isError, true);
assert.match(
JSON.stringify(rejected),
values.expected === "stale"
? /changed while being read|artifact.changed/iu
: /imported-byte digest does not match/u,
);
await writeFile(
join(values.output, "rejection.json"),
JSON.stringify(rejected),
);
report[`rejected_${values.expected}`] = true;
} else {
const summary = await call(
"list_procedures",
{ detail: "summary" },
"inventory-cold",
);
const parentId = summary.normalized_result.parent_evidence_id;
const count = summary.normalized_result.summary.row_count;
assert.ok(count > 0);
report.procedure_count = count;
const page = await call(
"inspect_analysis_view",
{
source: { kind: "retained-evidence", evidence_id: parentId },
view: {
kind: "native-inventory",
offset: Math.floor(count / 2),
limit: Math.min(count - Math.floor(count / 2), 32),
fields: ["value", "procedure"],
},
},
"selected-procedures",
);
const selected = page.normalized_result.rows
.map((row) => row.record)
.filter(
(row) =>
row.procedure?.external === false && row.procedure?.thunk === false,
)
.slice(0, 2);
assert.equal(selected.length, 2);
await call("binary_overview", {}, "overview");
for (const row of selected)
await call(
"procedure_pseudo_code",
{ procedure: row.address },
`pseudocode:${row.address}`,
);
const warm = await call(
"list_procedures",
{ detail: "summary" },
"inventory-warm",
);
assert.equal(warm.normalized_result.parent_evidence_id, parentId);
const bundlePath = join(values.output, "complete-evidence.json");
await call(
"export_evidence_bundle",
{ path: bundlePath },
"complete-export",
);
await call("close_binary", {}, "close");
await client.close();
await transport.close();
const { parseEvidenceBundle } = await import(
pathToFileURL(join(values.runtime, "dist/domain/evidenceBundle.js")).href
);
const bundle = parseEvidenceBundle(
JSON.parse(await readFile(bundlePath, "utf8")),
);
const parent = bundle.records.find(
(record) => record.evidence_id === parentId,
);
assert.ok(parent);
if (values.history === "required") {
const historical = bundle.records.filter(
(record) =>
record.operation === "health" &&
record.parameters.facet === "historical-project-analysis-reports",
);
assert.equal(historical.length, 1);
assert.equal(historical[0].normalized_result.origin, "caller-project");
assert.ok(historical[0].normalized_result.analysis_seeds);
assert.ok(historical[0].normalized_result.disabled_analyzers);
report.historical_evidence_id = historical[0].evidence_id;
report.historical_evidence_linked =
parent.evidence_links?.includes(historical[0].evidence_id) ?? false;
}
assert.equal(parent.normalized_result.length, count);
const facts = {
procedures: parent.normalized_result,
pseudocode: bundle.records
.filter((record) => record.operation === "procedure_pseudo_code")
.map((record) => ({
procedure: record.parameters.procedure,
result: record.normalized_result,
})),
};
facts.pseudocode.sort((left, right) =>
left.procedure < right.procedure
? -1
: left.procedure > right.procedure
? 1
: 0,
);
report.facts_sha256 = createHash("sha256")
.update(JSON.stringify(facts))
.digest("hex");
await writeFile(join(values.output, "facts.json"), JSON.stringify(facts));
report.analysis_profile = parent.analysis_profile;
assert.equal(parent.subject.digest.sha256, report.target_sha256);
if (sourceCommitment !== undefined) {
assert.equal(
parent.analysis_profile.parameters.existing_project.sha256,
sourceCommitment.sha256,
);
assert.ok(
parent.limitations.some(
(value) =>
value.includes("completeness") && value.includes("unknown"),
),
);
}
if (values.control !== undefined) {
const control = JSON.parse(
await readFile(join(values.control, "facts.json"), "utf8"),
);
assert.deepEqual(facts, control);
}
}
if (sourceCommitment !== undefined) {
const { ArtifactResourceScope } = await import(
pathToFileURL(
join(values.runtime, "dist/artifacts/ArtifactResourceScope.js"),
).href
);
const resources = new ArtifactResourceScope();
let after;
try {
after = await resolveProject(
{
path: values.project,
...(values.program === undefined
? {}
: { programName: values.program }),
},
{ path: values.target },
resources,
);
} finally {
await resources.close();
}
assert.deepEqual(after, sourceCommitment);
report.original_project_unchanged = true;
}
report.outcome = "passed";
} catch (cause) {
report.outcome = "failed";
report.failure = String(cause);
process.exitCode = 1;
} finally {
report.elapsed_ms = performance.now() - began;
const published = await finalizeRetainedQueryReceipt({
report,
client,
transport,
account,
serverLog,
serverLogFinished,
output: values.output,
transcript,
});
if (!published) process.exitCode = 1;
}
@@ -1,74 +0,0 @@
"""Count verified Go tasks; tokenizer estimates are not model usage or billing."""
import argparse
import hashlib
import json
from pathlib import Path
import tiktoken
def compact(value):
return json.dumps(value, ensure_ascii=False, separators=(",", ":"), allow_nan=False)
def inspect(path, encoding):
report = json.loads((path / "report.json").read_text())
catalog = json.loads((path / "catalog.json").read_text())
transcript = json.loads((path / "transcript.json").read_text())
log = path.with_suffix(".log")
scope = next(json.loads(line) for line in reversed(log.read_text().splitlines())
if line.startswith('{"resource_scope":'))
observed = scope.get("payload_resource_observation") or {}
events = observed.get("resource_events", {})
memory = events.get("memory.events", {})
verified = (report["status"] == "observed"
and not report["cleanup_failures"] and not report["receipt_failures"]
and scope["abort"] is None
and scope.get("payload_resource_observation_verified") is True
and observed.get("payload_cleanup_verified") is True
and observed.get("payload_exit_code") == 0
and all(memory.get(key) == 0 for key in ["high", "max", "oom", "oom_kill"])
and events.get("pids.events", {}).get("max") == 0)
count = lambda value: len(encoding.encode(value, disallowed_special=()))
inputs = count(compact([{key: tool[key] for key in ["name", "description", "inputSchema"]}
for tool in catalog["tools"]]))
arguments = sum(count(compact({"name": event["name"], "arguments": event["input"]}))
for event in transcript)
replies = sum(count(block["text"]) for event in transcript
for block in event["reply"]["content"] if block["type"] == "text")
cpu = observed.get("cpu_stat", {}).get("usage_usec")
return {"case": path.name, "completed_with_verified_cleanup": verified,
"mode": report["mode"], "warm_diagnostic": report["warm_diagnostic"],
"answer": report["answer"], "parent_evidence_id": report["parent_evidence_id"],
"artifact_sha256": report["artifact_sha256"],
"complete_result_sha256": report["complete_result_file_sha256"],
"calls": report["calls"], "wall_seconds": scope["elapsed_seconds"],
"cpu_seconds": None if cpu is None else cpu / 1e6,
"kernel_family_peak_bytes": observed.get("memory_peak_bytes"),
"events_at_payload_exit": events,
"final_whole_scope_events": scope.get("resource_events_final"),
"token_proxy": {"input_catalog": inputs, "arguments": arguments,
"text_replies": replies, "combined": inputs + arguments + replies,
"complete_transport_catalog": count(compact(catalog))},
"sha256": {file.name: hashlib.sha256(file.read_bytes()).hexdigest()
for file in [path / "report.json", path / "catalog.json",
path / "transcript.json", log]}}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("cases", type=Path, nargs="+")
args = parser.parse_args()
encoding = tiktoken.get_encoding("cl100k_base")
records = [inspect(path, encoding) for path in args.cases]
receipt = {"tiktoken": tiktoken.__version__, "encoding": encoding.name,
"actual_model_usage": None,
"projection": "All discovered input schemas once, plus every call's arguments and text replies, including export. Full transport catalog counted separately; no model turn or host injection measured.",
"recordings": records}
args.output.write_text(json.dumps(receipt, ensure_ascii=False, indent=2) + "\n")
assert all(record["completed_with_verified_cleanup"] for record in records)
if __name__ == "__main__":
main()
@@ -1,81 +0,0 @@
"""Pin a completed Go report only after checking native build text and file bytes."""
import argparse
import base64
import hashlib
import json
from pathlib import Path
import subprocess
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--evidence", type=Path, required=True)
parser.add_argument("--input", type=Path, required=True)
parser.add_argument("--go", type=Path, required=True)
parser.add_argument("--dependency", required=True)
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
source = args.input.read_bytes()
evidence = json.loads(args.evidence.read_text())
if "records" in evidence:
evidence = next(record for record in evidence["records"]
if record["operation"] == "inspect_go_binary"
and record["parameters"] == {"path": str(args.input)})
report = evidence["normalized_result"]
info = report["build_info"]
digest = hashlib.sha256(source).hexdigest()
assert report["artifact"] == {"path": str(args.input), "sha256": digest,
"bytes": len(source)}
native = subprocess.check_output(
[str(args.go), "version", "-m", str(args.input)], text=True)
lines = native.splitlines()
assert info["go_version"] == lines[0].rsplit(" ", 1)[1]
text = "\n".join(line.removeprefix("\t") for line in lines[1:]) + "\n"
assert info["module_text"] == text
for name in ["version", "module"]:
location = info[name + "_location"]
key = ("go_version" if name == "version" else "module") + "_bytes_base64"
assert source[location["offset"]:location["offset"] + location["bytes"]] == base64.b64decode(info[key])
# Module identity columns are literal tab-separated fields. Preserve empty
# sums, development versions, original ordering and local replacements.
dependencies = []
main_module = None
previous = None
for line in text.splitlines():
columns = line.split("\t")
tag = columns[0]
if tag not in ["dep", "mod", "=>"]:
continue
assert len(columns) in [3, 4], columns
item = {"path": columns[1], "version": columns[2],
"sum": columns[3] if len(columns) == 4 else None}
if tag == "=>":
assert previous is not None
previous["replacement"] = item
else:
item["replacement"] = None
previous = item
if tag == "mod":
main_module = item
else:
dependencies.append(item)
assert info["module"]["main"] == main_module
assert info["module"]["dependencies"] == dependencies
answer = {"go_version": info["go_version"], "main": main_module,
"runtime_dependencies": "unknown",
"dependencies": [{"index": index, **item}
for index, item in enumerate(dependencies)
if item["path"] == args.dependency]}
complete = args.output.with_suffix(".complete-result.json")
complete.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n")
receipt = {"artifact_sha256": digest, "dependency_path": args.dependency,
"dependency_count": len(dependencies), "answer": answer,
"complete_result_path": str(complete.resolve()),
"native_text_sha256": hashlib.sha256(native.encode()).hexdigest(),
"native_go_path": str(args.go),
"native_go_sha256": hashlib.sha256(args.go.read_bytes()).hexdigest()}
args.output.write_text(json.dumps(receipt, ensure_ascii=False, indent=2) + "\n")
if __name__ == "__main__":
main()
File diff suppressed because it is too large. Load diff
@@ -1,195 +0,0 @@
# Selected Go build metadata
Addresses [#5499](https://github.com/morluto/rea/issues/5499). A compiler/main-module
identity question previously required the complete Go report, including raw framed
bytes, base64, module text and every parsed dependency. Opt-in `detail: summary`
retains the same complete parent and returns identity, source ranges, counts and
explicit unknowns. Default complete delivery remains compatible.
`inspect_analysis_view` reuses that parent for `kind: go`, collection
`dependencies` or `settings`. Exact paths/keys and original indices intersect;
fields select columns, while every row keeps its original index. Pagination
counts matching rows in producer order, including duplicate module paths. The
view never reopens a target or reparses module text. Unknown lines and original
bytes remain in the complete parent. CLI summaries save that parent in a new
Evidence bundle before returning their projection.
Only immutable authenticated Evidence results reuse schema validation, keyed
by their object identity for the Evidence lifetime. Mutable inline results are
validated on each call. Subject identity is checked separately on every view.
There is no live-file cache. Only page rows are projected; inherited messages
use the existing bounded, deduplicating view-message owner.
## Question and independent checks
The main artifact is the official GitHub CLI v2.102.0 Linux amd64 release:
42,086,560 bytes, SHA-256
`7469124f706944133d6a169691dd1c6c3511b12e85878d255e044e2948df4c9b`.
The archive digest matches its official GitHub asset metadata. Its embedded
compiler is `go1.27.1`, main module `github.com/cli/cli/[email protected]`, and it has
165 parsed dependency records. The selected `golang.org/x/[email protected]` record
keeps original index 152. These are embedded observations, not authentication
of the compiler, supply chain or runtime behavior.
Native `go version -m` independently matches module text, main identity and all
165 dependency identities in original order. Original file ranges match the
reported compiler and framed module bytes. Before/after exports have identical
complete normalized results and parent Evidence identity. The small real case
is the installed 12,366,352-byte Go tool: embedded `go1.23.5`, no decoded main
module and zero retained dependencies, with dependency coverage still unknown.
REA reads both inputs without executing them. The installed Go tool also runs
as the trusted native verifier; the downloaded GitHub CLI binary is never
executed.
Timing baseline is main `ee12d517a25d34853d40e31fc039c7d1576c60e1`. Packaged runtimes
and source patch are pinned in the [receipt](go-build-views.json); later changes
do not relabel these timings. Node 24.18.0/npm 11.16.0, MCP SDK 2.3.1, Linux
x86-64. Fresh processes use unflushed OS caches; the SDK controller pre-reads
the input for verification. These are not disk-cold measurements.
## Actual client context, including every request
The [wire driver](go-build-wire.py) starts actual Codex app-server 0.163.0 with a
fresh account/workspace and a loopback deterministic Responses SSE backend.
It starts a scripted turn and real MCP calls, with **no AI inference or paid
service**. All 210 tools remain discovered/advertised. Full input/output
profiles are explicit, and each relevant tool has a 30,000-token output budget
so complete JSON is delivered without retries or truncation. This does not
qualify another client's defaults or a real model's reasoning/schema acceptance.
All original facts and parent identities match the native oracle. Export cases
also verify the complete parent. The [counter](go-build-wire-costs.py) counts
**every captured request**, including repeated schemas and accumulated tool
outputs, using `tiktoken 0.12.0` / `cl100k_base` on compact request JSON. These are
serialization-token proxies; actual model rendering, usage and cache billing
remain unmeasured.
| Completed question | Requests before → after | Whole request JSON proxy before → after |
| ------------------------------------------------------ | ----------------------: | --------------------------------------: |
| Compiler, main module and dependency count | 2 → 2 | 158,529 → 123,368 (22.2% less) |
| Compiler, main module and one selected dependency | 2 → 3 | 158,538 → 186,812 (17.8% more) |
| Same selected dependency question plus Evidence export | 3 → 4 | 256,342 → 250,352 (2.3% less) |
The actual REA namespace costs 51,821 → 51,983 proxy tokens **per request**.
An extra round can outweigh a smaller reply. Summary is useful when identity
and counts already answer the question; complete delivery remains useful for
an initial compound question. Retained views provide selected follow-up facts,
without a claim that every such composition saves total context.
One recording per cell: identity wall/CPU/family peak is
6.970 s / 6.331 s / 861.5 MiB before and 7.069 s / 6.457 s / 857.0 MiB after.
Compound answer-only is 6.973 / 6.333 / 865.0 before and 6.969 / 6.348 / 861.6
after. Export is 7.175 / 6.537 / 866.7 before and 7.274 / 6.568 / 872.6 after.
These scripted-client observations establish no universal latency or memory
gain, and do not measure model inference time.
## Direct SDK task and cold/warm costs
The [SDK driver](go-build-views.mjs) discovers all tools, answers the compound
question, verifies exact facts and exports complete Evidence. Before: one
complete inspection and export. After: summary, selected dependency and export.
For the zero-dependency small case, the summary avoids an unnecessary empty
query. Two repetitions alternate version/target order. All 208 other named
catalog definitions agree; only Go inspection and the retained-view tool change.
| Input | Whole wall s before → after | Family CPU s before → after | Kernel family peak MiB before → after |
| ---------- | --------------------------- | --------------------------- | ------------------------------------- |
| GitHub CLI | 4.855–4.952 → 6.370–6.564 | 4.682–4.725 → 6.175–6.400 | 671.2–680.1 → 846.4–868.7 |
| Go tool | 4.349–4.351 → 4.250–4.450 | 4.093–4.140 → 4.092–4.179 | 548.4–548.6 → 544.3–555.6 |
Cold selected-query calls take 1.719–1.751 s in the SDK task, including its
output-schema validation. A separate warm diagnostic takes 1.712 s for the first
query and 32.86 ms for the repeat in one session; both the client validator and
retained facts are warm. This is not a paired whole-task speedup. The complete
warm diagnostic is separately recorded, not mixed into the eight matched runs.
The [SDK counter](go-build-view-costs.py) counts the discovered input catalog
once plus all arguments and text replies, including export: 229,183 → 195,643
for GitHub CLI and 194,854 → 194,724 for the small case. Catalog cost grows
193,192 → 193,652. This catalog-once projection is **not** the actual host
exchange above; omitting repeated schemas would hide the compound-task penalty.
All qualified recordings use one CPU, a 4 GiB hard/3 GiB early family memory
boundary, no swap and 192 tasks. Verified payload exits are zero, only the
observer remains, and observed memory/task resource events are zero. Kernel
family peaks include attributed file cache. Final whole-scope counters are
unavailable. Release acquisition, oracle preparation and compilation are outside
the timed tasks. An initial symlink input was correctly refused; qualification
explicitly selects its regular target. One development static check was stopped
by the host-available-memory guard, with owned cleanup verified and no observed
OOM; the unchanged guard succeeded after headroom recovered.
## Coverage and reproduction
Executed: 10 files / 202 tests covering existing Go readers, mappings, provider,
CLI/MCP and Rust readers, plus selected rows, duplicate paths, empty filters,
identity-only fields, malformed/mutable parents, source conflicts, bounded
messages and unavailable metadata. Actual CLI complete output is identical
before/after; summary bundles retain that exact parent, and file-based CLI
selected queries match the native oracle. Full `npm run check` passed.
After rebasing onto main `c374c2fe7`, the 202 focused tests, full
`npm run check` and `npm run docs:check` (including five README consistency
tests) pass again. A real SDK selected query and complete export also pass
against that rebuilt runtime, retaining the original parent identity and all
165 dependencies. These checks do not relabel the frozen timing cohort.
A review found that a transcript-write failure could skip the SDK driver's
cleanup. The shared finalizer now closes owned resources before publishing the
optional transcript and records publication failures separately. All 11 receipt
failure tests pass. A real selected SDK task and full export pass again; the
same task with an injected transcript-write failure exits 1, preserves its
answer and failure receipt, and verifies that the process family was cleaned.
These qualification runs do not replace the frozen performance measurements.
A second review found that a valid external parent could disagree with its
report's architecture. Go domain now owns the existing architecture-to-subject
mapping; both Evidence production and every Go view use it. The view checks
mapped architecture as well as path, digest and format, including `null` for
unrecognized report names. All 213 focused tests pass, including authenticated
inline/file conflicts and known-alias/unknown-architecture cases.
Initial CI also exposed two existing baseline failures. On unmodified `c374c2fe7`,
`src/cliJsonPaths.test.ts` has the same three URL-message assertion failures.
The duplication gate is already above its 3.5% threshold: 31,737 / 899,602 lines
(3.52789%) on baseline versus 31,748 / 900,474 (3.52570%) on the initial PR head.
These were reproduced separately with the pinned toolchain, bounded resources
and verified cleanup; this PR leaves those unrelated failures unchanged.
Final qualification rebased onto main `cf5194f68`, preserving its new IL2CPP
view contract alongside the Go additions. All 258 tests in 12 files pass,
including 32 IL2CPP view tests and the now-fixed CLI path assertions. Full
`npm run check`, `npm run docs:check` plus five README tests, and the real SDK
selected query/full export pass again. Complete parent identity and all 165
dependencies remain exact. Duplication now passes: baseline 31,644 / 904,778
lines (3.49743%), PR 31,612 / 905,740 (3.49018%), against 3.5%.
One latest-main build was guard-stopped during a host-memory dip; its owned
scope was verified inactive with zero OOM events, then the unchanged guard's
retry passed. These results qualify the rebased source, not new task timings.
The broader metadata audit also read three real Rust binaries. Alpine ripgrep
14.1.1-r0's original `.dep-v0` matches native `readelf` coordinates and independent
zlib/JSON decoding: 42 packages, 88 indexed edges, root index 33. The other two
have no dependency section, which remains unknown coverage. Compiler-path
inferences in all three match native read-only ranges and original byte offsets;
they remain inferences. These fast metadata reads did not justify a Rust cache.
The Alpine archive is pinned by observed SHA-256; APK signatures were not checked.
Real provider verification here covers Linux ELF; PE/Mach-O use authored fixtures.
To reproduce, use built baseline/candidate runtimes and the pinned public input.
Create a baseline complete CLI Evidence JSON, then pin the independent oracle:
```sh
python3 experiments/analysis-efficiency/go-build-view-oracle.py \
--evidence /tmp/baseline-evidence.json --input /absolute/gh \
--go /absolute/go --dependency golang.org/x/crypto --output /tmp/go-oracle.json
```
Run `go-build-views.mjs` with absolute `--runtime`, `--input`, `--oracle` and
new `--output` directories, and `--mode complete` or `selected`, under
`resource-run.py --observe-payload-exit`. `--warm` adds a separate diagnostic.
The wire driver additionally requires pinned `--codex` and `--node`, and takes
`--question identity --mode complete|summary` or
`--question dependency --mode complete|selected`; `--export` adds export.
All wire recordings require the same resource guard for owned-scope cleanup.
Counter scripts take `--output` and recording directories, with their guard
logs at the same path plus `.log`. Both counters require `tiktoken==0.12.0`.
@@ -1,228 +0,0 @@
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { finished } from "node:stream/promises";
import { pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { createRetainedQueryCaller } from "./retained-query-calls.mjs";
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
const { values } = parseArgs({
options: {
runtime: { type: "string" },
input: { type: "string" },
oracle: { type: "string" },
output: { type: "string" },
mode: { type: "string" },
warm: { type: "boolean", default: false },
},
});
for (const key of ["runtime", "input", "oracle", "output"])
assert.equal(values[key], resolve(values[key]));
assert.ok(["complete", "selected"].includes(values.mode));
const oracle = JSON.parse(await readFile(values.oracle, "utf8"));
const sha256 = (bytes) => createHash("sha256").update(bytes).digest("hex");
const bytes = await readFile(values.input);
assert.equal(sha256(bytes), oracle.artifact_sha256);
const require = createRequire(join(values.runtime, "package.json"));
const { Client } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client"))
);
const { StdioClientTransport } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio"))
);
await mkdir(values.output, { mode: 0o700 });
const account = await mkdtemp(join(tmpdir(), "rea-go-build-views-"));
const report = {
question:
"Report compiler, main module and the selected embedded dependency; preserve complete original Evidence.",
mode: values.mode,
warm_diagnostic: values.warm,
calls: [],
artifact_sha256: oracle.artifact_sha256,
};
const client = new Client({ name: "rea-go-build-views", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
env: {
HOME: account,
PATH: process.env.PATH,
NODE_OPTIONS: "--max-old-space-size=2048",
UV_THREADPOOL_SIZE: "1",
REA_LOG_LEVEL: "silent",
},
stderr: "pipe",
});
const serverLog = createWriteStream(join(values.output, "server.log"));
const serverLogFinished = finished(serverLog);
serverLogFinished.catch(() => {});
transport.stderr?.pipe(serverLog);
const transcript = [];
const call = createRetainedQueryCaller(client, (receipt, event) => {
report.calls.push(receipt);
transcript.push(event);
});
async function result(name, input, phase = name) {
const reply = await call(name, input, phase);
assert.ok(!reply.isError, JSON.stringify(reply));
assert.ok(reply.structuredContent);
assert.equal(reply.content.length, 1);
assert.equal(reply.content[0].type, "text");
assert.deepEqual(JSON.parse(reply.content[0].text), reply.structuredContent);
return reply.structuredContent;
}
try {
await client.connect(transport);
const catalog = await client.listTools();
assert.equal(catalog.nextCursor, undefined);
report.catalog_tools = catalog.tools.length;
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
const original = client.callTool.bind(client);
client.callTool = (input, options) =>
original(input, {
...options,
timeout: 60000,
toolDefinition: definitions.get(input.name),
});
const inspected = await result(
"inspect_go_binary",
{
path: values.input,
...(values.mode === "selected" ? { detail: "summary" } : {}),
},
"inspection:first",
);
let answer;
if (values.mode === "complete") {
report.parent_evidence_id = inspected.evidence_id;
const info = inspected.normalized_result.build_info;
assert.ok(info);
answer = {
go_version: info.go_version,
main: info.module.main,
runtime_dependencies: "unknown",
dependencies: info.module.dependencies.flatMap((row, index) =>
row.path === oracle.dependency_path ? [{ index, ...row }] : [],
),
};
} else {
const summary = inspected.normalized_result;
assert.equal(summary.kind, "summary");
report.parent_evidence_id = summary.parent_evidence_id;
assert.equal(summary.source_coverage.runtime_dependencies, "unknown");
assert.equal(
summary.summary.build_info.module.available_counts.dependencies,
oracle.dependency_count,
);
const input = {
source: {
kind: "retained-evidence",
evidence_id: report.parent_evidence_id,
},
view: {
kind: "go",
collection: "dependencies",
paths: [oracle.dependency_path],
fields: ["path", "version", "sum", "replacement"],
offset: 0,
limit: 64,
},
};
let dependencies = [];
if (summary.summary.build_info.module.available_counts.dependencies > 0) {
const selected = await result(
"inspect_analysis_view",
input,
"retained-query:first",
);
assert.equal(selected.normalized_result.coverage.exhausted, true);
assert.equal(
selected.normalized_result.source_coverage.runtime_dependencies,
"unknown",
);
dependencies = selected.normalized_result.page.rows;
if (values.warm) {
const warm = await result(
"inspect_analysis_view",
input,
"retained-query:warm-diagnostic",
);
assert.deepEqual(warm, selected);
}
}
answer = {
go_version: summary.summary.build_info.go_version,
main: summary.summary.build_info.module.main,
runtime_dependencies: "unknown",
dependencies,
};
}
assert.deepEqual(answer, oracle.answer);
report.answer = answer;
await result("export_evidence_bundle", {
path: join(values.output, "evidence.json"),
});
const bundle = JSON.parse(
await readFile(join(values.output, "evidence.json"), "utf8"),
);
const parent = bundle.records.find(
(e) => e.evidence_id === report.parent_evidence_id,
);
assert.ok(parent);
assert.equal(parent.operation, "inspect_go_binary");
assert.deepEqual(parent.parameters, { path: values.input });
assert.equal(
parent.normalized_result.artifact.sha256,
oracle.artifact_sha256,
);
// The pre-change normalized result was independently checked against native
// go version -m and its file-backed bytes before pinning this exact JSON.
const expected = JSON.parse(
await readFile(oracle.complete_result_path, "utf8"),
);
assert.deepEqual(parent.normalized_result, expected);
assert.equal(
parent.normalized_result.build_info.module.dependencies.length,
oracle.dependency_count,
);
for (const name of ["version", "module"]) {
const info = parent.normalized_result.build_info;
const location = info[name + "_location"];
const encoded =
info[(name === "version" ? "go_version" : "module") + "_bytes_base64"];
assert.deepEqual(
bytes.subarray(location.offset, location.offset + location.bytes),
Buffer.from(encoded, "base64"),
);
}
assert.equal(sha256(await readFile(values.input)), oracle.artifact_sha256);
report.oracle_file_sha256 = sha256(await readFile(values.oracle));
report.complete_result_file_sha256 = sha256(
await readFile(oracle.complete_result_path),
);
report.status = "observed";
} catch (cause) {
report.status = "failed";
report.failure = String(cause?.stack ?? cause);
process.exitCode = 1;
} finally {
if (
!(await finalizeRetainedQueryReceipt({
report,
client,
transport,
account,
serverLog,
serverLogFinished,
output: values.output,
transcript,
}))
)
process.exitCode = 1;
}
@@ -1,86 +0,0 @@
"""Count entire captured Codex request sequences, without claiming model usage."""
import argparse
import hashlib
import json
from pathlib import Path
import tiktoken
def compact(value):
return json.dumps(value, ensure_ascii=False, separators=(",", ":"))
def request_text(body):
pieces = []
if isinstance(body.get("instructions"), str):
pieces.append(body["instructions"])
for item in body.get("input", []):
kind = item.get("type")
if kind == "function_call_output":
pieces.append(item["output"])
elif kind == "function_call":
pieces.append(item["arguments"])
elif kind == "message" or "role" in item:
for part in item.get("content", []):
if isinstance(part, dict) and isinstance(part.get("text"), str):
pieces.append(part["text"])
return pieces
def inspect(root, encoding):
report = json.loads((root / "report.json").read_text())
assert (report["status"] == "passed" and not report["cleanup_failures"]
and report["appserver_exit_code"] == 0)
requests = json.loads((root / "model-requests.json").read_text())
whole, namespaces, texts = [], [], []
count = lambda value: len(encoding.encode(value, disallowed_special=()))
for event in requests:
body = event["body"]
whole.append(count(compact(body)))
namespace = next(tool for tool in body["tools"]
if tool.get("type") == "namespace"
and tool["name"] == "mcp__rea_audit")
namespaces.append(count(compact(namespace)))
texts.append(sum(count(piece) for piece in request_text(body)))
log = root.with_suffix(".log")
scope = next(json.loads(line) for line in reversed(log.read_text().splitlines())
if line.startswith('{"resource_scope":'))
observed = scope["payload_resource_observation"]
assert (scope["abort"] is None and scope["payload_resource_observation_verified"]
and observed["payload_cleanup_verified"] and observed["payload_exit_code"] == 0)
memory = observed["resource_events"]["memory.events"]
assert all(memory[key] == 0 for key in ["high", "max", "oom", "oom_kill"])
assert observed["resource_events"]["pids.events"]["max"] == 0
return {"case": root.name, "question": report["question_kind"],
"request_count": len(requests), "whole_request_json_proxy": whole,
"whole_request_json_proxy_sum": sum(whole),
"rea_namespace_json_proxy": namespaces,
"rea_namespace_json_proxy_sum": sum(namespaces),
"unescaped_instruction_and_input_text_proxy": texts,
"rea_namespace_plus_text_proxy_sum": sum(namespaces) + sum(texts),
"wall_seconds": scope["elapsed_seconds"],
"family_cpu_seconds": observed["cpu_stat"]["usage_usec"] / 1e6,
"family_peak_mib": observed["memory_peak_bytes"] / 2**20,
"completed_with_verified_cleanup": True,
"sha256": {file.name: hashlib.sha256(file.read_bytes()).hexdigest()
for file in [root / "report.json", root / "model-requests.json",
root / "transcript.json", log]}}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("cases", type=Path, nargs="+")
args = parser.parse_args()
encoding = tiktoken.get_encoding("cl100k_base")
receipt = {"actual_model_usage": None,
"backend": "deterministic local Responses SSE; no AI inference or paid service; not a model-quality or billing test",
"client": "Codex app-server 0.163.0", "tiktoken": tiktoken.__version__,
"encoding": encoding.name,
"records": [inspect(root, encoding) for root in args.cases]}
args.output.write_text(json.dumps(receipt, ensure_ascii=False, indent=2) + "\n")
if __name__ == "__main__":
main()
@@ -1,292 +0,0 @@
"""Actual Codex wire capture with a deterministic local SSE backend; no AI inference."""
import argparse
import http.server
import threading
import asyncio
import hashlib
import json
import os
from pathlib import Path
import re
import time
import traceback
import subprocess
async def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--codex", type=Path, required=True)
parser.add_argument("--node", type=Path, required=True)
parser.add_argument("--input-profile", choices=["full", "compact"], default="full")
parser.add_argument("--output-profile", choices=["full", "off"], default="full")
parser.add_argument("--input", type=Path, required=True)
parser.add_argument("--oracle", type=Path, required=True)
parser.add_argument("--expected-tools", type=int, default=210)
parser.add_argument("--mode", choices=["complete", "selected", "summary"], required=True)
parser.add_argument("--export", action="store_true")
parser.add_argument("--question", choices=["identity", "dependency"], default="dependency")
args = parser.parse_args()
assert args.question != "identity" or args.mode != "selected"
assert args.mode != "summary" or args.question == "identity"
oracle=json.loads(args.oracle.read_text())
target=str(args.input)
assert args.input.is_absolute() and args.oracle.is_absolute() and args.output.is_absolute()
assert hashlib.sha256(args.input.read_bytes()).hexdigest()==oracle["artifact_sha256"]
def delivered(pieces):
assert isinstance(pieces,str) and pieces.startswith("Wall time: "), pieces[:300]
_,sep,payload=pieces.partition("Output:\n");assert sep
value=json.loads(payload)
return value["result"] if "result" in value and "normalized_result" not in value else value
def outputs_of(body):
return {item["call_id"]:delivered(item["output"]) for item in body.get("input",[]) if item.get("type")=="function_call_output"}
steps=1+(args.mode=="selected")+args.export
args.output.mkdir(mode=0o700)
audit_home = args.output / "account"
codex_state = audit_home / ".codex"
mcp_home = audit_home / "mcp"
workspace = audit_home / "workspace"
for directory in [codex_state, mcp_home, workspace]:
directory.mkdir(parents=True)
captured = []
class Capture(http.server.BaseHTTPRequestHandler):
def log_message(self, *_):
pass
def do_POST(self):
size = int(self.headers["Content-Length"])
assert 0 < size <= 16 * 1024 * 1024
assert self.headers.get("Content-Encoding", "identity") == "identity"
body = json.loads(self.rfile.read(size))
captured.append({"path": self.path, "body": body})
(args.output / "model-requests.json").write_text(json.dumps(captured) + "\n")
index = len(captured)
names = [t.get("name", "") for ns in body.get("tools", []) if ns.get("type") == "namespace" and ns.get("name") == "mcp__rea_audit" for t in ns.get("tools", [])]
assert index <= steps+1
response_id=f"resp_local_{index}"
if index <= steps:
previous=outputs_of(body)
if index==1:
operation="inspect_go_binary";arguments={"path":target}
if args.mode!="complete":arguments["detail"]="summary"
elif args.mode=="selected" and index==2:
parent=previous["call_local_1"]["normalized_result"]["parent_evidence_id"]
operation="inspect_analysis_view";arguments={"source":{"kind":"retained-evidence","evidence_id":parent},"view":{"kind":"go","collection":"dependencies","paths":[oracle["dependency_path"]],"fields":["path","version","sum","replacement"],"offset":0,"limit":64}}
else:
operation="export_evidence_bundle";arguments={"path":str(args.output/"evidence.json")}
selected=[name for name in names if name.endswith(operation)];assert len(selected)==1
item={"type":"function_call","id":f"fc_local_{index}","call_id":f"call_local_{index}","name":selected[0],"namespace":"mcp__rea_audit","arguments":json.dumps(arguments),"status":"completed"}
else:
item={"type":"message","id":"msg_local_done","role":"assistant","status":"completed","content":[{"type":"output_text","text":"Scripted local Go metadata query complete.","annotations":[]}]}
base = {"id": response_id, "object": "response", "model": body.get("model"), "status": "completed", "output": [item]}
events = [
{"type": "response.created", "response": {**base, "status": "in_progress", "output": []}},
{"type": "response.output_item.added", "output_index": 0, "item": {**item, "status": "in_progress"}},
{"type": "response.output_item.done", "output_index": 0, "item": item},
{"type": "response.completed", "response": base},
]
wire = "".join("event: " + e["type"] + "\ndata: " + json.dumps(e) + "\n\n" for e in events).encode()
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.send_header("Content-Length", str(len(wire)))
self.end_headers()
self.wfile.write(wire)
endpoint = http.server.HTTPServer(("127.0.0.1", 0), Capture)
endpoint_thread = threading.Thread(target=endpoint.serve_forever, daemon=True)
endpoint_thread.start()
task_path = f"{args.node.parent}:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
mcp_env = {"HOME": str(mcp_home), "PATH": task_path,
"NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1",
"REA_LOG_LEVEL": "silent", "REA_MCP_INPUT_SCHEMA_PROFILE": args.input_profile,
"REA_MCP_OUTPUT_SCHEMA_PROFILE": args.output_profile}
config = 'model_provider = "rea_capture"\nmodel = "rea-wire-probe"\n[analytics]\nenabled = false\n[mcp_servers.rea_audit]\n'
config += f'command = {json.dumps(str(args.node))}\n'
config += f'args = {json.dumps([str(args.runtime / "scripts/rea.mjs"), "mcp"])}\n'
config += 'startup_timeout_sec = 60\ntool_timeout_sec = 60\n'
config += '[mcp_servers.rea_audit.env]\n'
config += ''.join(f'{key} = {json.dumps(value)}\n' for key, value in mcp_env.items())
for name in ['inspect_go_binary','inspect_analysis_view','export_evidence_bundle']:
config += f'[mcp_servers.rea_audit.tools.{name}]\napproval_mode = "approve"\noutput_token_limit = 30000\n'
config += f'\n[model_providers.rea_capture]\nname = "Local scripted capture"\nbase_url = "http://127.0.0.1:{endpoint.server_port}/v1"\nwire_api = "responses"\nrequires_openai_auth = false\nsupports_websockets = false\nrequest_max_retries = 0\nstream_max_retries = 0\n'
(codex_state / "config.toml").write_text(config)
(args.output / "declared-config.toml").write_text(config)
environment = {"HOME": str(audit_home), "CODEX_HOME": str(codex_state), "PATH": task_path,
"NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1"}
report = {"question": "Read embedded Go identity/counts or a selected dependency. Export is a separate task variant.",
"client": "Codex app-server 0.163.0", "model_turns_started": 1, "backend": "local deterministic SSE fixture; no AI model or paid service",
"actual_model_usage": None, "input_profile": args.input_profile,
"output_profile": args.output_profile, "calls": [], "observations": [],
"unexpected_server_requests": [], "cleanup_failures": []}
client_version = subprocess.check_output([str(args.codex), "--version"], env=environment, cwd=workspace, text=True, timeout=10).strip()
assert client_version == "codex-cli 0.163.0", client_version
transcript = []
pending = {}
completed_turn = asyncio.Event()
next_id = 0
stderr = (args.output / "server.log").open("wb")
process = await asyncio.create_subprocess_exec(str(args.codex), "app-server", "--stdio",
stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE, stderr=stderr,
cwd=workspace, env=environment, limit=16 * 1024 * 1024)
async def send(message):
encoded = (json.dumps(message, separators=(",", ":")) + "\n").encode()
transcript.append({"direction": "client", "bytes": len(encoded), "message": message})
process.stdin.write(encoded)
await process.stdin.drain()
async def read_messages():
try:
while line := await process.stdout.readline():
value = json.loads(line)
transcript.append({"direction": "server", "bytes": len(line), "message": value})
if value.get("method") == "turn/completed":
report["turn"] = value["params"]["turn"]
completed_turn.set()
if "id" in value and "method" not in value:
future = pending.pop(value["id"], None)
if future and not future.done():
future.set_result(value)
elif "id" in value:
report["unexpected_server_requests"].append(value["method"])
await send({"id": value["id"], "error": {"code": -32601, "message": "Unexpected request in scripted metadata probe"}})
raise RuntimeError("app-server stdout closed")
except Exception as error:
for future in pending.values():
if not future.done():
future.set_exception(error)
reader = asyncio.create_task(read_messages())
async def request(method, params):
nonlocal next_id
assert method in ["initialize", "thread/start", "mcpServerStatus/list", "mcpServer/tool/call", "turn/start"]
next_id += 1
message = {"id": next_id, "method": method, "params": params}
future = asyncio.get_running_loop().create_future()
pending[next_id] = future
start = time.monotonic()
await send(message)
reply = await asyncio.wait_for(future, 90)
report["calls"].append({"method": method, "elapsed_ms": (time.monotonic() - start) * 1000,
"error": reply.get("error")})
if "error" in reply:
raise RuntimeError(json.dumps(reply["error"])[:4000])
return reply["result"]
try:
await request("initialize", {"clientInfo": {"name": "rea_host_probe", "title": "REA host compatibility probe", "version": "1"},
"capabilities": {"experimentalApi": True, "explicitGatewayOauth": True}})
await send({"method": "initialized"})
created = await request("thread/start", {"cwd": str(workspace), "ephemeral": True,
"approvalPolicy": "never", "sandbox": "read-only"})
thread_id = created["thread"]["id"]
assert not created["thread"].get("turns", [])
inventory = await request("mcpServerStatus/list", {"threadId": thread_id,
"serverName": "rea_audit", "detail": "toolsAndAuthOnly", "limit": 10})
assert inventory.get("nextCursor") is None
assert len(inventory["data"]) == 1
server = inventory["data"][0]
assert server["name"] == "rea_audit" and server.get("toolsError") is None
tools = list(server["tools"].values())
names = [tool["name"] for tool in tools]
assert len(names) == len(set(names)) == args.expected_tools
assert "inspect_go_binary" in names
output_schemas = sum(tool.get("outputSchema") is not None for tool in tools)
assert output_schemas == (args.expected_tools if args.output_profile == "full" else 0)
raw = json.dumps(inventory, separators=(",", ":"), ensure_ascii=False).encode()
(args.output / "catalog.json").write_bytes(raw)
report["catalog"] = {"tools": len(tools), "output_schemas": output_schemas,
"bytes": len(raw), "sha256": hashlib.sha256(raw).hexdigest(),
"server_info": server.get("serverInfo"), "runtime_status": server.get("runtimeStatus")}
await request("turn/start", {"threadId": thread_id, "input": [{"type": "text", "text": ("Read the embedded Go compiler, main module and embedded dependency count; state runtime dependency coverage." if args.question=="identity" else "Read the embedded Go compiler, main module and golang.org/x/crypto dependency and state runtime dependency coverage.")+" The local fixture scripts the required tools; no AI inference is used."}]})
await asyncio.wait_for(completed_turn.wait(), 90)
assert report["turn"]["status"] == "completed", report["turn"]
assert len(captured)==steps+1
outputs=outputs_of(captured[-1]["body"])
assert len(outputs)==steps
initial=outputs["call_local_1"]
assert initial["normalized_result"]["artifact"]["sha256"]==oracle["artifact_sha256"]
if args.question=="identity":
info=initial["normalized_result"]["build_info"] if args.mode=="complete" else initial["normalized_result"]["summary"]["build_info"]
count=len(info["module"]["dependencies"]) if args.mode=="complete" else info["module"]["available_counts"]["dependencies"]
facts={"go_version":info["go_version"],"main":info["module"]["main"],"runtime_dependencies":"unknown","dependency_count":count}
parent=initial["evidence_id"] if args.mode=="complete" else initial["normalized_result"]["parent_evidence_id"]
elif args.mode=="complete":
info=initial["normalized_result"]["build_info"]
facts={"go_version":info["go_version"],"main":info["module"]["main"],"runtime_dependencies":"unknown","dependencies":[{"index":i,**d} for i,d in enumerate(info["module"]["dependencies"]) if d["path"]==oracle["dependency_path"]]}
parent=initial["evidence_id"]
else:
info=initial["normalized_result"]["summary"]["build_info"]
selected=outputs["call_local_2"]["normalized_result"]
assert selected["coverage"]["exhausted"] and selected["source_coverage"]["runtime_dependencies"]=="unknown"
facts={"go_version":info["go_version"],"main":info["module"]["main"],"runtime_dependencies":"unknown","dependencies":selected["page"]["rows"]}
parent=initial["normalized_result"]["parent_evidence_id"]
expected=({key:oracle["answer"][key] for key in ["go_version","main","runtime_dependencies"]}|{"dependency_count":oracle["dependency_count"]}) if args.question=="identity" else oracle["answer"]
assert facts==expected
report["question_kind"]=args.question
if args.export:
bundle=json.loads((args.output/"evidence.json").read_text());evidence=next(e for e in bundle["records"] if e["evidence_id"]==parent)
assert evidence["normalized_result"]==json.loads(Path(oracle["complete_result_path"]).read_text())
report["observations"]={"answer":facts,"parent_evidence_id":parent,"exported":args.export}
report["captured_requests"]=len(captured)
report["mode"]=args.mode
report["request_tool_counts"] = [len(c["body"].get("tools", [])) for c in captured]
assert not report["unexpected_server_requests"]
assert hashlib.sha256(args.input.read_bytes()).hexdigest()==oracle["artifact_sha256"]
report["status"] = "passed"
except Exception:
report["status"] = "failed"
report["failure"] = traceback.format_exc()[-6000:]
finally:
process.stdin.close()
try:
await asyncio.wait_for(process.wait(), 15)
except asyncio.TimeoutError:
report["cleanup_failures"].append("app-server did not exit after stdin close; terminated owned process")
process.terminate()
try:
await asyncio.wait_for(process.wait(), 5)
except asyncio.TimeoutError:
process.kill()
await process.wait()
await reader
# app-server can acknowledge exit before its MCP child finishes EOF
# cleanup. Wait on the allocated resource scope; never target a PID.
try:
groups = [line[3:] for line in Path("/proc/self/cgroup").read_text().splitlines()
if line.startswith("0::")]
assert len(groups) == 1
group = Path("/sys/fs/cgroup") / groups[0].lstrip("/")
assert re.fullmatch(r"rea-efficiency-[0-9a-f]{32}\.scope", group.name)
observer = os.getppid()
observer_args = Path(f"/proc/{observer}/cmdline").read_bytes().split(b"\0")
assert any(Path(os.fsdecode(value)).name == "resource_payload.py" for value in observer_args if value)
expected = {os.getpid(), observer}
until = time.monotonic() + 10
while True:
actors = {int(value) for value in (group / "cgroup.procs").read_text().splitlines()}
assert expected <= actors
remaining = sorted(actors - expected)
if not remaining or time.monotonic() >= until:
break
await asyncio.sleep(0.05)
report["owned_scope_drain"] = {"remaining_processes": remaining}
if remaining:
report["cleanup_failures"].append("owned MCP child did not settle after app-server exit")
except (OSError, ValueError, AssertionError) as error:
report["owned_scope_drain"] = {"status": "unavailable", "reason": type(error).__name__}
report["cleanup_failures"].append("allocated resource-scope cleanup could not be verified")
endpoint.shutdown()
endpoint.server_close()
endpoint_thread.join(5)
stderr.close()
report["appserver_exit_code"] = process.returncode
if process.returncode != 0:
report["cleanup_failures"].append(f"app-server exit code: {process.returncode}")
(args.output / "transcript.json").write_text(json.dumps(transcript) + "\n")
(args.output / "report.json").write_text(json.dumps(report, indent=2) + "\n")
print(json.dumps({key: report.get(key) for key in ["status", "failure", "catalog", "observations", "cleanup_failures", "appserver_exit_code"]}))
return 0 if report["status"] == "passed" and not report["cleanup_failures"] else 1
raise SystemExit(asyncio.run(main()))
Loaded 100 of 622 files, more files were not shown because too many files have changed in this diff. Show more