admin/rea
1
0 Fork 0
mirror of https://github.com/morluto/rea.git synced 2026-10-29 12:00:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(browser): reuse saved response values across scenario steps (#7267)

Browse Source 
Save typed response fields and reuse generated IDs or authentication values across browser scenario requests, UI observations and navigation through the shared CLI/MCP workflow.

Preserve exact numbers, retained Evidence, explicit source/coverage limits and prospective page-value semantics. Bound response decoding, retained spellings and request serialization before allocation, and protect marked response values across source records, follow-up text and planned screenshots.

Integrates Edgar Arakelyan's #7162, including follow-ups through 9f5191ab0. Final local validation: 560 feature-branch tests and 561 current-main integration tests, static/build/docs checks and real PocketBase CLI/MCP privacy workflows. Whole-task comparison and verification limits are recorded in docs/verification/browser-saved-values-2026-10-26.md.

Co-authored-by: Edgar Arakelyan <[email protected]>
This commit is contained in:
gensokyo-devandEdgar Arakelyan authored and GitHub committed 2026-10-27 08:15:52 +08:00
1 parent 60822e0702
commit 3fcbabe593
54 files changed
+7287 -370
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.agents/skills/reverse-engineer-anything/SKILL.md
+1 -1
View File
@@ -2,7 +2,7 @@
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "49"
version: "50"
---
# REA
.agents/skills/reverse-engineer-anything/references/environments.md
+72 -26
View File
@@ -80,20 +80,37 @@ shape, with the application's own locators, paths and headers:
"path": "/api/folders",
"expected_status": [200]
},
{
"step_id": "new-folder-1",
"action": "click",
"timeout_ms": 15000,
"locator": { "kind": "role", "role": "button", "name": "New folder" }
},
{
"step_id": "folder-title-1",
"action": "fill",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=title]" },
"value": { "source": "literal", "value": "Scratch folder 1" }
},
{
"step_id": "create-folder-1",
"action": "click",
"timeout_ms": 15000,
"locator": { "kind": "role", "role": "button", "name": "Create" },
"save_response": {
"method": "POST",
"path": "/api/folders",
"save": [{ "name": "folder_1", "json_pointer": "/id" }]
}
},
{
"step_id": "read-folder-1",
"action": "request",
"timeout_ms": 15000,
"method": "POST",
"path": "/api/folders",
"json": { "title": "Scratch folder 1" },
"headers": [
{
"name": "X-CSRFToken",
"value": { "source": "cookie", "name": "csrftoken" }
}
],
"expected_status": [200, 201]
"method": "GET",
"path": "/api/folders/{saved:folder_1}",
"expected_status": [200]
}
]
}
@@ -109,13 +126,19 @@ client with it. Match `wait_for` locators to exactly one element.
2. Read with `request` steps: each collection the environment needs, twice
before any write, so fields that change on their own show up; one item of
each.
3. Use `request` steps only when the API lets the caller choose each scratch
object's ID or a prior read already established it. Request paths are fixed
strings; no action substitutes a value from an earlier response. For
server-assigned IDs, create through the UI and open the new object's detail
view to record its read, edit and delete through UI actions. If that journey
cannot be scripted, leave the write unserved and report the limitation;
never guess an ID or create objects with no supported cleanup path.
3. Write the way the application's UI does: fill its forms and click its
buttons, so each request carries the fields the UI sends, including ones
the server would otherwise derive. Give the click or key press that submits
a create `save_response` to keep the new object's ID, then `goto` its page
(`/items/{saved:item_1}`) or read it with a `request` step. Never guess an
ID, and never create a probe object to learn the next one. Write with
`request` steps instead only when the UI cannot reach the operation (an
API-only resource, or a UI that does not call the JSON API), or when coverage
withholds the UI-shaped write as `unexplained-change` or `contradicted`;
that counts as one of the two re-recordings. A `request` step write that
coverage withholds that way usually lacks fields the UI sends: record it
through the UI instead. If a journey cannot be scripted, leave the write
unserved and report the limitation.
Record two numbered scratch objects per write type,
with varied free text. After each write, and before the next one, read the
object back by id and then its list: create 1, read 1, list, create 2,
@@ -129,13 +152,34 @@ About `request` steps:
- `path` is on the start URL's origin; the page must be on that origin.
- Header values are literals, declared secrets, a cookie the browser holds
(`{source: "cookie", name}`), or a local storage entry
(`{source: "local_storage", key, json_pointer}`), with an optional
`prefix` such as `"Bearer "`. Copy the headers the application's own
requests send (a CSRF header, a bearer token). Values read from the page
are redacted from records made after they are read, not from the login
response that returned them, and values shorter than 8 characters are not
redacted at all.
(`{source: "cookie", name}`), a local storage entry
(`{source: "local_storage", key, json_pointer}`), a saved value
(`{source: "saved", name}`), or Basic credentials
(`{source: "basic_credentials", parts: [username, password]}`, sent as the
base64 of `username:password` and redacted), with an optional `prefix` such as
`"Bearer "` or `"Basic "`. Each credential part is a literal, secret, saved
value, cookie or local storage entry, so a header the frontend builds from a
login token (the username and the token) can be copied too; the password is
a secret or saved value, never a literal. Copy the
headers the application's own requests send (a CSRF header such as
`X-CSRFToken` from the `csrftoken` cookie, a bearer token). Values read from the page are redacted from records made after they
are read, not from the login response that returned them, and values
shorter than 8 characters are not redacted at all.
- `save: [{name, json_pointer, secret}]` keeps a string or number from the
step's JSON response, such as the created object's `/id`, for later steps;
`secret: true` redacts it, as for a token the API returns, so mark every
saved token, key or session value secret. This also protects the selected
source response before its body or headers are retained. Compressed or
unknown-size UI responses cannot supply saved values. On a `click` or `press`,
`save_response: {method, path, save}` keeps values from the response to the
new request observed while the step waits (`{*}` in `path` matches one segment).
Matching does not prove UI causality: choose a path distinct from polling;
multiple observed matches fail as ambiguous. The step
fails unless a 2xx JSON response arrives before its `timeout_ms`. Later
steps send a saved value as `{saved:name}` in a request `path` (and its
query) or a `goto` URL; in the `json` body, where a string that is only
`{saved:name}` keeps the saved JSON type; or as `{source: "saved", name}` in
a header or credential part.
- Reading a page value makes later bodies costlier to keep: after a CSRF
token, bodies over about 47 KiB are kept as `unavailable`. When the API
returns larger responses, raise `capture.limits`
@@ -184,8 +228,10 @@ withheld with its hint:
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
| `body-unusable` | Stop: the body is not a JSON object. |
Use new scratch names on each attempt; the first attempt's objects may still
exist.
Use new scratch names on each attempt, and delete what an earlier attempt
created before recording again (through the UI, or a `request` step with the
id that capture shows): the environment starts from what the recording first
reads, so a leftover object becomes seeded data that write tasks then find.
New browser scenario captures include request times, so a fully timed capture
can learn a write clock and serve updates that stamp modification times. Older
captures or ones missing a request start keep record order and learn no clock.
docs/agent-environments.md
+4 -1
View File
@@ -205,7 +205,10 @@ attached and a transaction no record represents, and each transaction URL's
query once. Bodies larger than 4 MiB of UTF-8 text are not
scanned; `credential-scan-incomplete` counts them and lists up to 20 record or
event pointers, including base64 bodies whose decoded scan was cut. An empty name
warning does not establish that these bodies were checked. Names do not establish sensitivity,
warning does not establish that these bodies were checked. A browser scenario's
saved values (`save` and `save_response`) named like credentials but not marked
`secret: true` are recorded unredacted; `credential-named-saved-values` lists up
to 20 of them by parameter pointer. Names do not establish sensitivity,
so the build never refuses on them. Capture
Evidence also records the capture file's absolute path (`subject.local_path`,
docs/browser-scenario-contract.md
+149 -1
View File
@@ -213,7 +213,8 @@ HTTP(S) URLs may include ordinary query values and fragments directly. Use
ordered structured query entries when a value needs to reference a declared
environment-backed secret. Form, storage, and cookie values are either literal
strings or declared secret references; request step headers can also read a
cookie or local storage value from the page. Raw URL userinfo credentials are rejected.
cookie or local storage value from the page, and request steps and `goto`
destinations can send [saved values](#saved-values). Raw URL userinfo credentials are rejected.
Captured credential-header
values are not retained, and declared secret values are redacted automatically.
A secret may be declared solely to
@@ -406,6 +407,17 @@ data you may change. This scenario creates one item:
by default) get smaller: after a 32-character `X-CSRFToken` value, a body over
about 47 KiB is kept as `unavailable`. Raise `capture.limits` when the
application's responses are larger.
- A header can send Basic credentials, `{"source": "basic_credentials", "parts":
[username, password]}`: the base64 of `username:password`, after a `prefix`
such as `"Basic "` or with none, as some APIs take encoded credentials in
their own header. Each part is a literal, a declared secret, a saved value, a
cookie or a local storage entry, so a header a frontend builds from a login
token can be sent too; the password is never a literal other than empty, since
Evidence records literals as written, and a username with a colon fails the
step, since Basic authentication ends the username at its first colon. The
encoded value is redacted however short, as
`[REDACTED:page:<step_id>.<header>]`, and parts read from the page are
redacted as page values are.
- Validation refuses a header name given twice in a step, header names a
page's `fetch` cannot set, such as `Cookie`, `Origin`, `Host` or `Sec-*`, and
`User-Agent`, which Chrome drops; it refuses line breaks and other ASCII
@@ -431,6 +443,142 @@ Captures with request steps state in `limitations` that the steps sent their
declared requests through the page's fetch. A scenario without request steps
is unchanged.
## Saved values
A step can keep a string or number from a JSON response, by name, and a later
step can send it, so a scenario can read back, edit and delete an object whose
id the application assigns:
```json
{
"browser": {
"mode": "launch",
"executable_path": "/absolute/path/to/chromium"
},
"start_url": { "url": "http://127.0.0.1:3000/items" },
"actions": [
{
"step_id": "save-item-1",
"action": "click",
"timeout_ms": 10000,
"locator": { "kind": "role", "role": "button", "name": "Save" },
"save_response": {
"method": "POST",
"path": "/api/items",
"save": [{ "name": "item_1", "json_pointer": "/id" }]
}
},
{
"step_id": "open-item-1",
"action": "goto",
"timeout_ms": 10000,
"destination": { "url": "http://127.0.0.1:3000/items/{saved:item_1}" },
"wait_until": "load"
},
{
"step_id": "copy-item-1",
"action": "request",
"timeout_ms": 10000,
"method": "POST",
"path": "/api/items/{saved:item_1}/copies",
"json": {
"source_id": "{saved:item_1}",
"title": "Copy of {saved:item_1}"
},
"expected_status": [201]
}
]
}
```
- A `request` step saves with `save: [{name, json_pointer, secret}]`, from its
own response. A `click` or `press` step saves with `save_response: {method,
path, save}`, from a newly observed matching response while the step waits.
The recorder watches before the action and excludes requests already running,
such as an earlier autosave. Method/path matching does not prove that the UI
caused the request: use a path that distinguishes the intended operation from
background polling. Multiple observed matches before binding fail as ambiguous;
requests starting after this observation window cannot be ruled out. It matches the method and a path on the
start origin as the browser spells it, so a path may name `My List` or
`My%20List` (`{*}` matches one segment, and the query is ignored), and fails
the step unless a 2xx JSON response arrives before its `timeout_ms`; one still
being read when the step times out or is cancelled saves nothing. A pointer
that names no string or number fails the step, so later steps never send a
missing value, as does a response over 4,000,000 characters: a request step
stops reading and aborts its fetch there. A UI step reads a body only when
its positive received size is known, at most 12,000,000 bytes, and its
Content-Encoding is absent or identity. Compressed responses and unknown or
zero cache sizes are refused before materializing the body; decoded text
over the character limit also fails. The UI action may already have applied.
- `{saved:name}` sends a saved value in a request step's `path`, including its
query, and in a `goto` destination's `url`, percent-encoded; a destination's
structured `query` values are sent as declared. A saved value that makes a
path segment `.` or `..`, alone or with the text beside it (including `%2e`),
fails the step, since the URL would drop it. In a request step's `json` body,
a string that is only `{saved:name}` becomes the saved value with the JSON
type it was saved with, so a saved number id is sent as a number, and
`{saved:name}` inside a longer string is filled as text. A header value or
Basic credentials part sends one as `{"source": "saved", "name"}`. Other
braces, such as `{id}`, are sent as written, and so is a percent-encoded
`%7Bsaved:name%7D` anywhere but a `goto` URL, whose parsed form spells the
braces that way.
- Validation refuses a saved value that no earlier step saves. A later save of
the same name replaces the value. An integer past 2^53, which JSON parsing
would round to another number, and a number that is not finite, such as
`1e400`, negative zero, or a value that underflows to zero, such as `1e-400`, fail the step
that saves it.
- A scenario keeps at most 4,000,000 characters of saved values at once,
counting distinct retained redaction spellings, including trimmed variants,
of values saved as secret. Replacing a binding releases its live value
charge, while secret spellings remain protected for the scenario lifetime.
A save past the budget fails before changing the binding.
- A value saved with `secret: true` is redacted however short, as a declared
secret is, as `[REDACTED:page:<step_id>.<name>]`. The selected source
response is protected before retaining its body or header values, including
when only headers were requested. If that body cannot be read safely, its
body and header values remain unavailable. Before publishing the capture,
the selected source transaction's earlier URL, request headers and request
body are also protected using the admitted values. Unresolved source values
withhold that transaction's affected metadata/content with an explicit
limitation. Before publication, retained events are rechecked
with the final marked-response values, including immediate follow-up requests
and their response bodies, console/page errors, WebSocket text and event URL/name fields. Learned redaction facts are independent of source-identity readiness. Unknown source values withhold only the selected
source transaction. Other saved values, such as ids, stay visible, since
environments model them. The environment build warns about saved values
named like credentials that are not marked secret
(`credential-named-saved-values`).
- Page-derived cookie/local-storage values and constructed Basic credentials
remain redacted only from observations made after they were read/built.
They do not retroactively rewrite earlier facts. The final event projection
borrows only declared and response-marked patterns; promotion of an already
held page value to an explicit response secret preserves its first marker.
Projection state is session-local, references admitted pattern/byte owners,
and is invalidated on admission or response classification changes.
- Screenshots are withheld as `missing` from the start when the scenario plans
any response save marked `secret: true`, page-derived or Basic authentication
header values, or whenever nonempty sensitive values
are already known. Raw pixels cannot be safely text-redacted, so the provider
does not request pixels in those cases. This also protects earlier steps and
partial failures without retaining an image to scrub later. Other requested
artifacts remain available; affected screenshot/aggregate completeness is
partial and is not eligible for equality.
- Saved URLs and expanded request JSON are admitted before building their expanded strings or
serializing. Its allowance is the smaller of Node's string limit and
one sixteenth of current V8 available heap bytes, allowing for the expanded
tree, serialization, Playwright transport and browser-side copies. Size
accounting includes JSON escaping and repeated references; each distinct
saved value's size is measured once per request. An oversized request fails
before browser dispatch; the complete request envelope, including repeated
header values, is checked as well. URL encoding uses a conservative
nine-characters-per-UTF-16-unit allowance before encoding a saved value.
Reduce the payload/repetitions, or raise the parent
Node heap and restart; this is a conservative allocation estimate, not a
measurement or guarantee of browser/process-family memory.
- Evidence records a request step's empty `save` list as absent, so a scenario
that saves nothing records the same parameters and `scenario_sha256` as
before saved values existed, and REA versions without saved values read its
captures.
## Network evidence
`capture.events: ["network"]` retains request, response, completion, failure,
docs/specs/agent-environments/scenario-saved-values/plan.md
+22
View File
@@ -0,0 +1,22 @@
# Plan
1. Schema: `save` on request steps, `save_response` on click and press, the
`saved` value source in headers and credential parts, `basic_credentials`
with two parts, and the `{saved:name}` placeholder.
2. Validation: every saved value a step sends was saved by an earlier step.
3. Runtime: read the response body when a step saves, start waiting for a UI
step's response before it acts, keep saved values with their JSON type,
fill placeholders, build Basic credentials, and redact secret saves.
4. Evidence: leave an empty `save` out of the recorded step.
5. Environment build: warn about credential-named saved values not marked
secret.
6. Skill and docs: write through the UI first and save ids instead of guessing
them; bump the skill version.
7. Tests: schema, runtime with a fake page, Evidence parameters, the build
warning, the catalog budget, and a real-Chrome recording.
8. Write model: compose strings from generated ids and other fields, learn the
`&` spelling of slugs, tell apart addresses whose parts are the same text,
and check templates on later creates; census every recorded capture.
9. End to end: agents build environments from prompts with main and with this
branch on the same applications, and the branch's recordings are rebuilt
before and after the write-model changes.
docs/specs/agent-environments/scenario-saved-values/spec.md
+74
View File
@@ -0,0 +1,74 @@
# Saved values in browser scenarios
## Outcome
A browser scenario could not carry a value from one step to a later one. An
agent recording an application from a prompt therefore guessed the id of the
object it had just created, created probe objects to learn the next id, or
read the object back by list only. An API that takes Basic auth, or a header a
frontend builds from a login token, could not be sent at all, so the agent
stopped and asked for an API key.
Steps now save strings and numbers from JSON responses, and later steps send
them. A click or key press saves from a newly observed matching response,
so a scenario can create an object the way the application's UI does and then
open, read, edit or delete it by the id the application assigned. A request
header can send Basic credentials built from literal, secret, saved or page
values.
## Requirements
- A `request` step saves `[{name, json_pointer, secret}]` from its own JSON
response. A `click` or `press` step's `save_response: {method, path, save}`
saves from the first response that matches its method and a path on the start
origin (`{*}` matches one segment; the query is ignored), which must be 2xx
JSON. The wait starts before the step acts, and the step fails, saving
nothing, unless such a response arrives and is read before its timeout or
cancellation.
- A pointer that names no string or number, a number that is not finite (such as
`1e400`), a response that is not JSON or is larger than 4,000,000 characters
(a UI step reads none received as more than 12,000,000 bytes, three per
character), and a non-2xx `save_response` fail the step.
- `{saved:name}` sends a saved value, percent-encoded, in a request path and its
query and in a goto URL, and in a request's JSON body: a string that is only
the placeholder becomes the saved value with its JSON type, and a placeholder
inside a longer string is filled as text. Header values and Basic credential
parts send `{source: "saved", name}`. Other braces are sent as written. A
saved value that makes a path segment `.` or `..`, alone or with the text
beside it, fails the step.
- `basic_credentials` sends the base64 of `username:password` from `parts:
[username, password]`, each a literal, declared secret, saved value, cookie or
local storage entry. The encoded value and parts read from the page are
redacted as page values are. A username with a colon fails the step.
- Validation refuses a saved value no earlier step saves.
- A scenario keeps at most 4,000,000 characters of saved values at once, counting
the redaction spellings of values saved as secret; a save past that fails its
step before the value is kept.
- A value saved with `secret: true` is redacted from the selected source response
before retention and from subsequent records. The environment build warns about saved values named like credentials
that are not marked secret (`credential-named-saved-values`).
- A scenario that saves nothing records the same Evidence parameters and
`scenario_sha256` as before: an empty `save` is left out of the recorded
step.
- Earlier REA versions read captures that use saved values.
- The compact MCP catalog stays under its measured one-megabyte frame.
- The skill records writes the way the application's UI does and falls back to
request steps when the UI cannot reach an operation or coverage withholds the
UI-shaped write.
- A UI-made create must be served as faithfully as one made through the API:
- a string composed from a generated id and another field, such as Grafana's
folder address, is learned as composed, not as a template around the id;
- a slug learns how the application spells `&` from the call that shows it,
and held entities that spell it both ways leave no slug rule;
- an id that equals the title's slug is told apart by the entities the type
holds, or by a later create;
- a template a later create answers otherwise is withheld, as a composed
string is.
## Non-goals
- No general value transforms, string templates or script steps: the value
kinds are fixed.
- No saved values in `fill` or `select_option` values.
- No retroactive redaction of earlier unrelated records.
- No multi-factor sign-in.
docs/specs/agent-environments/scenario-saved-values/tasks.md
+15
View File
@@ -0,0 +1,15 @@
# Tasks
- [x] Record the outcome, requirements and non-goals.
- [x] Add saved values, `save_response` and Basic credentials to the scenario schema.
- [x] Validate that saved values are saved before use.
- [x] Save and send values in the Playwright runtime.
- [x] Keep scenarios that save nothing byte-identical in Evidence.
- [x] Warn about credential-named saved values not marked secret.
- [x] Update the skill and docs.
- [x] Add unit, boundary and real-browser checks.
- [x] Run the gates.
- [x] Build prompt-recorded environments with main and with this branch.
- [x] Make the skill's recording example write through the UI.
- [x] Learn the addresses a UI-made create spells, and check templates on later creates.
- [x] Rebuild the branch's recordings with the write-model changes and record every application again.
docs/specs/agent-environments/scenario-saved-values/verification.md
+138
View File
@@ -0,0 +1,138 @@
# Verification
This is Edgar Arakelyan's historical verification record from upstream PR #7162
(head `d39f386d202fe8e98592fb2d9d26e43d8d0fb94f`), retained with attribution.
These multi-application and model evaluations were not rerun during integration.
The independent integration measurements and their limits are recorded in
[saved-values integration verification](../../../verification/browser-saved-values-2026-10-26.md).
Executed on macOS with Node v24.22.1 at `feat/recorder-saved-values` e42362c9 (on main 2b84efe0), and on Ubuntu 24.04
VMs with snap Chromium for the agent recordings.
## Gates
- `npm run typecheck`, `lint:built`, `format:check`, `metadata:check`, `verify:architecture-guards`, `knip:production`,
`lint:typed`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery`, `scan:todos` and `jscpd`: passed.
`knip` reports one unused export, `isToolFamilyId` in `src/contracts/toolFamilies.ts`, which this branch does not
touch and main 2b84efe0 reports too.
- `npx vitest run --project domain --project services --project adapters`: 18,261 passed and 1 failed, "stops Inspector
target authorization after the current stat settles", which fails the same way on main in this environment.
- Browser scenario CLI and MCP boundaries, scenario input diagnostics, cancellation and cleanup, the analysis-view CLI
and MCP boundaries, the environment build, export and serve CLI boundaries, the environment serve, served-writes and
bundle MCP boundaries, tool schema validity, MCP schema validation, and output schema constraints and profile: 17
files, 128 passed.
- The compact MCP catalog must stay under the one-megabyte frame `outputSchemaProfile.test.ts` measures. A first
design with `path_params` on request and goto steps, `json_values`, and separate `username` and `password` unions
added 9,391 bytes on main ff6d96bc and failed the test, which then had 5,731 bytes to spare. Saved values now travel
as `{saved:name}` placeholders, Basic credentials repeat their part union once, and repeated inner descriptions were
dropped: on main 2b84efe0 the catalog takes 1,038,975 bytes and this branch 1,042,639.
## Compatibility
- Scenarios that save nothing record the same Evidence parameters and `scenario_sha256` on main and on this branch: a
UI-only scenario with a goto to a URL holding literal braces, and a request step with a secret query value, a cookie
header, and literal braces in its path and JSON body.
- Main reads a capture that uses saved values: `environment-build --dry-run` of the real-Chrome capture below gives the
same manifest, state, coverage and file plan on main and on this branch. This branch adds the
`credential-named-saved-values` warning.
- Write plans for 109 captures (the 50 published captures, 29 earlier agent recordings and the 30 recorded in the
comparison below), built by main and by this branch: 524 plans and 247 served on each. Three plans differ, all
Grafana folder creates recorded below, whose `url` the branch composes from the uid and the title's slug where main
learned a template or nothing.
## Real Chrome
A local application with a cookie-session UI and an API that takes only Basic auth, recorded with Google Chrome:
- fill a name and click Save with `save_response` on `POST /ui/items`, saving the new id;
- `goto` `/items/{saved:item1}` and wait for the item's page;
- `GET /api/items/{saved:item1}` with Basic credentials whose username comes from local storage and password from a
declared secret;
- `POST /api/tokens`, saving the token both as a secret and, unmarked, as `session_token`;
- `POST /api/items` with the saved token in a header and `{"name": "Child of {saved:item1}", "ref": "{saved:item1}"}`;
- `PATCH /api/items/{saved:child}` and `GET /api/items?parent={saved:item1}`.
All 11 steps completed. The requests went to `/items/101`, `/api/items/101`, `/api/items/108` and `?parent=101`, and the
body sent was `{"name":"Child of 101","ref":101}`. The Evidence parameters keep the placeholders, the password appears
nowhere in the capture, and the token header is recorded as `[REDACTED:page:token.api_token]`. The token's own response
was redacted too in this run; the docs say it may not be, since the recorder reads bodies on their own schedule.
`environment-build` warned about `/parameters/actions/7/save/1`.
## Agent-built environments, main and this branch
Claude Code (`claude-sonnet-5-5`) built an environment of each application from the same prompt, with REA main
ff6d96bc (skill 49) and with this branch (skill 50), on freshly seeded instances. Each bundle was rebuilt by the REA it
was recorded with, and its write tasks run with `fireworks_ai/gpt-oss-120b` and `fireworks_ai/glm-5p3-flash`. Each task
checks its result from the environment's state, including that nothing else changed.
The branch was recorded three times as it changed:
1. With the skill as first written, EspoCRM's agent wrote through request steps with Basic credentials instead of the
UI, and EspoCRM's API answers in a shape the write model withholds (0/3). The skill's recording example created its
object with a request step, which agents copy; it now creates through the UI with `save_response`.
2. With that example, Grafana's agent created folders through the UI, where Grafana generates the uid. REA then learned
the folder's `url` as a template around the uid that kept the first folder's slug, and coverage did not check the
template against the second folder, so served creates answered with wrong addresses (0/2). The write-model fixes
in this PR compose the address from the uid and the title's slug, learn Grafana's `&` spelling, and check templates
on later creates.
3. With both, every application was recorded again.
**The same recordings, before and after the write-model fixes.** Run 2's captures, rebuilt by this branch before and
after the fixes (gpt-oss, GLM):
| Application | Before | After |
| ----------- | ------------- | ------------- |
| Miniflux | 3/3 3/3 | 3/3 3/3 |
| Paperless | 1/3 2/3 | 1/3 1/3 |
| Vikunja | 1/3 1/3 + 0/2 | 1/3 1/3 + 0/2 |
| Linkding | 0/2 0/2 | 0/2 0/2 |
| Homebox | 0/2 0/2 | 0/2 0/2 |
| Grafana | 0/2 0/2 | 2/2 2/2 |
| EspoCRM | 1/3 1/3 | 1/3 1/3 |
Only Grafana's plans change, and its tasks pass. Paperless's plans are identical in both builds; its GLM difference is
the eval model's.
**Fresh recordings, main and the final branch:**
| Application | Write tasks | Main: served, tasks | Branch: served, tasks |
| ----------- | ----------- | ---------------------- | --------------------- |
| Miniflux | 3 | no bundle | 4, 3/3 3/3 |
| Paperless | 3 | 3, 1/3 1/3; 6, 2/3 2/3 | 6, 1/3 2/3 |
| Vikunja | 3 + 2 | 1, 0/3 0/3 + 0/2 0/2 | 3, 1/3 1/3 + 2/2 2/2 |
| Linkding | 2 | 0, 0/2 0/2 | 3, 0/2 0/2 |
| Homebox | 2 | 1, 0/2 0/2 | 5, 0/2 0/2 |
| Grafana | 2 | 2, 2/2 2/2; 3, 2/2 2/2 | 1, 1/2 1/2 |
| EspoCRM | 3 | 2, 1/3 1/3 | 2, 0/3 0/3 |
| Total | 20 | 9, 4/20 4/20 | 24, 8/20 9/20 |
"Served" counts the write tools the final capture serves; tasks are gpt-oss then GLM, and Miniflux's count as failed
without a bundle. Paperless and Grafana were recorded twice on main; totals use the first run.
- **Miniflux:** main's agent stopped and asked for an API key, since the API takes Basic auth. With Basic credentials
the branch served category create, rename and delete in all three runs, and every category task passed with both
models.
- **Vikunja:** label edits and deletes are served and pass in two of the three branch runs, and task deletes in the
last two.
- **Saved ids:** every branch agent saved ids, with `save_response` on UI writes, and read objects back by them. On main
no step can carry a value to a later one, so agents chose their own ids or read lists back instead.
- **Paperless:** the two main runs differ as much as main and the branch do.
- **Linkding and Homebox** serve more writes, but their create tasks still fail. Linkding's served create requires every
member its recorded calls sent (`description`, `notes`, `unread`, `shared`, `tag_names`), so a task that sends a URL
and a title is refused.
- **Grafana:** the folder create is served with the right address and its task passes. The rename is withheld:
Grafana lists folders sorted by title, the recording renamed a folder to a title that sorts first, and REA keeps an
updated entity where it was in a list. Main's REA reads this recording the same way, and serves its create with the
template that keeps the first folder's slug, so the create task would fail too. Main's two recordings passed because
their titles kept their order.
- **EspoCRM:** the recording started with a contact an earlier attempt in the same session created and left, so the
environment holds it as seeded data and every task's "nothing else changed" check fails. The skill now asks agents to
delete what an earlier attempt created before recording again; that change was not measured.
- Agents spent $9.98 and 3,149 seconds in total on the fresh branch run, against $7.45 and 2,235 on main, a third of it
Homebox's.
## Not verified
- One fresh recording per application and version, two for Paperless and Grafana on main; agent runs vary.
- Lists sorted by a field that an update changes, and the skill's cleanup of earlier attempts.
- Multi-factor sign-in, and saved values in `fill` or `select_option` steps, which this change does not add.
docs/verification/browser-saved-values-2026-10-26.md
+271
View File
@@ -0,0 +1,271 @@
# Browser saved values: integration verification
## Pinned scope
Executed on Linux x64, Node 24.22.1 and Chromium through `/usr/bin/chromium`.
The primary comparison uses **Playwright 1.64.1 and the same integration tree
for both workflows**. Local verification commit `dd343d3b1092d697772f1447d94179d23bcc5cf2`
contains GitHub merge `1a3ef66c8e25f25fdd06053bb33255f10f65c5b6`
(main `030fa886f94e894e34ed2539e9d8dedff19c3327` plus PR head
`ed110935f520d56f6eaf5710ad2f9a58340492b4`), followed by response-scope fix
`2c5f9cc14b3395ef4a99071222e3feaf9501bb62`. The local verification commit is
not a published release. Recreate its tree from the public main/PR inputs,
then apply that fix. Final reconciliation adds `0d89c65409d28fec269cfdbb3ee62e0f153819fa`
(local current-dependency tree `221ba53af`) for query spelling and refused-slug
source constraints; the primary timing pair predates these two corrections.
Historical comparisons against pre-feature main
`6d5532a204b338a6c770fcbae61e0d55bda70793` used Playwright 1.58.2 and remain
below. They do not isolate the dependency upgrade. The implementation integrates
Edgar Arakelyan's #7162; upstream `d39f386d202fe8e98592fb2d9d26e43d8d0fb94f`
and subsequent fixes through `9f5191ab0` were compared. Its query redaction and refused-slug source
pinning were incorporated. Existing heap-derived URL/JSON/envelope admission
remains the owner of expansion bounds, rather than adding the upstream fixed
fill ceiling. Original multi-app/model
results are [attributed historical evidence](../specs/agent-environments/scenario-saved-values/verification.md),
not executions performed here.
## Real application and correctness oracles
PocketBase **0.40.6** runs on an owned loopback instance. Its official Linux amd64
archive is 12,824,710 bytes, SHA-256
`0e1f609158ed945f809f8a02ae012aa0427129c66ebb63c1aee76f2c0309373f`.
The [owned frontend](https://github.com/morluto/rea/blob/938052704/experiments/analysis-efficiency/fixtures/browser-saved-values/index.html)
uses actual superuser authentication and record APIs. It is a small test
frontend rather than PocketBase's entire administrative application.
Collection `rea_items` has required text `title` and numeric `quantity`, public
list/view, and superuser-only create/update/delete rules.
The completed task signs in, creates a record, reuses its generated ID and
numeric quantity, PATCHes it, GETs it with a query, and navigates to its API URL.
Independent PocketBase GETs verify identity, final title and numeric quantity;
retained request bytes verify a JSON number. Every declared action must finish.
The selected login response must actually occur once; no missing response can
pass the redaction oracle.
- Current integration public CLI: all **eight steps** completed, eight retained
JSON bodies, zero plaintext marked token or declared password. **4.351 s elapsed**,
**4.832 s CPU**, **1,381,789,696 B** sampled process-family peak.
- Current integration MCP: two equivalent tasks completed in both legacy and
saved-value modes. Legacy uses a retained event page and another capture;
saved bindings finish the same analyst task in one capture.
- Both current legacy runs actually observed the login body before the later
page-value read. That earlier unmarked token remained present, while later
observations used redaction. The mixed-scope unit cases additionally establish
that an unrelated page/Basic value remains prospective when response secrets
are also learned.
- A real negative control on the imported implementation retained the marked
authentication token despite completing its actions. The integrated source
response and retained-event checks reject that outcome.
## Privacy, provenance and resource boundaries
The public MCP source probe verifies:
1. Source and later JSON bodies with escaped strings and `1e3` become valid
redacted JSON while preserving unmarked `1e-400` and whitespace exactly.
2. Metadata-only capture redacts the selected source URL without retaining
network-content records. Unknown source values withhold the affected source
fields and expose a limitation.
3. A declared whole JSON string seeded into storage and POSTed to PocketBase is
withheld if a safe whole-body replacement cannot preserve JSON. HTTP **403**
and an unchanged integer record count independently confirm the outcome.
4. The frontend consumes the selected response, logs and visibly renders its
token, and immediately POSTs it in a URL/header/body. Retained console and
network data are protected. All three requested screenshots are `missing`,
with screenshot completeness ineligible for equality.
5. A separate nonsecret scenario captures a real PNG and validates its signature.
This expanded probe passed in **8.417 s elapsed**, **9.567 s CPU**,
**1,680,818,176 B** sampled family peak. A focused public CLI run of the shared
CI browser fixture also passed: three withheld sensitive screenshots, all other
selected artifacts captured, and explicit screenshot/network-cutoff gaps.
It took **5.956 s elapsed**, **6.868 s CPU**, **1,495,924,736 B** family peak;
that fixture server is inside the measured family.
The source parser owns strict UTF-8, exact selected numbers, pointer lookup and
source-token redaction. UI whole-body saving requires a positive known
identity-encoded size before reading; compressed/cache-unknown responses are
refused. The original author's fallback is not enabled. Two initial cached UI
probes were correctly refused; the owned fixture uses `cache: "no-store"` for
the known-size case.
Response-marked values can backfill retained source/follow-up event text.
Page-derived cookie/storage values and constructed Basic credentials retain
their existing prospective-only policy. A session-local response projection
borrows admitted compiled patterns/byte owners and is invalidated on admissions
or response classification. Promoting an already held value to an explicit
response secret preserves its first marker. Replacement work and retained output
are admitted before allocation; omissions and cleanup retry are accounted once.
UI matching is observational. Multiple observed matches fail without adopting
an ambiguous binding; a single observed match cannot prove UI causality or
exclude a later poll. Returned Playwright body bytes also do not establish
original-wire provenance. These limitations remain in the contract/Evidence.
Request JSON, saved URLs and the full request envelope are admitted before
expansion/dispatch using Node's string ceiling and a conservative V8-headroom
allowance. Repeated saved/header facts share request-local size/validation work.
Four cases reject 1,000 references to a 3,000,000-character value without dispatch;
those inputs were deliberately not run on unbounded code. The first repeated
header guard took 4.042 s in its source test; the final case no longer appears
in slow-test output (its final individual duration was not separately measured).
Raw pixels are not text-redactable. The secret owner establishes screenshot
withholding before execution for planned sensitive response/page/Basic values,
and whenever sensitive values are already known. The artifact producer then
returns `missing` without requesting pixels. No separate image-cleanup state or
publication hook remains.
## Completed-task measurements
Both columns use the current integration tree, provider and catalog. Each run
creates a fresh MCP server/browser context and completes a first and subsequent
task. The browser has already loaded the fixture; “first” is not a cold browser
or OS cache. One run per condition is an observation, not a general speed claim.
| Measurement | Legacy workflow | Saved values |
| -------------------------------------- | -----------------------: | -----------------------: |
| Task calls, each task | 3 | 1 |
| First task elapsed | 6.191 s | 1.879 s |
| Warm task elapsed | 1.120 s | 0.872 s |
| Whole verifier elapsed | 13.413 s | 8.092 s |
| Whole verifier CPU | 15.347 s | 9.387 s |
| Sampled family peak RSS | 2,018,435,072 B | 1,689,305,088 B |
| Input catalog + arguments/text replies | 34,885 estimated tokens | 25,881 estimated tokens |
| Full SDK catalog + arguments/results | 370,936 estimated tokens | 353,768 estimated tokens |
`o200k_base` BPE estimates serialize compact JSON and are not model billing.
The model projection counts all **53** tool names/descriptions/input schemas
(**18,579 tokens** in both columns), followed by the complete array of named
arguments and text-content results. The full SDK estimate counts the complete
catalog plus the array of full results, including output schemas and structured
results. Arrays are tokenized jointly rather than summing independently tokenized
cells. Both estimates include two extra correctness-proof exports: eight total
calls before and four after. No model ran; reasoning, retries, JSON-RPC framing
and initialization traffic are unmeasured.
Whole-verifier cost includes discovery, browser setup, proof/oracle work and
cleanup, distinct from timed task calls. PocketBase runs externally: its CPU
and memory are **unmeasured**. REA/Chromium families use two CPUs, a 3 GiB Node
heap and an 8 GiB family RSS guard. RSS is summed every 100 ms, may double-count
shared pages and miss short peaks. No guard stopped the completed runs.
Historical pre-feature comparisons (Playwright 1.58.2) remain observations:
| Candidate | First task before→after s | Warm before→after s | Whole elapsed before→after s | CPU before→after s | Peak GB before→after |
| ----------- | ------------------------: | ------------------: | ---------------------------: | -----------------: | -------------------: |
| `355e2dcb8` | 7.703→3.531 | 1.390→1.605 | 14.990→17.198 | 15.735→15.679 | 2.080→1.636 |
| `dd3085bf5` | 5.145→1.621 | 1.020→0.762 | 11.723→7.653 | 13.444→8.683 | 2.028→1.638 |
| `439a389fe` | 16.121→4.429 | 2.944→1.738 | 34.296→20.053 | 32.166→19.053 | 1.996→1.646 |
| `e7acc5d85` | 10.778→2.627 | 1.467→1.129 | 22.310→12.661 | 26.148→14.759 | 2.103→1.632 |
| `43aaaabbe` | 7.751→3.220 | 1.210→0.996 | 15.527→11.301 | 18.047→12.777 | 2.064→1.628 |
| `854b7a2c0` | 6.965→1.923 | 1.100→0.874 | 15.657→9.213 | 17.222→10.753 | 2.053→1.650 |
| `938052704` | 5.983→1.720 | 1.040→0.787 | 13.103→8.146 | 15.194→9.497 | 2.033→1.635 |
The first candidate was slower for warm/whole elapsed. Later differences also
reflect host/cache variance. Historical model estimates include 34,899→25,897
at `355e2dcb8`, 34,831→25,902 at `e7acc5d85`, 34,816→25,866 at `43aaaabbe`,
34,742→25,845 at `854b7a2c0`, and 34,820→25,897 at `938052704`. These are not
summed as independent model usage.
## Executed checks and negative controls
- Final feature branch: **560 tests in 12 source suites**, type/lint and build passed.
- Current integration/Playwright 1.64.1: **561 tests in the same 12 suites**,
build and `check:fast` passed; the additional test comes from main.
- Twenty explicit schema/profile tests passed earlier; all 41 distinct cases in
the broader five-suite run passed across its run and isolated rerun. One case
timed out at 30 s under two-CPU contention, then passed alone in 17.36 s.
An older-Node attempt failed before tests. Final real SDK calls also validate
the named tool definitions and advertised results.
- Negative controls fail before their fixes: decoded bounds/accounting/numeric/
slug cases; seven source-format/UI ambiguity cases; three source-publication
cases; two whole-JSON cases; two backslash substitutions plus immediate
follow-up; five final event/path cases; and ten screenshot cases. The
screenshot nonsecret control passes on the old revision.
- Final reconciliation re-ran the eight-step real CLI task and all four privacy
probes plus the positive PNG case on `221ba53af` successfully. A further
two-step public CLI case verifies the parsed query spelling of `ab'cd~ef`
is redacted. Together these checks used 16.550 s elapsed, 18.581 s CPU and
1,675,153,408 B sampled family peak; this is qualification, not a new timing pair.
- The final upstream query/slug regressions fail three cases on `34dc2c1c4`.
The query spelling lower-bound calculation also includes the new encoding
before allocating it.
- Three prospective-page/Basic regressions fail on `ed110935f`; final tests also
cover promotion, scope-cache invalidation and preservation of the first marker.
- The prior CI's own event-budget assertion was fixed with JSON admission order
and the whole-body guard. It was not bypassed. Two invalid newly added header
fixtures were corrected to canonical shapes before the final source run.
- A focused CI-fixture harness initially closed its server too early. Cleanup
order was corrected and the real CLI rerun passed. Failed receipts remain.
Initial CI/main controls reproduced a load-sensitive prewarm failure and two
SAZ partial-observation assertions on their exact parents. The prewarm case
passes alone on later main and is not claimed as a stable current failure.
Subsequent CI failures comprise those two SAZ assertions and Java diagnostic
`keeps empty Java bitness unresolved`; all three independently reproduce on
actual CI main parent `c6893d1ac6a9ccb8a11d64b4e9ba1b57e79cbfd8` with its isolated
1.64.1 locked dependencies. The first preparation refused stale shared 1.58.2
dependencies before any tests; the shared installation was preserved.
On head `34dc2c1c4`, all 18 distinct CI failures (artifact/inventory, snapshot,
cleanup, Apple-resource coverage and the two SAZ assertions) reproduce on exact
CI main parent `29ebd6740550bf3989129c39c65f17e5c61e3dd2`. The additional
control run also timed out two retained-ZIP cases and emitted two connection
closures; these were control-only incomplete results, not passing checks.
All fifteen actionable review threads were fixed and resolved. A further
final-head Codex review request was refused by the repository usage limit; no
fresh successful automated review is claimed.
Cancelled lanes remain incomplete, including cancelled shards that emitted
passing summaries. CI counts from overlapping revisions are not added together.
## Reproducing and limitations
Use an isolated worktree with the pinned Node/npm toolchain. For the primary
comparison, recreate the public main/PR integration inputs above, apply the
response-scope fix, install its lockfile, build once, and use **that same checkout**
as `--repository` for both modes below. Historical pre-feature measurements used
separate pinned checkouts.
On an owned loopback PocketBase instance, create the collection/rules described
above and a temporary superuser. Serve the committed `index.html`,
`whole-json.html` and `immediate-followup.html` from
`experiments/analysis-efficiency/fixtures/browser-saved-values/`. Store only
`{"email":"...","password":"..."}` in a private credentials file, mode 0600;
keep it and full retained Evidence out of Git.
Serve `escaped-values.json` with these exact minified bytes (unmarked spelling
is part of the oracle):
```text
{"token":"\u0073ource-token","code":1e3,"unknown":1e-400}
```
Run the committed driver with fresh private output directories:
```bash
node experiments/analysis-efficiency/browser-saved-values.mjs \
--repository /path/to/integration --mode before --output /private/before \
--origin http://127.0.0.1:18762 --credentials /private/credentials.json \
--browser /usr/bin/chromium
node experiments/analysis-efficiency/browser-saved-values.mjs \
--repository /path/to/integration --mode after --output /private/after \
--origin http://127.0.0.1:18762 --credentials /private/credentials.json \
--browser /usr/bin/chromium
node experiments/analysis-efficiency/browser-saved-values.mjs \
--repository /path/to/integration --mode after --source-only \
--output /private/privacy --origin http://127.0.0.1:18762 \
--credentials /private/credentials.json --browser /usr/bin/chromium
```
Run these under a process-family CPU/RSS supervisor for comparable resource
measurements. The driver owns its MCP client and browser/profile cleanup, checks
payload outcomes, records arguments/results/catalog, and exports retained parent
Evidence. Current legacy runs observed pre-read login bodies in both tasks; if
that ordering is not observed, the receipt says so rather than inventing timing.
Real-provider scope is Linux Chromium and the owned PocketBase/frontend plus
shared CI fixture. Compressed/cache-unknown UI saving is unsupported. Other
platforms, real Basic-auth servers, service-worker integrations and historical
multi-app/model evaluations were not rerun here. No model task success or actual
model-token billing is claimed.
docs/web-network-captures.md
+27 -5
View File
@@ -771,17 +771,39 @@ sorted stays sorted, unless a call stored its strings out of order as sent),
and a field neither rules holds one value. Members either call sent are accepted, only those every call sent are
required, and an echoed member a call left out takes the value that call stored
(`body.defaults`). A clock rule widens to `trimmed` only when no held entity or
earlier call showed that clock at a fixed width. A later create that disagrees
leaves the plan as it was. A create whose observed reference differs from the
earlier call showed that clock at a fixed width, and a slug to spelling `&` as
the word (`ampersand: "and"`) only when no call the plan accepted nor entity the
type held dropped one. A later
create that disagrees leaves the plan as it was. A field the first create left
unexplained, such as an address whose two parts were the same text, takes the
`composed` rule a later create learns for it when that rule spells the first
create's value too, and the first create's body contract is learned again with
it. A `composed` string or `template` a later create's answer shows otherwise
or leaves out is `unexplained-change`. A create whose observed reference
differs from the
entity the plan rebuilds, whether or not it sent the reference member, is
`unexplained-change`. So is a create that left out an echoed member yet shows a
value different from the learned default.
A `composed` string is literal text and parts read from the fields other rules
write: a field's value, its slug, or a clock's calendar digits
(`YYYYMMDDhhmmss`, `YYYYMMDD`, `YYYY-MM-DD` or `YYYY/MM/DD`, in the clock's
zone). A create composes a string only when at least one other entity of the
type spells it with the same parts and none spells it otherwise, and never when
one text could come from two sources serving could tell apart. A clock format
zone). A slug drops `&` unless the application spells it as the word, as Go's
gosimple/slug and JavaScript's slugify do (`ampersand: "and"`, on a `slug` rule
or part). When a create slugs no `&`, the entities the type holds show which,
and no `slug` rule is learned when they spell it both ways; a create whose slug
spells `&` otherwise than they do, or alongside one whose
slug neither spelling gives, has no `slug` rule; such a held slug also keeps a
later create from widening the spelling. An update learns the spelling the same way: a later update whose slug spells
`&` as the word refines a rule an earlier update learned from a name without
`&`, unless an accepted call or a held entity dropped it, and every earlier
update must still replay. An update whose slug spells no `&` takes the spelling the
entity's earlier value and the type's other entities show, as a create does,
and learns no slug rule when they spell it both ways. A create composes a string only when at least one other entity of the
type spells it with the same parts and none spells it otherwise. Where one text
could come from two sources serving could tell apart, such as an id that equals
the title's slug, the entities the type holds must leave exactly one reading. A
`composed` string that reads a field besides the generated value a `template`
would rebuild it around replaces the template. A clock format
is `trimmed` when the field's values differ only in how many fraction digits
they show and none ends a fraction in a zero, as Go spells nanoseconds: its
`fraction` is the most it shows, and serving drops trailing zeros and the point
experiments/analysis-efficiency/browser-saved-values.mjs
+713
View File
@@ -0,0 +1,713 @@
/** Compare complete real PocketBase create/update/read tasks through public MCP. */
import assert from "node:assert/strict";
import { readFile, writeFile, mkdir, mkdtemp, rm } from "node:fs/promises";
import { join, resolve } from "node:path";
import { tmpdir } from "node:os";
import { createRequire } from "node:module";
import { pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { performance } from "node:perf_hooks";
import { createVerifierCommands } from "../../scripts/lib/verify-commands.mjs";
import { withVerifierCleanup } from "../../scripts/lib/verifier-cleanup.mjs";
const { values } = parseArgs({
options: {
repository: { type: "string" },
mode: { type: "string" },
output: { type: "string" },
origin: { type: "string" },
credentials: { type: "string" },
browser: { type: "string" },
"source-only": { type: "boolean", default: false },
},
});
assert.ok(
values.repository &&
values.output &&
values.origin &&
values.credentials &&
values.browser,
);
assert.ok(["before", "after"].includes(values.mode));
const repo = resolve(values.repository),
output = resolve(values.output),
origin = new URL(values.origin).origin;
assert.equal(
new URL(origin).hostname,
"127.0.0.1",
"Use an owned loopback fixture",
);
await mkdir(output, { mode: 0o700 });
const credentials = JSON.parse(await readFile(values.credentials, "utf8"));
const require = createRequire(join(repo, "package.json"));
const { chromium } = require("playwright-core");
const { Client } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client"))
);
const { StdioClientTransport } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio"))
);
const commands = await createVerifierCommands();
const profile = await mkdtemp(join(tmpdir(), "rea-saved-values-"));
const env = {
PATH: process.env.PATH,
NODE_OPTIONS: "--max-old-space-size=3072",
UV_THREADPOOL_SIZE: "2",
REA_LOG_LEVEL: "silent",
REA_MCP_TOOL_FAMILIES: "browser,application,session",
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
REA_TEST_EMAIL: credentials.email,
REA_TEST_PASSWORD: credentials.password,
REA_TEST_WHOLE_JSON: JSON.stringify({ token: "private-token-729d" }),
};
const calls = [];
const runs = [];
let catalog;
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(repo, "scripts/rea.mjs"), "mcp"],
cwd: repo,
env,
stderr: "pipe",
});
transport.stderr?.resume();
const client = new Client({ name: "rea-saved-values-real-app", version: "1" });
commands.adopt(client, "saved-values MCP client");
const tokenHeader = (saved) => ({
name: "Authorization",
value: saved
? { source: "saved", name: "token" }
: { source: "local_storage", key: "rea_auth", json_pointer: "/token" },
});
const decodeBodies = (value) => {
const bodies = [];
const visit = (x) => {
if (x && typeof x === "object") {
if (x.encoding === "base64" && typeof x.content === "string") {
try {
bodies.push(
JSON.parse(Buffer.from(x.content, "base64").toString("utf8")),
);
} catch {}
}
for (const v of Object.values(x)) visit(v);
}
};
visit(value);
return bodies;
};
async function invoke(name, args) {
const defs = catalog.tools.filter((t) => t.name === name);
assert.equal(defs.length, 1);
const start = performance.now();
const result = await client.callTool(
{ name, arguments: args },
{ toolDefinition: defs[0], timeout: 90000 },
);
calls.push({
name,
arguments: args,
result,
elapsed_ms: performance.now() - start,
});
assert.notEqual(
result.isError,
true,
`${name} failed; retain the local receipt`,
);
return result.structuredContent;
}
const loginActions = (saved) => [
{
step_id: "email",
action: "fill",
locator: { kind: "css", selector: "#email" },
value: { source: "secret", secret_id: "email" },
},
{
step_id: "password",
action: "fill",
locator: { kind: "css", selector: "#password" },
value: { source: "secret", secret_id: "password" },
},
{
step_id: "login",
action: "click",
locator: { kind: "css", selector: "#login" },
...(saved
? {
save_response: {
method: "POST",
path: "/api/collections/_superusers/auth-with-password",
save: [{ name: "token", json_pointer: "/token", secret: true }],
},
}
: {}),
timeout_ms: 10000,
},
{
step_id: "ready",
action: "wait_for",
locator: { kind: "css", selector: '#result[data-ready="yes"]' },
state: "visible",
timeout_ms: 10000,
},
];
const edits = (id, saved) => [
{
step_id: "update",
action: "request",
method: "PATCH",
path: `/api/collections/rea_items/records/${id}`,
headers: [tokenHeader(saved)],
json: { title: `record-${id}`, quantity: saved ? "{saved:quantity}" : 11 },
expected_status: [200],
},
{
step_id: "read",
action: "request",
method: "GET",
path: `/api/collections/rea_items/records/${id}?at=${id}`,
headers: [tokenHeader(saved)],
expected_status: [200],
},
{
step_id: "open",
action: "goto",
destination: { url: `${origin}/api/collections/rea_items/records/${id}` },
wait_until: "load",
},
];
const captureOptions = {
after_each_step: ["url"],
at_end: ["url", "accessibility"],
events: ["network"],
network: { request_body: true, response_body: true, header_values: true },
};
try {
const context = await chromium.launchPersistentContext(profile, {
executablePath: resolve(values.browser),
headless: true,
args: ["--remote-debugging-port=0", "--no-sandbox"],
});
commands.adopt(context, "owned saved-values Chromium context");
const page = context.pages()[0];
await page.goto(origin, { waitUntil: "networkidle" });
const port = (
await readFile(join(profile, "DevToolsActivePort"), "utf8")
).split("\n")[0];
const endpoint = `http://127.0.0.1:${port}`;
const targets = await (await fetch(endpoint + "/json/list")).json();
const target = targets.find(
(t) => t.type === "page" && t.url === origin + "/",
);
assert.ok(target);
const browser = {
mode: "connect",
cdp_endpoint: endpoint,
target_id: target.id,
};
const declared = [
{ secret_id: "email", environment_variable: "REA_TEST_EMAIL" },
{ secret_id: "password", environment_variable: "REA_TEST_PASSWORD" },
];
const scenario = (actions, secrets = declared) => ({
browser,
start_url: { url: origin + "/" },
secrets,
actions,
capture: captureOptions,
detail: "summary",
});
await client.connect(transport);
catalog = await client.listTools();
await writeFile(join(output, "catalog.json"), JSON.stringify(catalog));
for (const temperature of values["source-only"] ? [] : ["cold", "warm"]) {
const saved = values.mode === "after";
const start = performance.now(),
firstCall = calls.length;
const create = {
step_id: "create",
action: "request",
method: "POST",
path: "/api/collections/rea_items/records",
headers: [tokenHeader(saved)],
json: { title: `created-${temperature}`, quantity: 11 },
expected_status: [200, 201],
...(saved
? {
save: [
{ name: "record_id", json_pointer: "/id" },
{ name: "quantity", json_pointer: "/quantity" },
],
}
: {}),
};
const firstActions = [...loginActions(saved), create];
if (saved) firstActions.push(...edits("{saved:record_id}", true));
const first = await invoke(
"capture_browser_scenario",
scenario(firstActions),
);
const firstSummary = first.normalized_result;
assert.ok(
firstSummary.summary.steps.every((step) => step.status === "completed"),
"Initial capture must complete",
);
let final = first;
let id;
let legacyPreReadObserved;
if (!saved) {
const selected = await invoke("inspect_analysis_view", {
source: {
kind: "retained-evidence",
evidence_id: firstSummary.parent_evidence_id,
},
view: {
kind: "browser-scenario",
collection: "events",
step_indices: [firstActions.length],
event_kinds: ["network-content"],
limit: 16,
},
});
assert.equal(selected.normalized_result.coverage.exhausted, true);
const records = decodeBodies(selected.normalized_result).filter(
(b) => b?.collectionName === "rea_items" && typeof b.id === "string",
);
assert.equal(records.length, 1);
id = records[0].id;
final = await invoke(
"capture_browser_scenario",
scenario(edits(id, false), []),
);
}
assert.ok(
final.normalized_result.summary.steps.every(
(step) => step.status === "completed",
),
"Completed task must not hide failed actions",
);
const completedMs = performance.now() - start,
taskCalls = calls.length - firstCall;
const path = join(output, `${temperature}-evidence.json`);
await invoke("export_evidence_bundle", { path });
const bundle = JSON.parse(await readFile(path, "utf8"));
const parent = bundle.records.find(
(r) => r.evidence_id === final.normalized_result.parent_evidence_id,
);
assert.ok(parent);
if (!saved) {
const legacy = bundle.records.find(
(record) => record.evidence_id === firstSummary.parent_evidence_id,
);
assert.ok(legacy);
const login = decodeBodies(legacy.normalized_result).filter(
(body) => typeof body?.token === "string",
);
assert.equal(login.length, 1);
const preReadLogin = decodeBodies({
events: legacy.normalized_result.events.items.filter(
(event) => event.step_index < firstActions.length,
),
}).filter((body) => typeof body?.token === "string");
legacyPreReadObserved = preReadLogin.length === 1;
if (legacyPreReadObserved) {
assert.ok(
!preReadLogin[0].token.startsWith("[REDACTED:"),
"An observed pre-read login response must preserve its unmarked token",
);
const later = legacy.normalized_result.events.items.filter(
(event) => event.step_index >= firstActions.length,
);
assert.ok(
!JSON.stringify({
events: later,
bodies: decodeBodies({ events: later }),
}).includes(preReadLogin[0].token),
"Post-read facts must use the current page-value policy",
);
}
}
const recordBodies = decodeBodies(parent.normalized_result).filter(
(b) => b?.collectionName === "rea_items" && typeof b.id === "string",
);
assert.ok(recordBodies.length >= 2);
if (saved) id = recordBodies[0].id;
assert.ok(recordBodies.every((b) => b.id === id));
assert.ok(
recordBodies.some((b) => b.title === `record-${id}` && b.quantity === 11),
);
const read = await fetch(
`${origin}/api/collections/rea_items/records/${id}`,
);
assert.equal(read.status, 200);
const actual = await read.json();
assert.equal(actual.title, `record-${id}`);
assert.equal(actual.quantity, 11);
const requestBodies = decodeBodies(parent.normalized_result).filter(
(b) => b?.title === `record-${id}` && !Object.hasOwn(b, "collectionName"),
);
assert.ok(requestBodies.length > 0);
assert.ok(requestBodies.every((b) => typeof b.quantity === "number"));
if (saved) {
const loginBodies = decodeBodies(parent.normalized_result).filter(
(b) => typeof b?.token === "string",
);
assert.equal(loginBodies.length, 1);
assert.ok(loginBodies[0].token.startsWith("[REDACTED:"));
}
runs.push({
temperature,
legacy_pre_read_observed: legacyPreReadObserved,
elapsed_ms: completedMs,
task_calls: taskCalls,
evidence_id: parent.evidence_id,
record_id: id,
quantity_type: typeof actual.quantity,
});
}
if (values["source-only"]) {
assert.equal(values.mode, "after");
const sourceScenario = scenario(
[
{
step_id: "source",
action: "request",
method: "GET",
path: "/escaped-values.json?credential=source-token",
save: [
{ name: "token", json_pointer: "/token", secret: true },
{ name: "code", json_pointer: "/code", secret: true },
],
},
{
step_id: "later",
action: "request",
method: "GET",
path: "/escaped-values.json",
},
],
[],
);
const captured = await invoke("capture_browser_scenario", sourceScenario);
assert.ok(
captured.normalized_result.summary.steps.every(
(step) => step.status === "completed",
),
);
const path = join(output, "source-evidence.json");
await invoke("export_evidence_bundle", { path });
const bundle = JSON.parse(await readFile(path, "utf8"));
const parent = bundle.records.find(
(record) =>
record.evidence_id === captured.normalized_result.parent_evidence_id,
);
assert.ok(parent);
const sourceRequests = parent.normalized_result.events.items.filter(
(event) =>
event.kind === "request" &&
event.step_index === 1 &&
event.url.url.includes("/escaped-values.json"),
);
assert.equal(sourceRequests.length, 1);
const transaction = sourceRequests[0].transaction_id;
const sourceEvents = parent.normalized_result.events.items.filter(
(event) => event.transaction_id === transaction,
);
assert.ok(sourceEvents.some((event) => event.kind === "response"));
assert.ok(
!JSON.stringify(sourceEvents).includes("source-token"),
"Source transaction metadata must not retain the value learned from its response",
);
const bodies = decodeBodies(parent.normalized_result).filter(
(body) =>
Object.hasOwn(body ?? {}, "token") && Object.hasOwn(body, "code"),
);
assert.equal(
bodies.length,
2,
"Both actual source and later response must be observed",
);
for (const body of bodies) {
assert.equal(body.token, "[REDACTED:page:source.token]");
assert.equal(body.code, "[REDACTED:page:source.code]");
}
const expected = Buffer.from(
'{"token":"[REDACTED:page:source.token]","code":"[REDACTED:page:source.code]","unknown":1e-400}',
).toString("base64");
assert.equal(
JSON.stringify(parent.normalized_result).split(JSON.stringify(expected))
.length - 1,
2,
"Both source bodies must preserve the unmarked numeric spelling exactly",
);
const metadata = await invoke("capture_browser_scenario", {
...sourceScenario,
capture: {
...sourceScenario.capture,
network: {
request_body: false,
response_body: false,
header_values: false,
},
},
});
assert.ok(
metadata.normalized_result.summary.steps.every(
(step) => step.status === "completed",
),
);
const metadataPath = join(output, "metadata-evidence.json");
await invoke("export_evidence_bundle", { path: metadataPath });
const metadataBundle = JSON.parse(await readFile(metadataPath, "utf8"));
const metadataParent = metadataBundle.records.find(
(record) =>
record.evidence_id === metadata.normalized_result.parent_evidence_id,
);
assert.ok(metadataParent);
const metadataRequest = metadataParent.normalized_result.events.items.find(
(event) =>
event.kind === "request" &&
event.step_index === 1 &&
event.url.url.includes("/escaped-values.json"),
);
assert.ok(metadataRequest);
assert.equal(metadataRequest.url.origin, origin);
assert.ok(metadataRequest.url.url.includes("/escaped-values.json"));
assert.ok(!metadataRequest.url.url.includes("source-token"));
assert.ok(
!metadataParent.normalized_result.events.items.some(
(event) => event.kind === "network-content",
),
);
const countUrl = `${origin}/api/collections/rea_items/records?perPage=1`;
const beforeCount = (await (await fetch(countUrl)).json()).totalItems;
assert.ok(Number.isSafeInteger(beforeCount));
const whole = await invoke("capture_browser_scenario", {
...sourceScenario,
start_url: { url: origin + "/whole-json.html" },
secrets: [
{
secret_id: "whole_json",
environment_variable: "REA_TEST_WHOLE_JSON",
},
],
storage: {
local_storage: [
{
origin,
entries: [
{
name: "whole_json",
value: { source: "secret", secret_id: "whole_json" },
},
],
},
],
},
actions: [
{
step_id: "settled",
action: "wait_for",
locator: { kind: "css", selector: '#result[data-ready="yes"]' },
state: "visible",
timeout_ms: 10000,
},
],
});
assert.ok(
whole.normalized_result.summary.steps.every(
(step) => step.status === "completed",
),
);
const wholePath = join(output, "whole-json-evidence.json");
await invoke("export_evidence_bundle", { path: wholePath });
const wholeBundle = JSON.parse(await readFile(wholePath, "utf8"));
const wholeParent = wholeBundle.records.find(
(record) =>
record.evidence_id === whole.normalized_result.parent_evidence_id,
);
assert.ok(wholeParent);
const post = wholeParent.normalized_result.events.items.find(
(event) => event.kind === "request" && event.method === "POST",
);
assert.ok(post);
const postEvents = wholeParent.normalized_result.events.items.filter(
(event) => event.transaction_id === post.transaction_id,
);
assert.ok(
postEvents.some(
(event) =>
event.kind === "network-content" &&
event.phase === "request" &&
event.body.state === "unavailable",
),
);
const postResponse = postEvents.find((event) => event.kind === "response");
assert.ok(postResponse && [400, 401, 403].includes(postResponse.status));
assert.ok(!JSON.stringify(wholeParent).includes("private-token-729d"));
assert.equal(
(await (await fetch(countUrl)).json()).totalItems,
beforeCount,
);
const followup = await invoke("capture_browser_scenario", {
...sourceScenario,
start_url: { url: origin + "/immediate-followup.html" },
capture: {
...sourceScenario.capture,
events: [...sourceScenario.capture.events, "console"],
after_each_step: [
...sourceScenario.capture.after_each_step,
"screenshot",
],
at_end: [...sourceScenario.capture.at_end, "screenshot"],
},
actions: [
{
step_id: "exchange",
action: "click",
locator: { kind: "css", selector: "#run" },
save_response: {
method: "GET",
path: "/escaped-values.json",
save: [{ name: "token", json_pointer: "/token", secret: true }],
},
timeout_ms: 10000,
},
{
step_id: "settled",
action: "wait_for",
locator: { kind: "css", selector: '#result[data-ready="yes"]' },
state: "visible",
timeout_ms: 10000,
},
],
});
const followupPath = join(output, "followup-evidence.json");
await invoke("export_evidence_bundle", { path: followupPath });
const followupBundle = JSON.parse(await readFile(followupPath, "utf8"));
const followupParent = followupBundle.records.find(
(record) =>
record.evidence_id === followup.normalized_result.parent_evidence_id,
);
assert.ok(followupParent);
assert.equal(await page.locator("#sensitive").innerText(), "source-token");
assert.equal(followupParent.normalized_result.steps.length, 3);
assert.ok(
followupParent.normalized_result.steps.every(
(step) =>
step.artifacts.screenshot.state === "missing" &&
step.completeness.missing_sections.includes("screenshot"),
),
);
assert.ok(
followupParent.normalized_result.completeness.missing_sections.includes(
"screenshot",
),
);
const consoleEvents = followupParent.normalized_result.events.items.filter(
(event) => event.kind === "console" && event.text.includes("REDACTED"),
);
assert.equal(consoleEvents.length, 1);
assert.ok(
!JSON.stringify(followupParent.normalized_result.events.items).includes(
"source-token",
),
);
assert.ok(
followup.normalized_result.summary.steps.every(
(step) => step.status === "completed",
),
);
const followupPost = followupParent.normalized_result.events.items.find(
(event) => event.kind === "request" && event.method === "POST",
);
assert.ok(followupPost);
const followupEvents = followupParent.normalized_result.events.items.filter(
(event) => event.transaction_id === followupPost.transaction_id,
);
assert.ok(!JSON.stringify(followupEvents).includes("source-token"));
const followupBodies = decodeBodies({ events: followupEvents });
assert.ok(
followupBodies.some(
(body) => body?.token === "[REDACTED:page:exchange.token]",
),
);
assert.ok(
followupEvents.some(
(event) => event.kind === "response" && event.status === 403,
),
);
assert.equal(
(await (await fetch(countUrl)).json()).totalItems,
beforeCount,
);
const publicScreenshot = await invoke("capture_browser_scenario", {
...sourceScenario,
start_url: { url: origin + "/immediate-followup.html" },
actions: [
{ step_id: "settle", action: "wait_for_timeout", duration_ms: 1 },
],
capture: {
...sourceScenario.capture,
events: [],
network: {
request_body: false,
response_body: false,
header_values: false,
},
after_each_step: [],
at_end: ["screenshot"],
},
});
const publicPath = join(output, "public-screenshot-evidence.json");
await invoke("export_evidence_bundle", { path: publicPath });
const publicBundle = JSON.parse(await readFile(publicPath, "utf8"));
const publicParent = publicBundle.records.find(
(record) =>
record.evidence_id ===
publicScreenshot.normalized_result.parent_evidence_id,
);
assert.ok(publicParent);
assert.equal(await page.locator("#sensitive").innerText(), "");
const png =
publicParent.normalized_result.steps.at(-1).artifacts.screenshot;
assert.equal(png.state, "captured");
assert.equal(
Buffer.from(png.value.data_base64, "base64")
.subarray(0, 8)
.toString("hex"),
"89504e470d0a1a0a",
);
runs.push({
verification: "source-and-later-JSON-spellings",
verified_bodies: bodies.length,
metadata_only_source_url: true,
whole_json_secret_withheld: true,
immediate_followup_protected: true,
immediate_console_protected: true,
rendered_secret_screenshots_withheld: true,
nonsecret_screenshot_captured: true,
whole_json_status: postResponse.status,
});
}
await writeFile(join(output, "calls.json"), JSON.stringify(calls), {
mode: 0o600,
});
await writeFile(
join(output, "metrics.json"),
JSON.stringify({ mode: values.mode, runs }, null, 2),
);
console.log(JSON.stringify({ mode: values.mode, runs }));
} finally {
await withVerifierCleanup(
() => commands.close(),
[
async () => {
if (commands.cleanupResources.length === 0)
await rm(profile, { recursive: true, force: true });
},
],
);
}
experiments/analysis-efficiency/fixtures/browser-saved-values/immediate-followup.html
+30
View File
@@ -0,0 +1,30 @@
<!doctype html>
<meta charset="utf-8" />
<title>Immediate saved-response follow-up fixture</title>
<button id="run">Exchange and follow up</button>
<output id="result">pending</output>
<output id="sensitive"></output>
<script>
document.querySelector("#run").onclick = async () => {
const source = await (
await fetch("/escaped-values.json", { cache: "no-store" })
).json();
document.querySelector("#sensitive").textContent = source.token;
console.log(source.token);
const response = await fetch(
"/api/collections/rea_items/records?token=" +
encodeURIComponent(source.token),
{
method: "POST",
headers: {
"Content-Type": "application/json",
"X-Followup": source.token,
},
body: JSON.stringify({ token: source.token }),
},
);
const result = document.querySelector("#result");
result.textContent = String(response.status);
result.dataset.ready = "yes";
};
</script>
experiments/analysis-efficiency/fixtures/browser-saved-values/index.html
+39
View File
@@ -0,0 +1,39 @@
<!doctype html><meta charset="utf-8" /><title>
PocketBase saved value verifier
</title>
<label>Email<input id="email" /></label
><label>Password<input type="password" id="password" /></label
><button id="login">Sign in</button><button id="create">Create record</button>
<pre id="result"></pre>
<script>
document.querySelector("#login").onclick = async () => {
const r = await fetch("/api/collections/_superusers/auth-with-password", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
identity: document.querySelector("#email").value,
password: document.querySelector("#password").value,
}),
});
const data = await r.json();
localStorage.setItem("rea_auth", JSON.stringify(data));
document.querySelector("#result").textContent = r.ok
? "Signed in"
: "Sign in failed";
if (r.ok) document.querySelector("#result").dataset.ready = "yes";
};
document.querySelector("#create").onclick = async () => {
const token = JSON.parse(localStorage.getItem("rea_auth") || "{}").token;
const r = await fetch("/api/collections/rea_items/records", {
method: "POST",
headers: {
"Content-Type": "application/json",
...(token ? { Authorization: token } : {}),
},
body: JSON.stringify({ title: "rea-ui-record" }),
});
document.querySelector("#result").textContent = JSON.stringify(
await r.json(),
);
};
</script>
experiments/analysis-efficiency/fixtures/browser-saved-values/whole-json.html
+15
View File
@@ -0,0 +1,15 @@
<!doctype html>
<meta charset="utf-8" />
<title>Declared JSON payload privacy fixture</title>
<output id="result">pending</output>
<script>
fetch("/api/collections/rea_items/records", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: localStorage.getItem("whole_json"),
}).then((response) => {
const result = document.querySelector("#result");
result.textContent = String(response.status);
result.dataset.ready = "yes";
});
</script>
scripts/lib/browser-scenario-verifier.mjs
+14 -3
View File
@@ -233,11 +233,19 @@ export function assertScenarioCapture(capture) {
const final = capture.steps.at(-1);
if (
final === undefined ||
Object.values(final.artifacts).some(({ state }) => state !== "captured")
Object.entries(final.artifacts).some(
([kind, { state }]) => kind !== "screenshot" && state !== "captured",
)
)
throw new Error(
`Scenario launch did not capture every requested artifact: ${JSON.stringify(final?.artifacts ?? null)}`,
);
for (const step of capture.steps)
assert.ok(
step.artifacts.screenshot.state === "missing" &&
step.artifacts.screenshot.reason.includes("sensitive"),
"Sensitive scenario screenshots must be withheld from its first step",
);
const storageNames =
final.artifacts.storage.state === "captured"
? final.artifacts.storage.value.local_storage.map(({ name }) => name)
@@ -281,10 +289,13 @@ export function assertScenarioCapture(capture) {
);
// This fixture polls continuously. A request crossing the capture cutoff
// makes later activity unknown even when no response body was selected.
const expectedMissing = unfinished.length === 0 ? [] : ["events"];
const expectedMissing = [
...(unfinished.length === 0 ? [] : ["events"]),
"screenshot",
].sort();
if (
capture.events.items.length === 0 ||
capture.completeness.equality_eligible !== (unfinished.length === 0) ||
capture.completeness.equality_eligible !== false ||
JSON.stringify(capture.completeness.missing_sections) !==
JSON.stringify(expectedMissing) ||
capture.completeness.truncated_sections.length !== 0
src/application/BrowserScenarioEvidence.test.ts
+63
View File
@@ -0,0 +1,63 @@
import { describe, expect, it } from "vitest";
import { scenarioCaptureFixture } from "../../tests/fixtures/browserScenarioCapture.js";
import { browserScenarioSchema } from "../domain/browserScenario.js";
import { digestCanonicalValue } from "../domain/canonicalDigest.js";
import { createBrowserScenarioEvidence } from "./BrowserScenarioEvidence.js";
const ORIGIN = "http://127.0.0.1:3101";
const parameters = (actions: readonly unknown[]) =>
createBrowserScenarioEvidence(
browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: "/opt/chromium/chrome" },
start_url: { url: `${ORIGIN}/` },
actions,
}),
scenarioCaptureFixture(),
{ id: "fixture", name: "fixture", version: "1" },
).parameters;
const open = (fields: Record<string, unknown> = {}) => ({
step_id: "open",
action: "goto",
destination: { url: `${ORIGIN}/items` },
wait_until: "load",
...fields,
});
const create = (fields: Record<string, unknown> = {}) => ({
step_id: "create",
action: "request",
method: "POST",
path: "/api/items",
json: { title: "Item 1" },
...fields,
});
describe("browser scenario evidence parameters", () => {
it("records a scenario that saves nothing as it was recorded before saved values existed", () => {
const recorded = parameters([open(), create()]);
const actions = recorded["actions"] as Record<string, unknown>[];
expect(Object.keys(actions[1]!)).not.toContain("save");
expect(parameters([open(), create({ save: [] })])).toEqual(recorded);
const { scenario_sha256, ...scenario } = recorded;
expect(scenario_sha256).toBe(
digestCanonicalValue(scenario, "Browser scenario"),
);
});
it("records the values a step saves and the placeholders that send them", () => {
const recorded = parameters([
create({ save: [{ name: "item", json_pointer: "/id" }] }),
open({ destination: { url: `${ORIGIN}/items/{saved:item}` } }),
]);
expect(recorded["actions"]).toMatchObject([
{ save: [{ name: "item", json_pointer: "/id", secret: false }] },
{ destination: { url: `${ORIGIN}/items/%7Bsaved:item%7D` } },
]);
expect(recorded["scenario_sha256"]).not.toBe(
parameters([create(), open()])["scenario_sha256"],
);
});
});
src/application/BrowserScenarioEvidence.ts
+19 -4
View File
@@ -9,12 +9,27 @@ import {
} from "../domain/evidence.js";
import { jsonObjectSchema } from "../domain/jsonValue.js";
/** A step as Evidence records it: a request step leaves out an empty `save`, so a scenario that saves nothing records and hashes as it did before saved values existed. */
const recordedAction = (
action: BrowserScenario["actions"][number],
): Record<string, unknown> => {
if (action.action !== "request" || action.save.length > 0) return action;
const { save, ...rest } = action;
return rest;
};
const browserScenarioParameters = (
scenario: BrowserScenario,
): EvidenceObservation["parameters"] => ({
...jsonObjectSchema.parse(scenario),
scenario_sha256: digestCanonicalValue(scenario, "Browser scenario"),
});
): EvidenceObservation["parameters"] => {
const recorded = {
...scenario,
actions: scenario.actions.map(recordedAction),
};
return {
...jsonObjectSchema.parse(recorded),
scenario_sha256: digestCanonicalValue(recorded, "Browser scenario"),
};
};
/** Create Evidence without retaining resolved scenario secret values. */
export const createBrowserScenarioEvidence = (
src/application/environment/environmentBuild.ts
+5 -1
View File
@@ -25,6 +25,7 @@ import {
import {
anonymousOrigin,
credentialScanWarnings,
savedCredentialWarnings,
nameFromCaptureFile,
type EnvironmentBuildWarning,
} from "./environmentBuildReport.js";
@@ -317,7 +318,10 @@ export const buildEnvironment = async (
name,
operation: ENVIRONMENT_EXPORT_OPERATION,
});
warnings.push(...credentialScanWarnings(recorded.value));
warnings.push(
...credentialScanWarnings(recorded.value),
...savedCredentialWarnings(capture.parameters),
);
const packageSource: EnvironmentPackageSource =
input.packageTarball === undefined
src/application/environment/environmentBuildReport.test.ts
+52
View File
@@ -6,6 +6,7 @@ import {
anonymousOrigin,
credentialNamed,
nameFromCaptureFile,
savedCredentialWarnings,
scanCredentialNames,
scanScenarioCredentialNames,
} from "./environmentBuildReport.js";
@@ -455,3 +456,54 @@ it.each([
});
},
);
describe("savedCredentialWarnings", () => {
const entry = (name: string, secret?: boolean) => ({
name,
json_pointer: "/x",
...(secret === undefined ? {} : { secret }),
});
it("points at saved values named like credentials that are not marked secret", () => {
expect(
savedCredentialWarnings({
actions: [
{ action: "goto" },
{
action: "request",
save: [
entry("item_id"),
entry("access_token"),
entry("api_key", true),
],
},
{
action: "click",
save_response: { save: [entry("password", false), entry("list")] },
},
],
}),
).toEqual([
{
code: "credential-named-saved-values",
message: expect.stringContaining("2 saved values"),
pointers: [
"/parameters/actions/1/save/1",
"/parameters/actions/2/save_response/save/0",
],
},
]);
});
it("warns about nothing when every credential-named value is secret, or the parameters are not a scenario", () => {
expect(
savedCredentialWarnings({
actions: [
{ action: "request", save: [entry("token", true), entry("id")] },
],
}),
).toEqual([]);
expect(savedCredentialWarnings({ operation: "har" })).toEqual([]);
expect(savedCredentialWarnings(undefined)).toEqual([]);
});
});
src/application/environment/environmentBuildReport.ts
+52
View File
@@ -1,3 +1,4 @@
import { z } from "zod";
import { basename } from "node:path";
import { isIP } from "node:net";
@@ -332,6 +333,57 @@ export const credentialScanWarnings = (capture: {
return warnings;
};
const savedEntries = z.array(
z.object({ name: z.string(), secret: z.boolean().optional() }),
);
const scenarioSaves = z.object({
actions: z.array(
z
.object({
save: savedEntries.optional(),
save_response: z.object({ save: savedEntries }).optional(),
})
.loose(),
),
});
/** A browser scenario's saved values named like credentials but not marked secret, which the capture records unredacted. */
export const savedCredentialWarnings = (
parameters: unknown,
): EnvironmentBuildWarning[] => {
const parsed = scenarioSaves.safeParse(parameters);
if (!parsed.success) return [];
const pointers: string[] = [];
parsed.data.actions.forEach((action, index) => {
const unmarked = (
entries:
| readonly {
readonly name: string;
readonly secret?: boolean | undefined;
}[]
| undefined,
at: string,
) =>
entries?.forEach((entry, position) => {
if (entry.secret !== true && credentialNamed(entry.name))
pointers.push(
`/parameters/actions/${String(index)}/${at}/${String(position)}`,
);
});
unmarked(action.save, "save");
unmarked(action.save_response?.save, "save_response/save");
});
return pointers.length === 0
? []
: [
{
code: "credential-named-saved-values",
message: `${String(pointers.length)} saved values have credential-like names but are not marked secret, so the capture records them unredacted; mark them \`secret: true\` and record again before sharing.`,
pointers: pointers.slice(0, REPORTED_POINTERS),
},
];
};
/** Whether an origin's host is a loopback name or an IP literal, which says nothing about the application. */
export const anonymousOrigin = (origin: string): boolean => {
let host: string;
src/application/environment/environmentStateAssignedCoverage.test.ts
+37 -16
View File
@@ -4,24 +4,45 @@ import { servedFrom } from "../../../tests/fixtures/environmentStateServing.js";
const uuid = (n: number): string =>
`00000000-0000-4000-8000-${String(n).padStart(12, "0")}`;
/** Two held books and a create whose `mixed` joins the generated id and the sent slug; the held books put `heldPrefix` before theirs. */
const slugBooks = (heldPrefix = "") => {
const item = (id: number, slug: string, prefix = "") => ({
id,
code: slug,
url: `/${slug}`,
mixed: `/${id}/${prefix}${slug}`,
});
const first = item(1, "first-slug", heldPrefix);
const second = item(2, "second-slug", heldPrefix);
const third = item(3, "abcdefgh");
return servedFrom([
["GET", "/api/books", [first, second]],
["GET", "/api/books/1", first],
["GET", "/api/books/2", second],
["POST", "/api/books", third, 201, { customSlug: "abcdefgh" }],
["GET", "/api/books/3", third],
]);
};
describe("assigned values preserve every recorded relation", () => {
it("freezes a copied member when another template would retain its old value", () => {
const item = (id: number, slug: string) => ({
id,
code: slug,
url: `/${slug}`,
mixed: `/${id}/${slug}`,
it("rebuilds a field a generated id and a copied member compose, accepting any value of the member", () => {
const books = slugBooks();
expect(books.state.writes.get("post_books")).toMatchObject({
served: true,
body: { fixed: [] },
});
const first = item(1, "first-slug");
const second = item(2, "second-slug");
const third = item(3, "abcdefgh");
const books = servedFrom([
["GET", "/api/books", [first, second]],
["GET", "/api/books/1", first],
["GET", "/api/books/2", second],
["POST", "/api/books", third, 201, { customSlug: "abcdefgh" }],
["GET", "/api/books/3", third],
]);
expect(
books.answer("post_books", { body: { customSlug: "ijklmnop" } }).body,
).toMatchObject({
code: "ijklmnop",
url: "/ijklmnop",
mixed: expect.stringMatching(/^\/\d+\/ijklmnop$/u),
});
});
it("freezes a copied member when another template would retain its old value", () => {
// The held books spell `mixed` otherwise, so no composition explains it and a template around the id keeps the slug.
const books = slugBooks("x-");
expect(books.state.writes.get("post_books")).toMatchObject({
served: true,
body: { fixed: [{ member: "customSlug", sent: "abcdefgh" }] },
src/browser/BrowserScenarioResponseValues.ts
+246
View File
@@ -0,0 +1,246 @@
import { visit, type JSONPath } from "jsonc-parser";
import { isExactJsonNumber } from "../domain/exactJsonNumber.js";
import { safeParseJson } from "../domain/safeJson.js";
import { err, ok, type Result } from "../domain/result.js";
import type { BrowserScenarioSaveEntry } from "../domain/browserScenarioValues.js";
/** Bound decoded JSON before parsing; UTF-8 needs at most three bytes per UTF-16 code unit. */
export const MAX_SAVED_RESPONSE_CHARACTERS = 4_000_000;
export const MAX_SAVED_RESPONSE_BYTES = 3 * MAX_SAVED_RESPONSE_CHARACTERS;
interface ResponseValueIssue {
readonly _tag: "ResponseValueIssue";
readonly message: string;
}
const issue = (message: string): ResponseValueIssue => ({
_tag: "ResponseValueIssue",
message,
});
export interface ResponseValue {
readonly entry: BrowserScenarioSaveEntry;
readonly value: string | number;
}
const pointerTokens = (pointer: string): readonly string[] =>
pointer === ""
? []
: pointer
.slice(1)
.split("/")
.map((token) => token.replaceAll("~1", "/").replaceAll("~0", "~"));
/** Exact own-property lookup: an absent pointer remains distinct from a non-scalar value. */
export const responsePointerValue = (
value: unknown,
pointer: string,
): unknown => {
let current = value;
for (const key of pointerTokens(pointer)) {
if (Array.isArray(current) && /^(?:0|[1-9][0-9]*)$/u.test(key))
current = current[Number(key)];
else if (
typeof current === "object" &&
current !== null &&
Object.hasOwn(current, key)
)
current = (current as Record<string, unknown>)[key];
else return undefined;
}
return current;
};
/** Both bindings and source-body redaction use the same decoded values and exact-number policy. */
export const readResponseValues = (
body: string | Uint8Array | undefined,
entries: readonly BrowserScenarioSaveEntry[],
missing: "fail" | "ignore",
): Result<
{ readonly values: readonly ResponseValue[]; readonly text: string },
ResponseValueIssue
> => {
if (
body === undefined ||
(typeof body !== "string" && body.byteLength > MAX_SAVED_RESPONSE_BYTES)
)
return err(issue("the response is too large to save values from."));
let text: string;
try {
text =
typeof body === "string"
? body
: new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(
body,
);
} catch {
return err(
issue(
"the response is not valid UTF-8 JSON, so no value can be saved from it.",
),
);
}
if (text.length > MAX_SAVED_RESPONSE_CHARACTERS)
return err(issue("the response is too large to save values from."));
const parsed = safeParseJson(text, (_key, value, context) =>
typeof value === "number" && !isExactJsonNumber(value, context?.source)
? Number.NaN
: value,
);
if (!parsed.ok)
return err(
issue("the response is not JSON, so no value can be saved from it."),
);
const selected: ResponseValue[] = [];
for (const entry of entries) {
const value = responsePointerValue(parsed.value, entry.json_pointer);
if (value === undefined && missing === "ignore") continue;
if (typeof value !== "string" && typeof value !== "number")
return err(
issue(
`the response has no string or number at ${entry.json_pointer} to save as ${entry.name}.`,
),
);
if (typeof value === "number" && !Number.isFinite(value))
return err(
issue(
`the number at ${entry.json_pointer} cannot be saved exactly (not finite or JSON conversion would lose precision).`,
),
);
selected.push({ entry, value });
}
return ok({ values: selected, text });
};
interface ResponseBodySource {
headers(): Readonly<Record<string, string>>;
request(): { sizes?(): Promise<{ readonly responseBodySize: number }> };
body(): Promise<Buffer>;
}
/** The caller owns deadline/cancellation; unknown decoded bounds never request a whole body. */
export const readBoundedResponseBody = async (
response: ResponseBodySource,
maxBytes: number,
): Promise<Result<Buffer, ResponseValueIssue>> => {
const request = response.request();
const size =
request.sizes === undefined
? undefined
: (await request.sizes()).responseBodySize;
if (size !== undefined && size > maxBytes)
return err(issue("the response is too large to save values from."));
const encoding = response.headers()["content-encoding"]?.trim().toLowerCase();
if (
size === undefined ||
!Number.isSafeInteger(size) ||
size <= 0 ||
(encoding !== undefined && encoding !== "" && encoding !== "identity")
)
return err(
issue(
"the decoded response size cannot be bounded before reading it; this response body was not read.",
),
);
const body = await response.body();
return body.length > maxBytes
? err(issue("the response is too large to save values from."))
: ok(body);
};
/** Replace selected JSON tokens and known decoded string secrets; preserve other source spellings and whitespace. */
export const redactResponseSource = (
text: string,
values: readonly ResponseValue[],
redact: (value: string) => string,
maxBytes: number,
maxAllocationBytes: number,
): Result<Buffer | undefined, ResponseValueIssue> => {
const markers = new Map(
values.map(({ entry, value }) => [
JSON.stringify(pointerTokens(entry.json_pointer)),
redact(String(value)),
]),
);
const parts: string[] = [];
let cursor = 0;
let bytes = Buffer.byteLength(text);
const replace = (offset: number, length: number, marker: string): void => {
const replacement = JSON.stringify(marker);
bytes +=
Buffer.byteLength(replacement) -
Buffer.byteLength(text.slice(offset, offset + length));
if (bytes > maxBytes || bytes * 4 > maxAllocationBytes)
throw new Error(
"Structural source redaction exceeds the selected body allocation budget.",
);
parts.push(text.slice(cursor, offset), replacement);
cursor = offset + length;
};
const markerAt = (path: JSONPath): string | undefined =>
markers.get(JSON.stringify(path.map(String)));
const container = (
_offset: number,
_length: number,
_line: number,
_column: number,
path: () => JSONPath,
): void => {
if (markerAt(path()) !== undefined)
throw new Error(
"A marked source field has conflicting scalar/container representations; its source content was withheld.",
);
};
try {
visit(
text,
{
onObjectBegin: container,
onArrayBegin: container,
onLiteralValue: (
value: unknown,
offset,
length,
_line,
_column,
path,
) => {
let marker = markerAt(path());
if (marker === undefined) {
const raw = text.slice(offset, offset + length);
const rendered =
typeof value === "number" && !isExactJsonNumber(value, raw)
? undefined
: String(value);
const decoded =
rendered === undefined ? undefined : redact(rendered);
if (decoded !== undefined && decoded !== rendered) marker = decoded;
if (marker === undefined && typeof value !== "string") {
const redacted = redact(raw);
if (redacted !== raw) marker = redacted;
}
}
if (marker !== undefined) replace(offset, length, marker);
},
onObjectProperty: (property, offset, length) => {
const redacted = redact(property);
if (redacted !== property) replace(offset, length, redacted);
},
onError: () => {
throw new Error("The marked source is not valid JSON.");
},
},
{ disallowComments: true, allowTrailingComma: false },
);
if (parts.length === 0) return ok(undefined);
parts.push(text.slice(cursor));
return ok(Buffer.from(parts.join("")));
} catch (cause: unknown) {
return err(
issue(
cause instanceof Error
? cause.message
: "Structural source redaction failed.",
),
);
}
};
src/browser/BrowserScenarioSecrets.ts
+388 -33
View File
@@ -7,8 +7,14 @@ import { createHash } from "node:crypto";
import type { BrowserScenario } from "../domain/browserScenario.js";
import type {
BrowserScenarioUrl,
BrowserScenarioRequestValue,
BrowserScenarioValue,
BrowserScenarioSaveEntry,
} from "../domain/browserScenarioValues.js";
import {
readResponseValues,
redactResponseSource,
} from "./BrowserScenarioResponseValues.js";
import {
sanitizeBrowserUrl,
type SanitizedBrowserUrl,
@@ -20,11 +26,29 @@ const REDACTION_PREFIX = "[REDACTED:";
/** Page values shorter than this are not redacted: redacting them would rewrite ids, ports and hosts throughout the capture. */
export const MIN_REDACTED_PAGE_VALUE_LENGTH = 8;
const encodedSecretValues = (secret: string): ReadonlySet<string> =>
new Set([
encodeURIComponent(secret),
/** The most saved-value text a scenario keeps at once, with the redaction spellings of values saved as secret: as much as one saved response can carry. */
export const MAX_RETAINED_SAVED_CHARACTERS = 4_000_000;
export type ScenarioRedactionScope = "current" | "response";
interface ResponseRedactionFacts {
readonly text: readonly SecretReplacement[];
readonly url: readonly SecretReplacement[];
readonly needles: readonly SecretNeedle[];
readonly costPerCharacter: number;
}
const normalizedSecretValues = (value: string): readonly string[] => [
...new Set([value.replace(/^[\t ]+|[\t ]+$/gu, ""), value]),
];
const encodedSecretValues = (secret: string): ReadonlySet<string> => {
const component = encodeURIComponent(secret);
return new Set([
component,
// Special-URL queries escape apostrophes while keeping the component's tilde.
component.replace(/'/gu, "%27"),
new URLSearchParams([["value", secret]]).toString().slice("value=".length),
]);
};
type SecretSpellings = "url" | "text";
interface SecretPattern {
@@ -76,6 +100,38 @@ const secretPatterns = (
});
};
/** The code units a secret's redaction spellings keep, each distinct spelling once. */
const spellingUnits = (secret: string): number =>
[
...new Set(
[
...secretPatterns(secret, "text"),
...secretPatterns(secret, "url"),
].flatMap(({ text, searchText }) => [text, searchText]),
),
].reduce((units, spelling) => units + spelling.length, 0);
const URI_KEPT = /[\w.!~*'()-]/u;
const FORM_KEPT = /[\w .*-]/u;
/** A lower bound that refuses over-budget spellings before allocating encoded copies. */
const leastSpellingUnits = (secret: string): number => {
let uri = 0;
let query = 0;
let form = 0;
for (const character of secret) {
const point = character.codePointAt(0) ?? 0;
const escaped =
3 * (point < 0x80 ? 1 : point < 0x800 ? 2 : point < 0x10000 ? 3 : 4);
const uriKept = URI_KEPT.test(character);
uri += uriKept ? 1 : escaped;
query += uriKept && character !== "'" ? 1 : escaped;
form += FORM_KEPT.test(character) ? 1 : escaped;
}
const encoded = Math.max(uri, query, form);
return secret.length + (encoded > secret.length ? encoded : 0);
};
const declaredPatterns = (
values: ReadonlyMap<string, string>,
spellings: SecretSpellings,
@@ -185,6 +241,7 @@ const replaceSecretLiterals = (
value: string,
replacements: readonly SecretReplacement[],
patterns: readonly SecretPattern[],
markerPatterns: readonly SecretPattern[] = patterns,
): string => {
const normalizedValue = replacements.some(
({ percentInsensitive }) => percentInsensitive,
@@ -210,7 +267,7 @@ const replaceSecretLiterals = (
next = { ...replacement, index };
}
if (next === undefined) break;
const marker = redactionMarker(next.id, patterns);
const marker = redactionMarker(next.id, markerPatterns);
parts.push(value.slice(offset, next.index), marker);
offset = next.index + next.text.length;
}
@@ -238,8 +295,27 @@ export class BrowserScenarioSecrets {
private textReplacements: readonly SecretReplacement[] = [];
private textNeedles: readonly SecretNeedle[] = [];
private redactionCostPerCharacter = 8;
private readonly declaredIds: ReadonlySet<string>;
private readonly declaredValues: ReadonlySet<string>;
private readonly responseValues = new Set<string>();
private responseFacts: ResponseRedactionFacts | undefined;
private constructor(private readonly values: Map<string, string>) {
/** Pixel privacy is established before the scenario can render a future marked response. */
withholdsScreenshots(): boolean {
return this.plansSensitiveValues || this.textPatterns.length > 0;
}
/** Explicit response facts, independent of source readiness and subsequent-only page credentials. */
hasLearnedResponseSecrets(): boolean {
return this.responseValues.size > 0;
}
private constructor(
private readonly values: Map<string, string>,
private readonly plansSensitiveValues: boolean,
) {
this.declaredIds = new Set(values.keys());
this.declaredValues = new Set(values.values());
this.compile();
}
@@ -250,19 +326,54 @@ export class BrowserScenarioSecrets {
this.urlReplacements = secretReplacements(values, "url");
this.textReplacements = secretReplacements(values, "text");
this.textNeedles = secretNeedles(this.textReplacements, this.textPatterns);
this.redactionCostPerCharacter = 8;
this.redactionCostPerCharacter = this.costOf(
this.textReplacements,
this.urlReplacements,
);
this.responseFacts = undefined;
}
private costOf(
text: readonly SecretReplacement[],
url: readonly SecretReplacement[],
): number {
let cost = 8;
for (const [replacements, patterns] of [
[this.textReplacements, this.textPatterns],
[this.urlReplacements, this.urlPatterns],
[text, this.textPatterns],
[url, this.urlPatterns],
] as const)
for (const replacement of replacements)
this.redactionCostPerCharacter = Math.max(
this.redactionCostPerCharacter,
cost = Math.max(
cost,
8 +
(Buffer.byteLength(redactionMarker(replacement.id, patterns)) * 8 +
128) /
replacement.text.length,
);
return cost;
}
/** Borrow compiled pattern/byte owners; invalidate on admission or scope promotion, not per event. */
private responseRedactions(): ResponseRedactionFacts {
if (this.responseFacts !== undefined) return this.responseFacts;
const eligible = ({ id }: { readonly id: string }): boolean =>
this.declaredIds.has(id) ||
this.responseValues.has(this.values.get(id) ?? "");
const text = this.textReplacements.filter(eligible),
url = this.urlReplacements.filter(eligible);
return (this.responseFacts = {
text,
url,
needles: this.textNeedles.filter(eligible),
costPerCharacter: this.costOf(text, url),
});
}
private markResponseValue(value: string): void {
for (const candidate of normalizedSecretValues(value))
if (candidate.length > 0 && !this.declaredValues.has(candidate))
this.responseValues.add(candidate);
this.responseFacts = undefined;
}
/**
@@ -273,15 +384,50 @@ export class BrowserScenarioSecrets {
* {@link MIN_REDACTED_PAGE_VALUE_LENGTH} is not redacted.
*/
admitPageValue(label: string, value: string): void {
// fetch sends a header value without surrounding spaces and tabs, so that is the spelling records hold.
const sent = value.replace(/^[\t ]+|[\t ]+$/gu, "");
if (sent !== value) this.admitPageValue(label, sent);
if (value.length < MIN_REDACTED_PAGE_VALUE_LENGTH) return;
for (const held of this.values.values()) if (held === value) return;
let id = `page:${label}`;
for (let suffix = 2; this.values.has(id); suffix += 1)
id = `page:${label}~${String(suffix)}`;
this.values.set(id, value);
this.retainValues(
label,
this.valuesToAdmit(value, MIN_REDACTED_PAGE_VALUE_LENGTH),
);
}
/** Build only bounded Basic text; all distinct encoded spellings share the saved-value budget. */
basicCredential(label: string, username: string, password: string): string {
const bytes = Buffer.byteLength(username) + 1 + Buffer.byteLength(password);
const encodedCharacters = 4 * Math.ceil(bytes / 3);
if (encodedCharacters > MAX_RETAINED_SAVED_CHARACTERS)
throw new Error(
`Basic credentials cannot be built: their encoded form exceeds ${String(MAX_RETAINED_SAVED_CHARACTERS)} characters.`,
);
const input = Buffer.allocUnsafe(bytes);
let offset = input.write(username);
offset += input.write(":", offset);
input.write(password, offset);
const encoded = input.toString("base64");
this.retainSavedRedactions(
label,
this.valuesToAdmit(encoded, 1),
this.savedUnits,
"Basic credentials",
);
return encoded;
}
/** Fetch trims surrounding header whitespace; both source and sent spellings remain owned here. */
private valuesToAdmit(value: string, shortest: number): readonly string[] {
const held = new Set(this.values.values());
return normalizedSecretValues(value).filter(
(candidate) => candidate.length >= shortest && !held.has(candidate),
);
}
private retainValues(label: string, values: readonly string[]): void {
if (values.length === 0) return;
for (const value of values) {
let id = `page:${label}`;
for (let suffix = 2; this.values.has(id); suffix += 1)
id = `page:${label}~${String(suffix)}`;
this.values.set(id, value);
}
this.compile();
}
@@ -317,11 +463,190 @@ export class BrowserScenarioSecrets {
);
values.set(declaration.secret_id, value);
}
return ok(new BrowserScenarioSecrets(values));
const plansSensitiveValues = scenario.actions.some((action) =>
action.action === "request"
? action.save.some((entry) => entry.secret) ||
action.headers.some(
({ value }) =>
value.source === "cookie" ||
value.source === "local_storage" ||
value.source === "basic_credentials",
)
: (action.action === "click" || action.action === "press") &&
(action.save_response?.save.some((entry) => entry.secret) ?? false),
);
return ok(new BrowserScenarioSecrets(values, plansSensitiveValues));
}
value(source: BrowserScenarioValue): string {
/** Values steps saved from JSON responses, by name, as strings or numbers. */
private readonly saved = new Map<string, string | number>();
/** Code units the current saved values keep, and those the redaction spellings of secret saves keep for the rest of the scenario. */
private savedUnits = 0;
private secretSpellingUnits = 0;
/** Policies are attached before dispatch to the browser's exact request identity. */
private readonly responseSecrets = new WeakMap<
object,
{
readonly stepId: string;
readonly entries: readonly BrowserScenarioSaveEntry[];
}
>();
private readonly readyResponses = new WeakSet<object>();
responseSecretsReady(request: object): boolean {
return this.readyResponses.has(request);
}
markResponseSecretsReady(request: object): void {
this.readyResponses.add(request);
}
responseProtector(
stepId: string,
entries: readonly BrowserScenarioSaveEntry[],
): ((request: object) => void) | undefined {
const sensitive = entries.filter((entry) => entry.secret);
if (sensitive.length === 0) return undefined;
const policy = { stepId, entries: sensitive };
return (request) => {
this.responseSecrets.set(request, policy);
};
}
hasResponseProtection(request: object): boolean {
return this.responseSecrets.has(request);
}
/** Redact the selected source fields before network Evidence can retain the body. */
admitResponseSecrets(
request: object,
body: Buffer,
retention?: {
readonly maxBytes: number;
readonly maxAllocationBytes: number;
},
): Buffer {
const source = this.responseSecrets.get(request);
if (source === undefined) return body;
const selected = readResponseValues(body, source.entries, "ignore");
if (!selected.ok)
throw new Error(`Step ${source.stepId}: ${selected.error.message}`);
for (const { entry, value } of selected.value.values) {
const additions = this.valuesToAdmit(String(value), 1);
this.retainSavedRedactions(
`${source.stepId}.${entry.name}`,
additions,
this.savedUnits,
entry.name,
);
this.markResponseValue(String(value));
}
if (selected.value.values.length === source.entries.length)
this.markResponseSecretsReady(request);
if (retention === undefined) return body;
const redacted = redactResponseSource(
selected.value.text,
selected.value.values,
(value) => this.redact(value),
retention.maxBytes,
retention.maxAllocationBytes,
);
if (!redacted.ok) throw new Error(redacted.error.message);
return redacted.value ?? body;
}
/** Decode JSON only when redaction is active; other bytes retain their existing binary policy. */
redactJsonBody(
body: Buffer,
retention: {
readonly maxBytes: number;
readonly maxAllocationBytes: number;
},
scope: ScenarioRedactionScope = "current",
): Buffer {
if (
body.length === 0 ||
(scope === "response"
? this.responseRedactions().text
: this.textPatterns
).length === 0
)
return body;
const text = new TextDecoder("utf-8", {
fatal: true,
ignoreBOM: true,
}).decode(body);
const redacted = redactResponseSource(
text,
[],
(value) => this.redact(value, scope),
retention.maxBytes,
retention.maxAllocationBytes,
);
if (!redacted.ok) throw new Error(redacted.error.message);
return redacted.value ?? body;
}
private retainSavedRedactions(
label: string,
additions: readonly string[],
savedUnits: number,
name: string,
): void {
const available =
MAX_RETAINED_SAVED_CHARACTERS - savedUnits - this.secretSpellingUnits;
const minimum = additions.reduce(
(units, value) => units + leastSpellingUnits(value),
0,
);
const spellings =
minimum > available
? minimum
: additions.reduce((units, value) => units + spellingUnits(value), 0);
if (available < 0 || spellings > available)
throw new Error(
`${name} cannot be saved: the scenario would keep more than ${String(MAX_RETAINED_SAVED_CHARACTERS)} characters of saved values.`,
);
this.secretSpellingUnits += spellings;
this.retainValues(label, additions);
}
/** Keep a value a step saved from a JSON response; a secret one is redacted from everything recorded after it. Refused before it is kept when the scenario would hold more than {@link MAX_RETAINED_SAVED_CHARACTERS}. */
save(
label: string,
name: string,
value: string | number,
secret: boolean,
): void {
const text = String(value);
const replaced = this.saved.get(name);
const savedUnits =
this.savedUnits -
(replaced === undefined ? 0 : String(replaced).length) +
text.length;
const available =
MAX_RETAINED_SAVED_CHARACTERS - savedUnits - this.secretSpellingUnits;
const additions =
secret && available >= 0 ? this.valuesToAdmit(text, 1) : [];
this.retainSavedRedactions(label, additions, savedUnits, name);
if (secret) this.markResponseValue(text);
this.saved.set(name, value);
this.savedUnits = savedUnits;
}
/** A saved value with the JSON type it was saved with. */
savedValue(name: string): string | number {
const value = this.saved.get(name);
if (value === undefined) throw new Error(`No earlier step saved ${name}`);
return value;
}
value(source: BrowserScenarioValue | BrowserScenarioRequestValue): string {
if (source.source === "literal") return source.value;
if (source.source === "saved") return String(this.savedValue(source.name));
const value = this.values.get(source.secret_id);
if (value === undefined)
throw new Error("Validated browser secret was not resolved");
@@ -339,8 +664,16 @@ export class BrowserScenarioSecrets {
* Callers measuring untrusted observations must reserve this cost before
* invoking redaction. A one-byte secret can otherwise expand every byte.
*/
redactionAllocationCost(length: number): number {
return Math.ceil(length * this.redactionCostPerCharacter);
redactionAllocationCost(
length: number,
scope: ScenarioRedactionScope = "current",
): number {
return Math.ceil(
length *
(scope === "response"
? this.responseRedactions().costPerCharacter
: this.redactionCostPerCharacter),
);
}
redactFailure(error: AnalysisError): AnalysisError {
@@ -348,13 +681,18 @@ export class BrowserScenarioSecrets {
return redactSensitiveFailure(error, (value) => this.redact(value));
}
redact(value: string): string {
const patterns = this.textPatterns;
redact(value: string, scope: ScenarioRedactionScope = "current"): string {
const replacements =
scope === "response"
? this.responseRedactions().text
: this.textReplacements;
const patterns = scope === "response" ? replacements : this.textPatterns;
if (patterns.length === 0) return value;
const output = replaceSecretLiterals(
value,
this.textReplacements,
replacements,
patterns,
this.textPatterns,
);
return containsSecretPattern(
output,
@@ -366,10 +704,15 @@ export class BrowserScenarioSecrets {
}
/** Replace declared UTF-8, URI/form, and JSON-escaped secret spellings. */
redactBytes(value: Buffer): Buffer {
const patterns = this.textPatterns;
if (patterns.length === 0) return value;
const candidates = this.textNeedles;
redactBytes(
value: Buffer,
scope: ScenarioRedactionScope = "current",
): Buffer {
const candidates =
scope === "response"
? this.responseRedactions().needles
: this.textNeedles;
if (candidates.length === 0) return value;
const hasPercentPatterns = candidates.some(
({ percentInsensitive }) => percentInsensitive,
);
@@ -424,10 +767,22 @@ export class BrowserScenarioSecrets {
}
/** Remove only declared secret values from one URL and report that action. */
sanitizeUrl(value: string): SanitizedBrowserUrl {
const patterns = this.urlPatterns;
sanitizeUrl(
value: string,
scope: ScenarioRedactionScope = "current",
): SanitizedBrowserUrl {
const replacements =
scope === "response"
? this.responseRedactions().url
: this.urlReplacements;
const patterns = scope === "response" ? replacements : this.urlPatterns;
if (patterns.length === 0) return sanitizeBrowserUrl(value);
const output = replaceSecretLiterals(value, this.urlReplacements, patterns);
const output = replaceSecretLiterals(
value,
replacements,
patterns,
this.urlPatterns,
);
if (
containsSecretPattern(output, normalizePercentEscapes(output), patterns)
)
src/browser/PlaywrightBrowserScenarioProvider.test.ts
+58
View File
@@ -1,3 +1,6 @@
import type { BrowserContext, Page } from "playwright-core";
import { capturePlaywrightStepArtifacts } from "./PlaywrightScenarioArtifacts.js";
import { resolveScenarioSecrets } from "../../tests/fixtures/browserScenarioSecrets.js";
import {
failBrowserScenarioOperation,
PlaywrightScenarioBrowserCleanupOwner,
@@ -1273,3 +1276,58 @@ it("keeps an earlier physical failure primary when final cleanup also observes c
await provider.close();
}
});
it.each([false, true])(
"keeps planned secret screenshots unavailable through success and cleanup failure (failure: %s)",
async (cleanupFailure) => {
const input = browserScenarioSchema.parse({
...scenario({ captures: ["screenshot"] }),
actions: [
{
step_id: "request",
action: "request",
method: "POST",
path: "/api/items",
save: [{ name: "token", json_pointer: "/token", secret: true }],
},
],
});
const secrets = resolveScenarioSecrets(input, {});
const screenshot = vi.fn(async () =>
Buffer.from("private screenshot pixels"),
);
const session = new FakeSession(
"launch",
cleanupFailure ? { close: new Error("cleanup failed") } : {},
);
session.capture = async (requested) =>
capturePlaywrightStepArtifacts({
context: {} as BrowserContext,
page: { screenshot } as unknown as Page,
secrets,
requested,
});
session.onPerform = () =>
secrets.save("request.token", "token", "late-pixel-token", true);
const provider = new PlaywrightBrowserScenarioProvider(async () => session);
const result = await provider.captureScenario(input);
const step = {
artifacts: { screenshot: { state: "missing" } },
completeness: {
equality_eligible: false,
missing_sections: ["screenshot"],
},
};
if (cleanupFailure)
expect(result).toMatchObject({
ok: false,
error: { partialObservation: { capture: { steps: [step, step] } } },
});
else {
if (!result.ok) throw result.error;
expect(result.value.steps).toMatchObject([step, step]);
expect(result.value.completeness.equality_eligible).toBe(false);
}
expect(screenshot).not.toHaveBeenCalled();
},
);
src/browser/PlaywrightScenarioActions.ts
+61 -16
View File
@@ -1,11 +1,17 @@
import type { Locator, Page } from "playwright-core";
import type {
BrowserScenarioAction,
BrowserScenarioValue,
import {
SAVED_URL_PLACEHOLDER,
type BrowserScenarioAction,
type BrowserScenarioValue,
} from "../domain/browserScenarioValues.js";
import type { BrowserScenarioSecrets } from "./BrowserScenarioSecrets.js";
import { performScenarioRequest } from "./PlaywrightScenarioRequest.js";
import {
awaitSavedResponse,
performScenarioRequest,
withSavedPlaceholders,
type SavedResponseWait,
} from "./PlaywrightScenarioRequest.js";
type ScenarioLocator = Extract<
BrowserScenarioAction,
@@ -41,6 +47,23 @@ interface PerformActionInput {
readonly signal?: AbortSignal | undefined;
}
/** Watch the response selector before the UI action; matching does not prove action causality. */
const savingResponse = (
input: PerformActionInput,
action: Extract<BrowserScenarioAction, { action: "click" | "press" }>,
timeout: number | undefined,
): SavedResponseWait | undefined =>
action.save_response === undefined
? undefined
: awaitSavedResponse(input.page, {
stepId: action.step_id,
spec: action.save_response,
startOrigin: input.startOrigin,
timeout,
secrets: input.secrets,
signal: input.signal,
});
/** Execute one admitted declarative action; a request step sends one declared request through the page's own fetch, and no action runs caller-supplied code. */
export const performPlaywrightScenarioAction = async (
input: PerformActionInput,
@@ -50,27 +73,49 @@ export const performPlaywrightScenarioAction = async (
const options = timeout === undefined ? {} : { timeout };
switch (action.action) {
case "goto":
await page.goto(secrets.url(action.destination), {
...options,
waitUntil: action.wait_until,
});
// Placeholders fill the URL as written, before structured query values join it.
await page.goto(
secrets.url({
...action.destination,
url: withSavedPlaceholders(
action.destination.url,
secrets,
SAVED_URL_PLACEHOLDER,
),
}),
{ ...options, waitUntil: action.wait_until },
);
return;
case "click":
await resolveLocator(page, action.locator).click({
...options,
button: action.button,
clickCount: action.click_count,
});
case "click": {
const saving = savingResponse(input, action, timeout);
try {
await resolveLocator(page, action.locator).click({
...options,
button: action.button,
clickCount: action.click_count,
});
await saving?.saved();
} finally {
saving?.stop();
}
return;
}
case "fill":
await resolveLocator(page, action.locator).fill(
resolveValue(action.value, secrets),
options,
);
return;
case "press":
await resolveLocator(page, action.locator).press(action.key, options);
case "press": {
const saving = savingResponse(input, action, timeout);
try {
await resolveLocator(page, action.locator).press(action.key, options);
await saving?.saved();
} finally {
saving?.stop();
}
return;
}
case "select_option":
await resolveLocator(page, action.locator).selectOption(
resolveValue(action.value, secrets),
src/browser/PlaywrightScenarioArtifacts.test.ts
+143 -1
View File
@@ -1,4 +1,4 @@
import { expect, it } from "vitest";
import { expect, it, vi } from "vitest";
import type { BrowserContext, Page } from "playwright-core";
import { browserScenarioSchema } from "../domain/browserScenario.js";
@@ -159,3 +159,145 @@ it("redacts a declared secret used as a cookie or storage name", async () => {
},
});
});
it.each(["declared", "saved"])(
"withholds selected screenshots with %s sensitive values before requesting pixels",
async (kind) => {
const selected = browserScenarioSchema.parse({
...scenario,
secrets:
kind === "declared"
? [{ secret_id: "token", environment_variable: "TOKEN" }]
: [],
});
const protectedSecrets = resolveScenarioSecrets(
selected,
kind === "declared" ? { TOKEN: "pixel-token-729a" } : {},
);
if (kind === "saved")
protectedSecrets.save(
"response.token",
"token",
"pixel-token-729a",
true,
);
const screenshot = vi.fn(async () => Buffer.from("private pixels"));
const result = await capturePlaywrightStepArtifacts({
page: { screenshot } as unknown as Page,
context: {} as BrowserContext,
secrets: protectedSecrets,
requested: new Set(["screenshot"]),
});
expect(result.screenshot).toMatchObject({
state: "missing",
reason: expect.stringContaining("sensitive"),
});
expect(screenshot).not.toHaveBeenCalled();
},
);
it.each(["request", "click", "press"])(
"withholds pixels from the first capture when a future %s response is marked sensitive",
async (action) => {
const save = [{ name: "token", json_pointer: "/token", secret: true }];
const selected = browserScenarioSchema.parse({
...scenario,
actions: [
action === "request"
? {
step_id: "request",
action,
method: "POST",
path: "/api/items",
save,
}
: {
step_id: "ui",
action,
locator: { kind: "css", selector: "#save" },
...(action === "press" ? { key: "Enter" } : {}),
save_response: { method: "POST", path: "/api/items", save },
},
],
});
const screenshot = vi.fn(async () => Buffer.from("private pixels"));
const result = await capturePlaywrightStepArtifacts({
page: { screenshot } as unknown as Page,
context: {} as BrowserContext,
secrets: resolveScenarioSecrets(selected, {}),
requested: new Set(["screenshot"]),
});
expect(result.screenshot).toMatchObject({
state: "missing",
reason: expect.stringContaining("sensitive"),
});
expect(screenshot).not.toHaveBeenCalled();
},
);
it.each(["cookie", "local_storage", "basic_credentials"])(
"withholds pixels before planned %s authentication values are read",
async (source) => {
const value =
source === "basic_credentials"
? {
source,
parts: [
{ source: "literal", value: "user" },
{ source: "cookie", name: "password" },
],
}
: source === "local_storage"
? { source, key: "session" }
: { source, name: "session" };
const selected = browserScenarioSchema.parse({
...scenario,
actions: [
{
step_id: "request",
action: "request",
method: "POST",
path: "/api/items",
headers: [{ name: "Authorization", value }],
},
],
});
const screenshot = vi.fn(async () => Buffer.from("private pixels"));
const result = await capturePlaywrightStepArtifacts({
page: { screenshot } as unknown as Page,
context: {} as BrowserContext,
secrets: resolveScenarioSecrets(selected, {}),
requested: new Set(["screenshot"]),
});
expect(result.screenshot).toMatchObject({ state: "missing" });
expect(screenshot).not.toHaveBeenCalled();
},
);
it("preserves requested pixels when saved response values are not marked sensitive", async () => {
const selected = browserScenarioSchema.parse({
...scenario,
actions: [
{
step_id: "request",
action: "request",
method: "POST",
path: "/api/items",
save: [{ name: "id", json_pointer: "/id", secret: false }],
},
],
});
const pixels = Buffer.from("public pixels");
const screenshot = vi.fn(async () => pixels);
const result = await capturePlaywrightStepArtifacts({
page: { screenshot } as unknown as Page,
context: {} as BrowserContext,
secrets: resolveScenarioSecrets(selected, {}),
requested: new Set(["screenshot"]),
});
expect(result.screenshot).toMatchObject({
state: "captured",
value: { data_base64: pixels.toString("base64") },
});
expect(screenshot).toHaveBeenCalledTimes(1);
});
src/browser/PlaywrightScenarioArtifacts.ts
+11 -2
View File
@@ -43,7 +43,14 @@ const textArtifact = (text: string) => {
};
};
const captureScreenshot = async (page: Page) => {
const SCREENSHOT_PRIVACY_REASON =
"Screenshot withheld because this scenario may display explicitly sensitive values; safe pixel redaction is unavailable.";
const captureScreenshot = async (
page: Page,
secrets: BrowserScenarioSecrets,
) => {
if (secrets.withholdsScreenshots()) return missing(SCREENSHOT_PRIVACY_REASON);
try {
const bytes = await page.screenshot({
type: "png",
@@ -196,7 +203,9 @@ export const capturePlaywrightStepArtifacts = async (input: {
Value | ReturnType<typeof notRequested> | ReturnType<typeof missing>
> => (!requested.has(kind) ? Promise.resolve(notRequested()) : capture());
return browserStepArtifactsSchema.parse({
screenshot: await state("screenshot", () => captureScreenshot(page)),
screenshot: await state("screenshot", () =>
captureScreenshot(page, secrets),
),
dom: await state("dom", () => captureDom({ page, secrets })),
accessibility: await state("accessibility", () =>
captureAccessibility({ page, secrets }),
src/browser/PlaywrightScenarioEvents.test.ts
+509 -6
View File
@@ -110,6 +110,7 @@ const captureNetworkBudgetFixture = async (
});
const secrets = resolveScenarioSecrets(scenario, {});
const redactBytes = vi.spyOn(secrets, "redactBytes");
const redactJson = vi.spyOn(secrets, "redactJsonBody");
const capture = new PlaywrightScenarioEvents({
page,
context,
@@ -155,6 +156,7 @@ const captureNetworkBudgetFixture = async (
return {
result: capture.result(),
redactedByteValues: redactBytes.mock.calls.length,
processedJsonBodies: redactJson.mock.calls.length,
};
};
@@ -174,12 +176,12 @@ describe("PlaywrightScenarioEvents", () => {
expect(
baseline.result.omissions.network_transactions.length,
).toBeGreaterThan(0);
expect(baseline.redactedByteValues).toBe(
2 *
baseline.result.items.filter(
(event) => event.kind === "network-content",
).length,
);
const retainedBodies = baseline.result.items.filter(
(event) =>
event.kind === "network-content" && event.body.state === "captured",
).length;
expect(baseline.processedJsonBodies).toBe(retainedBodies);
expect(baseline.redactedByteValues).toBe(2 * retainedBodies);
expect(raised.result).toMatchObject({
dropped: 0,
omissions: { network_transactions: [] },
@@ -401,3 +403,504 @@ it("caps live descendant listeners and reuses capacity after closure", () => {
listeners.get("websocket")?.(sockets[2048]);
expect(sockets[2048]?.on).toHaveBeenCalledTimes(3);
});
/** Emit the request before attaching the source policy, matching Playwright listener order. */
const sourceTransactionCapture = async (
options: {
fail?: boolean;
followup?: boolean;
metadataOnly?: boolean;
learnDirect?: boolean;
latePageValue?: string;
value?: string;
padding?: number;
metadataLimit?: number;
} = {},
) => {
const listeners = new Map<string, (value: unknown) => void>();
const context = {
on: (name: string, listener: (value: unknown) => void) =>
listeners.set(name, listener),
off: vi.fn(),
} as unknown as BrowserContext;
const page = {
context: () => context,
on: vi.fn(),
off: vi.fn(),
} as unknown as Page;
const value = options.value ?? "credential-82ea";
const url = `https://example.test/exchange?token=${value.repeat(options.padding ?? 1)}&safe=visible`;
const body = JSON.stringify({
token: value,
...(options.latePageValue === undefined
? {}
: { page: options.latePageValue }),
public: "kept",
});
const parsed = browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: "/fixture/browser" },
start_url: { url: "https://example.test/" },
actions: [{ step_id: "click", action: "wait_for_timeout", duration_ms: 1 }],
capture: {
events: ["network"],
network: {
request_body: !options.metadataOnly,
response_body: !options.metadataOnly,
header_values: !options.metadataOnly,
},
...(options.metadataLimit === undefined
? {}
: {
limits: {
max_network_metadata_allocation_bytes: options.metadataLimit,
},
}),
},
});
const secrets = resolveScenarioSecrets(parsed, {});
const capture = new PlaywrightScenarioEvents({
page,
context,
ownsContext: true,
enabled: new Set(["network"]),
network: parsed.capture.network,
limits: parsed.capture.limits,
secrets,
});
const request = {
method: () => "POST",
url: () => url,
resourceType: () => "fetch",
redirectedFrom: () => null,
postDataBuffer: () => Buffer.from(body),
failure: () => null,
sizes: async () => ({ responseBodySize: Buffer.byteLength(body) }),
headers: () => ({
"content-type": "application/json",
"x-source-token": value,
}),
headersArray: async () => [
{ name: "Content-Type", value: "application/json" },
{ name: "X-Source-Token", value },
],
};
listeners.get("request")!(request);
if (!options.metadataOnly)
await vi.waitFor(() =>
expect(capture.result().items).toContainEqual(
expect.objectContaining({ kind: "network-content", phase: "request" }),
),
);
if (options.latePageValue !== undefined)
secrets.admitPageValue("later.cookie", options.latePageValue);
const sourceId = capture
.result()
.items.find((event) => event.kind === "request")!.transaction_id;
// This same-URL transaction has no source policy; unknown source values must not withhold it.
const unrelated = { ...request };
listeners.get("request")!(unrelated);
listeners.get("requestfinished")!(unrelated);
secrets.responseProtector("click", [
{ name: "token", json_pointer: "/token", secret: true },
])?.(request);
if (options.learnDirect)
secrets.responseProtector("click", [
{ name: "token", json_pointer: "/token", secret: true },
])?.(unrelated);
listeners.get("response")!({
request: () => request,
url: () => url,
status: () => 200,
headers: () => ({ "content-type": "application/json" }),
headersArray: async () => [],
body: async () => {
if (options.fail) throw new Error(`Read failed for ${url}`);
return Buffer.from(body);
},
});
if (options.followup) {
const followup = {
...request,
url: () => `https://example.test/followup?token=${value}`,
};
listeners.get("request")!(followup);
listeners.get("response")!({
request: () => followup,
url: () => followup.url(),
status: () => 200,
headers: () => ({ "content-type": "application/json" }),
headersArray: async () => [{ name: "X-Followup", value }],
body: async () => Buffer.from(body),
});
listeners.get("requestfinished")!(followup);
if (!options.metadataOnly)
await vi.waitFor(() =>
expect(
capture
.result()
.items.filter(
(event) =>
event.kind === "network-content" && event.phase === "response",
),
).toHaveLength(1),
);
}
if (options.learnDirect) secrets.save("click.token", "token", value, true);
listeners.get("requestfinished")!(request);
await capture.finish();
return { capture, sourceId, value, url };
};
it("redacts earlier source transaction metadata and request content before publishing", async () => {
const { capture, sourceId, value } = await sourceTransactionCapture();
const source = capture
.result()
.items.filter(
(event) => "transaction_id" in event && event.transaction_id === sourceId,
);
expect(source.filter((event) => event.kind === "request")).toHaveLength(1);
expect(source).toContainEqual(
expect.objectContaining({
kind: "request",
url: expect.objectContaining({
origin: "https://example.test",
query_parameter_names: ["token", "safe"],
redacted: true,
url: expect.stringContaining("safe=visible"),
}),
}),
);
expect(JSON.stringify(source)).not.toContain(value);
for (const event of source)
if (event.kind === "network-content" && event.body.state === "captured") {
const text = Buffer.from(event.body.content, "base64").toString();
expect(text).not.toContain(value);
expect(JSON.parse(text).token).toBe("[REDACTED:page:click.token]");
}
expect(capture.result().items).toContainEqual(
expect.objectContaining({
kind: "request",
transaction_id: expect.not.stringMatching(new RegExp(`^${sourceId}$`)),
url: expect.objectContaining({
origin: "https://example.test",
redacted: true,
url: expect.stringContaining("safe=visible"),
}),
}),
);
const finished = JSON.stringify(capture.result());
await capture.finish();
expect(JSON.stringify(capture.result())).toBe(finished);
});
it("withholds source URLs and earlier content if source values cannot be read", async () => {
const { capture, sourceId, value } = await sourceTransactionCapture({
fail: true,
});
const source = capture
.result()
.items.filter(
(event) => "transaction_id" in event && event.transaction_id === sourceId,
);
expect(JSON.stringify(source)).not.toContain(value);
expect(source).toContainEqual(
expect.objectContaining({
kind: "request",
url: { url: "", origin: null, query_parameter_names: [], redacted: true },
}),
);
expect(source).toContainEqual(
expect.objectContaining({
kind: "network-content",
phase: "request",
body: expect.objectContaining({ state: "unavailable" }),
}),
);
expect(capture.limitations().join(" ")).toContain(
"marked response values were unavailable",
);
});
it("accounts for source metadata redaction expansion before publishing replacements", async () => {
const { capture, sourceId } = await sourceTransactionCapture({
value: "a",
padding: 1000,
metadataLimit: 50_000,
});
expect(capture.result().omissions.network_transactions).toContainEqual(
expect.objectContaining({
transaction_id: sourceId,
reasons: expect.arrayContaining([
"max-network-metadata-allocation-bytes",
]),
}),
);
});
it("re-protects follow-up URLs, headers and both body phases recorded before source secrets were learned", async () => {
const { capture, value } = await sourceTransactionCapture({ followup: true });
const request = capture
.result()
.items.find(
(event) =>
event.kind === "request" && event.url.url.includes("/followup?"),
);
expect(request).toBeDefined();
if (request?.kind !== "request")
throw new Error("Expected actual follow-up request");
const events = capture
.result()
.items.filter(
(event) =>
"transaction_id" in event &&
event.transaction_id === request.transaction_id,
);
expect(JSON.stringify(events)).not.toContain(value);
const bodies = events.flatMap((event) =>
event.kind === "network-content" && event.body.state === "captured"
? [Buffer.from(event.body.content, "base64").toString()]
: [],
);
expect(bodies).toHaveLength(2);
for (const body of bodies) {
expect(body).not.toContain(value);
expect(JSON.parse(body)).toEqual({
token: "[REDACTED:page:click.token]",
public: "kept",
});
}
});
it("protects metadata-only follow-ups when direct saves learn a secret without unambiguous source identity", async () => {
const { capture, value } = await sourceTransactionCapture({
followup: true,
metadataOnly: true,
learnDirect: true,
});
const request = capture
.result()
.items.find(
(event) =>
event.kind === "request" && event.url.url.includes("/followup?"),
);
expect(request).toMatchObject({
url: { redacted: true, origin: "https://example.test" },
});
expect(JSON.stringify(capture.result().items)).not.toContain(value);
expect(
capture.result().items.some((event) => event.kind === "network-content"),
).toBe(false);
});
it("protects earlier non-network event text after a response secret is learned, without network selection", async () => {
const listeners = new Map<string, (value: unknown) => void>();
const socketListeners = new Map<string, (value: unknown) => void>();
const page = {
on: (name: string, listener: (value: unknown) => void) =>
listeners.set(name, listener),
off: vi.fn(),
} as unknown as Page;
const parsed = browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: "/fixture/browser" },
start_url: { url: "https://example.test/" },
actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }],
});
const secrets = resolveScenarioSecrets(parsed, {});
const capture = new PlaywrightScenarioEvents({
page,
enabled: new Set([
"console",
"page-errors",
"websockets",
"frames",
"workers",
"popups",
"downloads",
]),
secrets,
});
const value = "late-secret-829a";
const url = `https://example.test/?token=${value}`;
listeners.get("console")!({
type: () => "log",
text: () => value,
location: () => ({ url }),
});
const error = new Error(value);
error.stack = value;
listeners.get("pageerror")!(error);
listeners.get("websocket")!({
url: () => url,
on: (name: string, listener: (value: unknown) => void) =>
socketListeners.set(name, listener),
off: vi.fn(),
});
socketListeners.get("framesent")!({ payload: value });
listeners.get("framenavigated")!({ url: () => url, name: () => value });
listeners.get("worker")!({ url: () => url, on: vi.fn(), off: vi.fn() });
listeners.get("popup")!({ url: () => url, on: vi.fn(), off: vi.fn() });
listeners.get("download")!({
url: () => url,
suggestedFilename: () => value,
cancel: async () => undefined,
});
expect(capture.result().items.length).toBeGreaterThanOrEqual(8);
expect(JSON.stringify(capture.result().items)).toContain(value);
secrets.save("late.token", "token", value, true);
await capture.finish();
expect(JSON.stringify(capture.result().items)).not.toContain(value);
expect(JSON.stringify(capture.result().items)).toContain("REDACTED");
const finished = JSON.stringify(capture.result());
await capture.finish();
expect(JSON.stringify(capture.result())).toBe(finished);
});
it("accounts for late non-network redaction expansion and reports omissions once", async () => {
const listeners = new Map<string, (value: unknown) => void>();
const page = {
on: (name: string, listener: (value: unknown) => void) =>
listeners.set(name, listener),
off: vi.fn(),
} as unknown as Page;
const parsed = browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: "/fixture/browser" },
start_url: { url: "https://example.test/" },
actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }],
capture: { limits: { max_event_allocation_bytes: 20_000 } },
});
const secrets = resolveScenarioSecrets(parsed, {});
const capture = new PlaywrightScenarioEvents({
page,
enabled: new Set(["console"]),
limits: parsed.capture.limits,
secrets,
});
listeners.get("console")!({
type: () => "log",
text: () => "x".repeat(1000),
location: () => ({ url: "" }),
});
expect(capture.result().retained).toBe(1);
secrets.save("late.token", "token", "x", true);
await capture.finish();
expect(capture.result()).toMatchObject({
retained: 0,
dropped: 1,
omissions: {
by_kind: [
{ kind: "console", count: 1, reasons: ["max-event-allocation-bytes"] },
],
network_transactions: [],
},
});
await capture.finish();
expect(capture.result().dropped).toBe(1);
});
it.each(["page", "basic"])(
"preserves pre-read %s facts while backfilling only response-marked values",
async (kind) => {
for (const responseSave of [false, true]) {
const listeners = new Map<string, (value: unknown) => void>();
const page = {
on: (name: string, listener: (value: unknown) => void) =>
listeners.set(name, listener),
off: vi.fn(),
} as unknown as Page;
const parsed = browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: "/fixture/browser" },
start_url: { url: "https://example.test/" },
actions: [
{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 },
],
});
const secrets = resolveScenarioSecrets(parsed, {});
const capture = new PlaywrightScenarioEvents({
page,
enabled: new Set(["console"]),
secrets,
});
const value =
kind === "page"
? "csrf-page-value-729d"
: Buffer.from("user:password").toString("base64");
const message = `${value} response-value-82ea`;
const emit = () =>
listeners.get("console")!({
type: () => "log",
text: () => message,
location: () => ({ url: "" }),
});
emit();
if (kind === "page") secrets.admitPageValue("request.cookie", value);
else secrets.basicCredential("request.auth", "user", "password");
emit();
if (responseSave)
secrets.save("response.token", "token", "response-value-82ea", true);
await capture.finish();
const events = capture.result().items;
expect(events).toHaveLength(2);
expect(events[0]).toMatchObject({
kind: "console",
text: expect.stringContaining(value),
});
expect(events[1]).toMatchObject({
kind: "console",
text: expect.not.stringContaining(value),
});
expect(JSON.stringify(events).includes("response-value-82ea")).toBe(
!responseSave,
);
}
},
);
it("preserves a response field recorded before a later page-value read while backfilling response secrets", async () => {
const pageValue = "csrf-page-value-729d";
const { capture, sourceId, value } = await sourceTransactionCapture({
latePageValue: pageValue,
});
const first = capture
.result()
.items.find(
(event) =>
event.kind === "network-content" &&
event.transaction_id === sourceId &&
event.phase === "request",
);
expect(first).toBeDefined();
if (first?.kind !== "network-content" || first.body.state !== "captured")
throw new Error("Expected retained earlier request bytes");
const body = JSON.parse(Buffer.from(first.body.content, "base64").toString());
expect(body.page).toBe(pageValue);
expect(body.token).toBe("[REDACTED:page:click.token]");
expect(body.token).not.toContain(value);
});
it("promotes an already held page value to response scope without changing its first marker", () => {
const parsed = browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: "/fixture/browser" },
start_url: { url: "https://example.test/" },
actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }],
});
const secrets = resolveScenarioSecrets(parsed, {});
const value = "existing-page-value-82ea";
secrets.admitPageValue("cookie.token", value);
expect(secrets.redact(value, "response")).toBe(value);
expect(secrets.hasLearnedResponseSecrets()).toBe(false);
secrets.save("response.token", "token", value, true);
expect(secrets.hasLearnedResponseSecrets()).toBe(true);
expect(secrets.redact(value, "response")).toBe(
"[REDACTED:page:cookie.token]",
);
expect(secrets.redactBytes(Buffer.from(value), "response").toString()).toBe(
"[REDACTED:page:cookie.token]",
);
secrets.admitPageValue("later.unrelated", "unrelated-page-value-729d");
expect(secrets.redact("unrelated-page-value-729d", "response")).toBe(
"unrelated-page-value-729d",
);
expect(secrets.redact(value, "response")).toBe(
"[REDACTED:page:cookie.token]",
);
});
src/browser/PlaywrightScenarioEvents.ts
+309 -1
View File
@@ -25,6 +25,7 @@ import type {
BrowserScenarioEventOmissions,
} from "../domain/browserScenarioCaptureValues.js";
import { DEFAULT_BROWSER_SCENARIO_CAPTURE_LIMITS } from "../domain/browserScenarioCaptureLimits.js";
import { captureNetworkBody } from "./network/PlaywrightNetworkContent.js";
import { PlaywrightScenarioNetwork } from "./network/PlaywrightScenarioNetwork.js";
type EventName =
@@ -71,6 +72,30 @@ const allocationCost = (value: unknown, remaining: number): number => {
return cost;
};
const retainedEventCost = (
event: BrowserScenarioEvent,
limit: number,
): number => {
const { sequence: _sequence, step_index: _stepIndex, ...unindexed } = event;
return allocationCost(unindexed, limit) + 128;
};
type TextEvent = Exclude<
BrowserScenarioEvent,
{ readonly transaction_id: string }
>;
/** Count parsed text without allocating another serialization or copying bodies. */
const eventTextUnits = (value: unknown): number => {
if (typeof value === "string") return value.length;
if (value === null || typeof value !== "object") return 0;
let units = 0;
for (const key in value)
if (Object.hasOwn(value, key))
units += eventTextUnits((value as Record<string, unknown>)[key]);
return units;
};
type EventKind = BrowserScenarioEvent["kind"];
/** Arrival-ordered Playwright event capture with current-step attribution. */
export class PlaywrightScenarioEvents {
@@ -100,6 +125,8 @@ export class PlaywrightScenarioEvents {
private stopNetwork: (() => void) | undefined;
private readonly stopListeners = new Map<object, Set<() => void>>();
private sealed = false;
private sourcePrivacyIncomplete = false;
private eventPrivacyFinalized = false;
constructor(options: EventCaptureOptions) {
const network = options.network ?? {
@@ -192,7 +219,11 @@ export class PlaywrightScenarioEvents {
try {
await this.network?.finish();
} finally {
this.sealed = true;
try {
await this.finalizeEventPrivacy();
} finally {
this.sealed = true;
}
}
}
@@ -203,6 +234,10 @@ export class PlaywrightScenarioEvents {
`Popup ${family} events before Page discovery are unavailable; later events are retained.`,
);
const gaps = [...popupGaps, ...(this.network?.limitations() ?? [])];
if (this.sourcePrivacyIncomplete)
gaps.push(
"Source transaction URLs or content were withheld because marked response values were unavailable.",
);
if (this.dropped > 0)
gaps.push(
"Browser event collection reached a selected resource budget; per-kind omissions and affected network transaction IDs are listed in events.omissions, and event coverage is incomplete.",
@@ -255,6 +290,279 @@ export class PlaywrightScenarioEvents {
};
}
/** Apply final known secrets to retained event text independently of source readiness. */
private async finalizeEventPrivacy(): Promise<void> {
if (this.eventPrivacyFinalized) return;
const sources = this.network?.sourcePrivacy() ?? new Map<string, boolean>();
const learnedSecrets = this.secrets.hasLearnedResponseSecrets();
if (sources.size === 0 && !learnedSecrets) return;
let retained = 0;
for (const original of this.items) {
const sourceReady =
"transaction_id" in original
? sources.get(original.transaction_id)
: undefined;
if (sourceReady === undefined && !learnedSecrets) {
this.items[retained++] = original;
continue;
}
const ready = sourceReady ?? true;
if (!ready) this.sourcePrivacyIncomplete = true;
const oldCost = retainedEventCost(
original,
this.limits.max_event_allocation_bytes,
);
this.retainedCost -= oldCost;
const available = Math.min(
this.limits.max_event_allocation_bytes,
this.limits.max_total_event_allocation_bytes - this.retainedCost,
);
const extra = eventTextUnits(
original.kind === "network-content"
? {
headers: original.headers,
message:
original.body.state === "unavailable"
? original.body.message
: "",
}
: original,
);
let reason: BrowserScenarioEventBudgetReason | undefined =
"transaction_id" in original &&
this.secrets.redactionAllocationCost(extra, "response") + 4096 >
this.limits.max_network_metadata_allocation_bytes
? "max-network-metadata-allocation-bytes"
: undefined;
const bodyCopy =
original.kind === "network-content" &&
original.body.state === "captured"
? Math.max(
this.secrets.redactionAllocationCost(
original.body.bytes,
"response",
),
Math.ceil(original.body.bytes / 3) * 4 * 8 + 2,
)
: 0;
const workingCost =
oldCost +
this.secrets.redactionAllocationCost(extra, "response") +
bodyCopy +
4096;
reason ??=
workingCost > this.limits.max_event_allocation_bytes
? "max-event-allocation-bytes"
: workingCost > available
? "max-total-event-allocation-bytes"
: undefined;
let event: BrowserScenarioEvent = original;
if (reason === undefined) {
event =
"transaction_id" in original
? await this.protectNetworkEvent(
original,
ready,
Math.max(1, available),
)
: this.protectTextEvent(original);
const cost = retainedEventCost(
event,
this.limits.max_event_allocation_bytes,
);
reason =
cost > this.limits.max_event_allocation_bytes
? "max-event-allocation-bytes"
: cost > available
? "max-total-event-allocation-bytes"
: undefined;
if (reason === undefined) {
this.retainedCost += cost;
this.items[retained++] = event;
continue;
}
}
this.omitEvent(original.kind, reason);
if ("transaction_id" in original)
this.omitNetwork(original.transaction_id, original.kind, reason);
}
this.items.length = retained;
this.eventPrivacyFinalized = true;
}
private protectUrl(value: SanitizedBrowserUrl): SanitizedBrowserUrl {
const url = this.secrets.sanitizeUrl(value.url, "response");
return { ...url, redacted: url.redacted || value.redacted };
}
private protectTextEvent(event: TextEvent): TextEvent {
switch (event.kind) {
case "console":
return {
...event,
url: event.url === null ? null : this.protectUrl(event.url),
level: this.secrets.redact(event.level, "response"),
text: this.secrets.redact(event.text, "response"),
};
case "page-error":
return {
...event,
message: this.secrets.redact(event.message, "response"),
stack:
event.stack === null
? null
: this.secrets.redact(event.stack, "response"),
};
case "websocket-frame-sent":
case "websocket-frame-received":
return event.payload_type === "text"
? {
...event,
url: this.protectUrl(event.url),
payload_text: this.secrets.redact(event.payload_text, "response"),
}
: { ...event, url: this.protectUrl(event.url) };
case "websocket-opened":
case "websocket-closed":
return { ...event, url: this.protectUrl(event.url) };
case "download-cancelled":
return {
...event,
url: this.protectUrl(event.url),
suggested_filename: this.secrets.redact(
event.suggested_filename,
"response",
),
};
default:
return {
...event,
url: event.url === null ? null : this.protectUrl(event.url),
name:
event.name === null
? null
: this.secrets.redact(event.name, "response"),
};
}
}
private async protectNetworkEvent(
event: BrowserScenarioEvent,
ready: boolean,
available: number,
): Promise<BrowserScenarioEvent> {
const message =
"Source transaction data was withheld because its marked response fields could not be read safely.";
if (event.kind === "network-content") {
if (!ready) {
return {
...event,
headers:
event.headers.state === "not_requested"
? event.headers
: { state: "unavailable", message },
body:
event.body.state === "not_requested" ||
event.body.state === "not_exposed"
? event.body
: { state: "unavailable", reason: "body-read-failed", message },
};
}
const headers: BrowserNetworkHeaders =
event.headers.state === "unavailable"
? {
...event.headers,
message: this.secrets.redact(event.headers.message, "response"),
}
: event.headers.state !== "captured"
? event.headers
: {
state: "captured",
items: event.headers.items.map((item) => {
const name = this.secrets.redact(item.name, "response"),
value =
item.value === null
? null
: this.secrets.redact(item.value, "response");
return {
...item,
name,
value,
redacted:
item.redacted ||
name !== item.name ||
value !== item.value,
};
}),
};
let body = event.body;
if (body.state === "unavailable")
body = {
...body,
message: this.secrets.redact(body.message, "response"),
};
if (body.state === "captured") {
const prior = body;
const source = {
headers: (): Record<string, string> =>
prior.media_type === null
? {}
: { "content-type": prior.media_type },
headersArray: async () => [],
};
const protectedBody = await captureNetworkBody(
async () => Buffer.from(prior.content, "base64"),
event.phase,
source,
this.secrets,
{
redactionScope: "response",
timeoutMs: 5000,
maxBodyBytes: this.limits.max_network_body_bytes,
maxBodyAllocationBytes: Math.min(
this.limits.max_network_body_allocation_bytes,
available,
),
maxHeaderAllocationBytes:
this.limits.max_network_header_allocation_bytes,
maxMetadataAllocationBytes:
this.limits.max_network_metadata_allocation_bytes,
},
);
body =
protectedBody.state === "omitted"
? { state: "unavailable", reason: "body-read-failed", message }
: protectedBody;
if (body.state === "captured")
body = { ...body, redacted: body.redacted || prior.redacted };
}
return { ...event, headers, body };
}
if (!("transaction_id" in event) || !("url" in event)) return event;
const url = ready
? this.protectUrl(event.url)
: { url: "", origin: null, query_parameter_names: [], redacted: true };
const metadata = {
url: { ...url, redacted: url.redacted || event.url.redacted },
method: this.secrets.redact(event.method, "response"),
resource_type: this.secrets.redact(event.resource_type, "response"),
header_names: event.header_names.map((name) =>
ready
? this.secrets.redact(name, "response")
: "[REDACTED:source-unavailable]",
),
};
return event.kind === "request-failed"
? {
...event,
...metadata,
failure: ready
? this.secrets.redact(event.failure, "response")
: message,
}
: { ...event, ...metadata };
}
private push(event: UnindexedEvent): BrowserScenarioEventAdmission {
this.sequence += 1;
const cost =
src/browser/PlaywrightScenarioRequest.test.ts
+1229 -2
View File
File diff suppressed because it is too large. Load diff
src/browser/PlaywrightScenarioRequest.ts
+521 -50
View File
@@ -1,9 +1,25 @@
import type { Page, Route } from "playwright-core";
import { constants as bufferConstants } from "node:buffer";
import { getHeapStatistics } from "node:v8";
import { jsonStringByteLength } from "../domain/jsonStringByteLength.js";
import { jsonValueByteLength } from "../domain/jsonValueByteLength.js";
import type { JsonValue } from "../domain/jsonValue.js";
import {
MAX_SAVED_RESPONSE_CHARACTERS,
MAX_SAVED_RESPONSE_BYTES,
responsePointerValue,
readResponseValues,
readBoundedResponseBody,
} from "./BrowserScenarioResponseValues.js";
import type { Page, Request, Route } from "playwright-core";
import {
HEADER_FIELD_VALUE,
SAVED_PLACEHOLDER,
type BrowserScenarioAction,
type BrowserScenarioCredentialPart,
type BrowserScenarioRequestHeader,
type BrowserScenarioSaveEntry,
type BrowserScenarioSaveResponse,
} from "../domain/browserScenarioValues.js";
import type { BrowserScenarioSecrets } from "./BrowserScenarioSecrets.js";
@@ -11,7 +27,7 @@ type RequestAction = Extract<BrowserScenarioAction, { action: "request" }>;
/** What the page's fetch returned; a redirect is not followed, so its status is unknown. */
type PageResponse =
| { readonly status: number }
| { readonly status: number; readonly body?: string }
| { readonly redirected: true }
| { readonly left: string }
| { readonly error: string };
@@ -31,39 +47,29 @@ const originOf = (url: string): string | undefined => {
/** The string or number at a JSON pointer, as text. */
const pointerText = (value: unknown, pointer: string): string | undefined => {
let current = value;
if (pointer !== "")
for (const token of pointer.slice(1).split("/")) {
const key = token.replaceAll("~1", "/").replaceAll("~0", "~");
if (Array.isArray(current) && /^(?:0|[1-9][0-9]*)$/u.test(key))
current = current[Number(key)];
else if (
typeof current === "object" &&
current !== null &&
Object.hasOwn(current, key)
)
current = (current as Record<string, unknown>)[key];
else return undefined;
}
return typeof current === "string" || typeof current === "number"
? String(current)
const found = responsePointerValue(value, pointer);
return typeof found === "string" || typeof found === "number"
? String(found)
: undefined;
};
/** Resolve one header value; values read from the page are redacted from later records. */
const headerValue = async (
step: {
readonly page: Page;
readonly url: URL;
readonly stepId: string;
readonly secrets: BrowserScenarioSecrets;
},
header: BrowserScenarioRequestHeader,
interface PageValueStep {
readonly page: Page;
readonly url: URL;
readonly stepId: string;
readonly secrets: BrowserScenarioSecrets;
}
/** Read a value from the page, a cookie or a local storage entry, and redact it from later records under a label. */
const readPageValue = async (
step: PageValueStep,
value: Extract<
BrowserScenarioCredentialPart,
{ source: "cookie" | "local_storage" }
>,
label: string,
): Promise<string> => {
const { page, url, stepId, secrets } = step;
const { value } = header;
if (value.source === "literal" || value.source === "secret")
return secrets.value(value);
let read: string | undefined;
if (value.source === "cookie") {
const cookies = await page.context().cookies(url.href);
@@ -113,10 +119,47 @@ const headerValue = async (
);
}
}
secrets.admitPageValue(`${stepId}.${header.name}`, read);
secrets.admitPageValue(label, read);
return read;
};
/** One part of Basic credentials: a literal, secret or saved value, or a value read from the page. */
const credentialPart = (
step: PageValueStep,
part: BrowserScenarioCredentialPart,
label: string,
): Promise<string> | string =>
part.source === "cookie" || part.source === "local_storage"
? readPageValue(step, part, label)
: step.secrets.value(part);
/** Resolve one header value; values read from the page, and credentials built from them, are redacted from later records. */
const headerValue = async (
step: PageValueStep,
header: BrowserScenarioRequestHeader,
): Promise<string> => {
const { stepId, secrets } = step;
const { value } = header;
const label = `${stepId}.${header.name}`;
if (
value.source === "literal" ||
value.source === "secret" ||
value.source === "saved"
)
return secrets.value(value);
if (value.source === "basic_credentials") {
const [username, password] = value.parts;
const user = await credentialPart(step, username!, `${label}.username`);
if (user.includes(":"))
throw new ScenarioRequestError(
`Request step ${stepId} has a Basic username with a colon, which the server would read as the end of the username.`,
);
const pass = await credentialPart(step, password!, `${label}.password`);
return secrets.basicCredential(label, user, pass);
}
return readPageValue(step, value, label);
};
/** Append declared query entries after any literal query the path carries, leaving that query as written. */
const withQuery = (
url: URL,
@@ -148,6 +191,8 @@ interface PageRequest {
readonly headers: [string, string][];
readonly body: string | undefined;
readonly deadline: number | undefined;
/** Return the response text up to this many characters, for a step that saves values from it. */
readonly readLimit: number | undefined;
}
/** Send the request with the page's own fetch, from the page's main world, and read the whole answer. */
@@ -191,6 +236,31 @@ const sendFromPage = (
signal: controller.signal,
...(request.body === undefined ? {} : { body: request.body }),
});
if (answer.type === "opaqueredirect") return { redirected: true };
if (request.readLimit !== undefined) {
// Keep the text only up to the limit, so a large answer is never held whole.
const reader = answer.body?.getReader();
if (reader === undefined) return { status: answer.status, body: "" };
const decoder = new TextDecoder("utf-8", {
fatal: true,
ignoreBOM: true,
});
let text = "";
try {
for (;;) {
const chunk = await reader.read();
if (chunk.done) break;
text += decoder.decode(chunk.value, { stream: true });
if (text.length > request.readLimit) {
await reader.cancel();
return { status: answer.status };
}
}
} finally {
reader.releaseLock();
}
return { status: answer.status, body: text + decoder.decode() };
}
const reader = answer.body?.getReader();
if (reader !== undefined)
try {
@@ -200,10 +270,9 @@ const sendFromPage = (
} finally {
reader.releaseLock();
}
return answer.type === "opaqueredirect"
? { redirected: true }
: { status: answer.status };
return { status: answer.status };
} catch (cause: unknown) {
controller.abort();
return {
error: cause instanceof Error ? cause.message : String(cause),
};
@@ -234,7 +303,7 @@ export const performScenarioRequest = async (input: {
throw new ScenarioRequestError(
`Request step ${action.step_id} runs from a page on the start origin ${startOrigin}; navigate back to it with a goto step first.`,
);
const url = new URL(action.path, startOrigin);
const url = new URL(withSavedPlaceholders(action.path, secrets), startOrigin);
if (url.origin !== startOrigin)
throw new ScenarioRequestError(
`Request step ${action.step_id} must stay on the start origin ${startOrigin}.`,
@@ -245,45 +314,100 @@ export const performScenarioRequest = async (input: {
);
const step = { page, url, stepId: action.step_id, secrets };
const headers: [string, string][] = [];
for (const header of action.headers) {
const value = `${header.prefix ?? ""}${await headerValue(step, header)}`;
if (!HEADER_FIELD_VALUE.test(value))
throw new ScenarioRequestError(
`Request step ${action.step_id}: the ${header.name} value holds characters a header cannot carry, such as a line break.`,
);
headers.push([header.name, value]);
}
const envelopeLimit = requestSerializationAllowance();
let headerBytes = 2;
const headerValues = new Map<string, number>();
const appendHeader = (name: string, value: string): void => {
const knownSize = headerValues.get(value);
const size =
knownSize ?? jsonStringByteLength(value, envelopeLimit - headerBytes);
headerBytes +=
(headers.length === 0 ? 0 : 1) +
jsonValueByteLength([name, null], envelopeLimit - headerBytes) -
4 +
size;
if (headerBytes > envelopeLimit) throw requestEnvelopeError();
if (knownSize === undefined) {
if (!HEADER_FIELD_VALUE.test(value))
throw new ScenarioRequestError(
`Request step ${action.step_id}: the ${name} value holds characters a header cannot carry, such as a line break.`,
);
headerValues.set(value, size);
}
headers.push([name, value]);
};
for (const header of action.headers)
appendHeader(
header.name,
`${header.prefix ?? ""}${await headerValue(step, header)}`,
);
const hasContentType = headers.some(
([name]) => name.toLowerCase() === "content-type",
);
if (action.json !== undefined && !hasContentType)
headers.push(["content-type", "application/json"]);
appendHeader("content-type", "application/json");
// A step the boundary abandoned, or whose time ran out while values were read, sends nothing.
const remaining = deadline === undefined ? undefined : deadline - Date.now();
if (signal?.aborted === true || (remaining !== undefined && remaining <= 0))
throw new ScenarioRequestError(
`Request step ${action.step_id} ran out of time before its request was sent.`,
);
if (action.json !== undefined) admitSavedJson(action.json, secrets);
const json =
action.json === undefined ? undefined : withSavedJson(action.json, secrets);
const request = {
url: target,
origin: startOrigin,
method: action.method,
headers,
body: action.json === undefined ? undefined : JSON.stringify(action.json),
body: json === undefined ? undefined : JSON.stringify(json),
deadline,
readLimit:
action.save.length > 0 ? MAX_SAVED_RESPONSE_CHARACTERS : undefined,
};
if (
jsonValueByteLength(
{
url: target,
origin: startOrigin,
method: action.method,
headers: [],
body: request.body ?? null,
deadline: deadline ?? null,
readLimit: request.readLimit ?? null,
},
envelopeLimit,
) +
headerBytes -
2 >
envelopeLimit
)
throw requestEnvelopeError();
// The browser can queue evaluate behind a busy page. Keep a host-side gate
// until that evaluation settles, including after the step boundary stops
// waiting. A queued fetch then observes live cancellation at dispatch.
const dispatchUrl = new URL(target);
dispatchUrl.hash = "";
const matches = (url: URL): boolean => url.href === dispatchUrl.href;
const protectResponse = secrets.responseProtector(
action.step_id,
action.save,
);
const matchesRequest = (sent: Request): boolean =>
sent.method() === request.method &&
sent.postData() === (request.body ?? null);
let observedRequest: Request | undefined;
let ambiguousRequest = false;
const watch = (sent: Request): void => {
if (sent.url() === dispatchUrl.href && matchesRequest(sent)) {
if (observedRequest === undefined) observedRequest = sent;
else if (observedRequest !== sent) ambiguousRequest = true;
protectResponse?.(sent);
}
};
const gate = async (route: Route): Promise<void> => {
const sent = route.request();
if (
sent.method() !== request.method ||
sent.postData() !== (request.body ?? null)
) {
if (!matchesRequest(sent)) {
await route.fallback();
return;
}
@@ -292,13 +416,17 @@ export const performScenarioRequest = async (input: {
(deadline !== undefined && Date.now() >= deadline)
)
await route.abort("aborted");
else await route.fallback();
else {
await route.fallback();
}
};
await page.route(matches, gate);
if (protectResponse !== undefined) page.on("request", watch);
let response: PageResponse;
try {
response = await sendFromPage(page, request);
} finally {
if (protectResponse !== undefined) page.off("request", watch);
await page.unroute(matches, gate);
}
if ("error" in response)
@@ -320,4 +448,347 @@ export const performScenarioRequest = async (input: {
throw new ScenarioRequestError(
`Request step ${action.step_id} answered ${String(response.status)}; expected ${action.expected_status.join(" or ")}.`,
);
if (action.save.length > 0) {
saveValues(action.step_id, response.body, action.save, secrets);
if (observedRequest !== undefined && !ambiguousRequest)
secrets.markResponseSecretsReady(observedRequest);
}
};
/** A URL path segment a URL parser drops or folds into its parent: `.`, `..` or a percent-encoded spelling of one. */
const DOT_SEGMENT = /^(?:\.|%2e){1,2}$/iu;
/** A path or URL with its `placeholder`s filled, each value percent-encoded; a request path's are spelled as written. A filled path segment a URL path would drop as a dot segment fails the step. */
export const withSavedPlaceholders = (
text: string,
secrets: BrowserScenarioSecrets,
placeholder: RegExp = SAVED_PLACEHOLDER,
): string => {
const limit = requestSerializationAllowance();
let expandedUnits = text.length;
if (expandedUnits > limit)
throw new ScenarioRequestError(
"Saved URL exceeds the current heap/string serialization allowance.",
);
const encoded = new Map<string, string>();
const fill = (part: string): string =>
part.replace(placeholder, (matched, name: string) => {
let replacement = encoded.get(name);
if (replacement === undefined) {
const raw = String(secrets.savedValue(name));
// A UTF-16 unit needs at most nine ASCII characters under URI encoding.
if (raw.length * 9 > limit)
throw new ScenarioRequestError(
"Saved URL encoding exceeds the current heap/string serialization allowance.",
);
replacement = encodeURIComponent(raw);
encoded.set(name, replacement);
}
expandedUnits += replacement.length - matched.length;
if (expandedUnits > limit)
throw new ScenarioRequestError(
"Expanded saved URL exceeds the current heap/string serialization allowance.",
);
return replacement;
});
const pathEnd = text.search(/[?#]/u);
const path = pathEnd < 0 ? text : text.slice(0, pathEnd);
// WHATWG special URLs treat literal backslashes as path separators too.
const filledPath = path.replace(/[^/\\]+/gu, (segment) => {
const filled = fill(segment);
if (filled !== segment && DOT_SEGMENT.test(filled))
throw new Error(
`A saved value makes the path segment ${JSON.stringify(segment)} a dot segment, which a URL path drops, so the step cannot send it.`,
);
return filled;
});
return filledPath + (pathEnd < 0 ? "" : fill(text.slice(pathEnd)));
};
/** Whether a request a UI step triggered is the one its `save_response` names: method, start origin, and path with `{*}` matching one segment. */
/** A `save_response` path's segments as a browser spells a URL path, percent-encoded as its URL parser would, `{*}` kept. */
const pathPattern = (path: string): readonly string[] => {
const pathname = path.split("?")[0]!;
let wildcard = "~rea-wildcard~";
while (pathname.includes(wildcard)) wildcard += "~";
const marked = pathname
.split(/[\\/]/u)
.map((segment) => (segment === "{*}" ? wildcard : segment))
.join("/");
return new URL(marked, "http://path.invalid").pathname
.split("/")
.map((segment) => (segment === wildcard ? "{*}" : segment));
};
const namesResponse = (
spec: BrowserScenarioSaveResponse,
pattern: readonly string[],
startOrigin: string,
request: { readonly method: string; readonly url: string },
): boolean => {
if (request.method !== spec.method) return false;
const url = URL.parse(request.url);
if (url === null || url.origin !== startOrigin) return false;
const segments = url.pathname.split("/");
return (
segments.length === pattern.length &&
pattern.every((part, index) => part === "{*}" || part === segments[index])
);
};
/** Settle as the work does, or fail the step once its deadline passes or it is cancelled, so late work saves nothing. */
const beforeStepEnds = <T>(
work: Promise<T>,
stepId: string,
deadline: number | undefined,
signal: AbortSignal | undefined,
): Promise<T> =>
new Promise<T>((resolve, reject) => {
let timer: ReturnType<typeof setTimeout> | undefined;
const settle = (): void => {
if (timer !== undefined) clearTimeout(timer);
signal?.removeEventListener("abort", end);
};
const end = (): void => {
settle();
reject(
new ScenarioRequestError(
`Step ${stepId} ended before its saved response was read, so nothing was saved.`,
),
);
};
work.catch(() => undefined);
const remaining =
deadline === undefined ? undefined : deadline - Date.now();
if (
signal?.aborted === true ||
(remaining !== undefined && remaining <= 0)
) {
end();
return;
}
if (remaining !== undefined) timer = setTimeout(end, remaining);
signal?.addEventListener("abort", end, { once: true });
work.then(
(value) => {
settle();
resolve(value);
},
(cause: unknown) => {
settle();
reject(cause);
},
);
});
/** A UI step's wait for the response its `save_response` names: `saved` waits and saves, `stop` stops watching requests. */
export interface SavedResponseWait {
readonly saved: () => Promise<void>;
readonly stop: () => void;
}
/**
* Observe a new matching response while the UI step waits; matching does not prove action causality.
* Requests already running are excluded. Competing requests observed before binding are ambiguous.
* The step still requires a 2xx JSON response within its timeout or the page's default timeout.
*/
export const awaitSavedResponse = (
page: Page,
step: {
readonly stepId: string;
readonly spec: BrowserScenarioSaveResponse;
readonly startOrigin: string;
readonly timeout: number | undefined;
readonly secrets: BrowserScenarioSecrets;
readonly signal?: AbortSignal | undefined;
},
): SavedResponseWait => {
const { stepId, spec, startOrigin, timeout, secrets, signal } = step;
const deadline = timeout === undefined ? undefined : Date.now() + timeout;
const pattern = pathPattern(spec.path);
const started = new Set<Request>();
const protectResponse = secrets.responseProtector(stepId, spec.save);
const watch = (request: Request): void => {
if (
namesResponse(spec, pattern, startOrigin, {
method: request.method(),
url: request.url(),
})
) {
if (started.size < 2) started.add(request);
protectResponse?.(request);
}
};
page.on("request", watch);
const waiting = page.waitForResponse(
(response) => started.has(response.request()),
timeout === undefined ? {} : { timeout },
);
// The step may fail before it waits; the rejection is reported by the wait, or not at all.
waiting.catch(() => undefined);
const saved = async (): Promise<void> => {
let response: Awaited<typeof waiting>;
try {
response = await beforeStepEnds(waiting, stepId, deadline, signal);
} catch {
throw new ScenarioRequestError(
`Step ${stepId}: no ${spec.method} ${spec.path} response arrived before the step's timeout, so nothing was saved.`,
);
}
const assertUnambiguous = (): void => {
if (started.size !== 1)
throw new ScenarioRequestError(
`Step ${stepId}: multiple matching requests make the response ambiguous, so nothing was saved. Narrow the response path or record without competing background requests.`,
);
};
assertUnambiguous();
if (response.status() < 200 || response.status() > 299)
throw new ScenarioRequestError(
`Step ${stepId}: the ${spec.method} ${spec.path} response answered ${String(response.status())}; values are saved only from a 2xx response.`,
);
const body = await beforeStepEnds(
readBoundedResponseBody(response, MAX_SAVED_RESPONSE_BYTES),
stepId,
deadline,
signal,
);
if (!body.ok)
throw new ScenarioRequestError(`Step ${stepId}: ${body.error.message}`);
assertUnambiguous();
saveValues(stepId, body.value, spec.save, secrets);
secrets.markResponseSecretsReady(response.request());
};
return { saved, stop: () => page.off("request", watch) };
};
const requestEnvelopeError = (): ScenarioRequestError =>
new ScenarioRequestError(
"Expanded request envelope exceeds the current heap/string serialization allowance; reduce repeated values or raise the parent Node heap and restart.",
);
/** Allow for the expanded tree, stringify, Playwright transport and browser-side copies. */
const REQUEST_JSON_ALLOCATION_FACTOR = 16;
const requestSerializationAllowance = (): number =>
Math.min(
bufferConstants.MAX_STRING_LENGTH,
Math.floor(
getHeapStatistics().total_available_size / REQUEST_JSON_ALLOCATION_FACTOR,
),
);
/** Count serialization expansion without allocating substituted strings or their serialized copies. */
const admitSavedJson = (
value: JsonValue,
secrets: BrowserScenarioSecrets,
): void => {
const limit = requestSerializationAllowance();
let bytes = 0;
const savedSizes = new Map<string, number>();
const charge = (size: number): void => {
bytes += size;
if (bytes > limit)
throw new ScenarioRequestError(
`Expanded request JSON exceeds its ${String(limit)}-byte serialization allowance, derived from current V8 heap headroom and the string limit. Reduce payload/repeated saved references, or raise the parent Node heap and restart.`,
);
};
const stringPart = (text: string): void =>
charge(jsonStringByteLength(text, limit - bytes + 2) - 2);
const visit = (item: JsonValue): void => {
if (typeof item === "string") {
const first = item.matchAll(SAVED_PLACEHOLDER).next().value;
if (first !== undefined && first[0] === item) {
const saved = secrets.savedValue(first[1]!);
if (typeof saved === "number") {
charge(jsonValueByteLength(saved));
return;
}
}
charge(2);
let offset = 0;
for (const match of item.matchAll(SAVED_PLACEHOLDER)) {
stringPart(item.slice(offset, match.index));
const name = match[1]!;
let size = savedSizes.get(name);
if (size === undefined) {
size =
jsonStringByteLength(
String(secrets.savedValue(name)),
limit - bytes + 2,
) - 2;
savedSizes.set(name, size);
}
charge(size);
offset = match.index + match[0].length;
}
stringPart(item.slice(offset));
} else if (Array.isArray(item)) {
charge(2 + Math.max(0, item.length - 1));
for (const child of item) visit(child);
} else if (item !== null && typeof item === "object") {
charge(2);
let count = 0;
for (const key in item) {
if (!Object.hasOwn(item, key)) continue;
charge(
(count++ === 0 ? 0 : 1) +
jsonStringByteLength(key, limit - bytes) +
1,
);
visit(item[key]!);
}
} else charge(jsonValueByteLength(item));
};
visit(value);
};
/**
* The step's JSON body with its `{saved:name}` placeholders filled: a string that is only a placeholder becomes the
* saved value, keeping its JSON type, and a placeholder inside a longer string is filled as text.
*/
const withSavedJson = (
value: unknown,
secrets: BrowserScenarioSecrets,
): unknown => {
if (typeof value === "string") {
const [whole] = [...value.matchAll(SAVED_PLACEHOLDER)];
if (whole !== undefined && whole[0] === value)
return secrets.savedValue(whole[1]!);
return value.replace(SAVED_PLACEHOLDER, (_placeholder, name: string) =>
String(secrets.savedValue(name)),
);
}
if (Array.isArray(value))
return value.map((item) => withSavedJson(item, secrets));
if (value !== null && typeof value === "object")
return Object.fromEntries(
Object.entries(value).map(([key, item]) => [
key,
withSavedJson(item, secrets),
]),
);
return value;
};
/** Save the values a step names from a JSON response, for later steps. */
const saveValues = (
stepId: string,
body: string | Uint8Array | undefined,
entries: readonly BrowserScenarioSaveEntry[],
secrets: BrowserScenarioSecrets,
): void => {
const selected = readResponseValues(body, entries, "fail");
if (!selected.ok)
throw new ScenarioRequestError(`Step ${stepId}: ${selected.error.message}`);
for (const {
entry: { name, secret },
value,
} of selected.value.values) {
try {
secrets.save(`${stepId}.${name}`, name, value, secret);
} catch (cause) {
throw new ScenarioRequestError(
`Step ${stepId}: ${cause instanceof Error ? cause.message : String(cause)}`,
);
}
}
};
src/browser/network/PlaywrightNetworkContent.test.ts
+86 -1
View File
@@ -1,4 +1,4 @@
import { expect, it } from "vitest";
import { expect, it, vi } from "vitest";
import { browserScenarioSchema } from "../../domain/browserScenario.js";
import { resolveScenarioSecrets } from "../../../tests/fixtures/browserScenarioSecrets.js";
import {
@@ -76,3 +76,88 @@ it("charges discarded credential header values only for their retained projectio
});
expect(await read("X-Retained")).toMatchObject({ state: "unavailable" });
});
it("redacts later JSON spellings of saved secrets without reserializing unmarked facts", async () => {
const held = resolveScenarioSecrets(scenario, { SECRET: "declared-value" });
held.save("login.token", "token", "source-token", true);
held.save("login.code", "code", 1000, true);
const body = '{ "token":"\\u0073ource-token", "code":1e3, "unknown":1e-400 }';
const captured = await captureNetworkBody(
async () => Buffer.from(body),
"response",
{
headers: () => ({
"content-type": "application/problem+json; charset=utf-8",
}),
headersArray: async () => [],
},
held,
{ ...options, maxBodyBytes: 4096 },
);
if (captured.state !== "captured") throw new Error("Expected captured JSON");
const text = Buffer.from(captured.content, "base64").toString();
expect(text).not.toMatch(/0073|source-token|1e3/u);
expect(text).toContain('"unknown":1e-400');
expect(text).toContain('{ "token":');
expect(JSON.parse(text)).toMatchObject({
token: "[REDACTED:page:login.token]",
code: "[REDACTED:page:login.code]",
});
});
it("keeps an empty JSON-labelled response as observed empty bytes", async () => {
const captured = await captureNetworkBody(
async () => Buffer.alloc(0),
"response",
{
headers: () => ({ "content-type": "application/json" }),
headersArray: async () => [],
},
secrets,
options,
);
expect(captured).toMatchObject({ state: "captured", bytes: 0, content: "" });
});
it("checks the JSON decoding budget before invoking its decoder", async () => {
const held = resolveScenarioSecrets(scenario, { SECRET: "declared-value" });
const decode = vi.spyOn(held, "redactJsonBody");
const captured = await captureNetworkBody(
async () => Buffer.from('{"value":"untouched"}'),
"response",
{
headers: () => ({ "content-type": "application/json" }),
headersArray: async () => [],
},
held,
{ ...options, maxBodyBytes: 4096, maxBodyAllocationBytes: 1 },
);
expect(captured).toMatchObject({
state: "unavailable",
reason: "body-allocation-limit",
});
expect(decode).not.toHaveBeenCalled();
});
it.each(['{"token":"private-token-729d"}', 'A","B'])(
"withholds JSON when an explicitly declared secret spans tokens: %s",
async (secret) => {
const held = resolveScenarioSecrets(scenario, { SECRET: secret });
const body = secret.startsWith("{") ? secret : '{"x":"A","B":"C"}';
const captured = await captureNetworkBody(
async () => Buffer.from(body),
"request",
{
headers: () => ({ "content-type": "application/json" }),
headersArray: async () => [],
},
held,
{ ...options, maxBodyBytes: 4096 },
);
expect(captured).toMatchObject({
state: "unavailable",
reason: "body-read-failed",
});
expect(JSON.stringify(captured)).not.toContain(secret);
},
);
src/browser/network/PlaywrightNetworkContent.ts
+90 -16
View File
@@ -1,4 +1,9 @@
import {
isJson,
mediaTypeEssence,
} from "../../domain/analysisView/recordedMediaType.js";
import { createHash } from "node:crypto";
import { safeParseJson } from "../../domain/safeJson.js";
import {
AnalysisCancelledError,
@@ -12,7 +17,10 @@ import type {
BrowserScenarioEventAdmission,
BrowserScenarioEventBudgetReason,
} from "../../domain/browserScenarioCaptureValues.js";
import type { BrowserScenarioSecrets } from "../BrowserScenarioSecrets.js";
import type {
BrowserScenarioSecrets,
ScenarioRedactionScope,
} from "../BrowserScenarioSecrets.js";
import { withPlaywrightExecutionBoundary } from "../PlaywrightExecutionBoundary.js";
/** The producer seam for ordered, duplicate-preserving HTTP headers. */
@@ -23,6 +31,7 @@ export interface NetworkHeadersSource {
/** A bounded read cannot delay browser cleanup indefinitely. */
export interface NetworkReadOptions {
readonly redactionScope?: ScenarioRedactionScope;
readonly timeoutMs: number;
readonly maxBodyBytes: number;
readonly maxBodyAllocationBytes: number;
@@ -42,7 +51,7 @@ export type NetworkBodyPreflight = (input: {
readonly observedBytes: number;
readonly allocationBytes: number;
readonly mediaType: string | undefined;
}) => BrowserScenarioEventAdmission;
}) => BrowserScenarioEventAdmission | Promise<BrowserScenarioEventAdmission>;
const credentialHeaders = new Set([
"authorization",
@@ -55,14 +64,15 @@ const errorMessage = (
cause: unknown,
secrets: BrowserScenarioSecrets,
maxMetadataAllocationBytes: number,
scope: ScenarioRedactionScope = "current",
): string => {
const message =
cause instanceof Error && cause.message !== ""
? cause.message
: "Network read failed";
return secrets.redactionAllocationCost(message.length) <=
return secrets.redactionAllocationCost(message.length, scope) <=
maxMetadataAllocationBytes
? secrets.redact(message)
? secrets.redact(message, scope)
: "Network read failed; diagnostic exceeds the allocation budget.";
};
@@ -88,16 +98,19 @@ export const captureNetworkHeaders = async (
cost +=
secrets.redactionAllocationCost(
name.length + (credential ? 0 : value.length),
options.redactionScope,
) + 128;
if (cost > options.maxHeaderAllocationBytes)
return {
state: "unavailable",
message: `Headers exceed the ${String(options.maxHeaderAllocationBytes)}-byte allocation budget.`,
};
const safeName = secrets.redact(name);
const safeName = secrets.redact(name, options.redactionScope);
const safeValue = credential
? null
: secrets.redactBytes(Buffer.from(value)).toString("utf8");
: secrets
.redactBytes(Buffer.from(value), options.redactionScope)
.toString("utf8");
items.push({
name: safeName,
value: safeValue,
@@ -108,7 +121,12 @@ export const captureNetworkHeaders = async (
} catch (cause: unknown) {
return {
state: "unavailable",
message: errorMessage(cause, secrets, options.maxMetadataAllocationBytes),
message: errorMessage(
cause,
secrets,
options.maxMetadataAllocationBytes,
options.redactionScope,
),
};
}
};
@@ -121,6 +139,7 @@ export const captureNetworkBody = async (
secrets: BrowserScenarioSecrets,
options: NetworkReadOptions,
preflight?: NetworkBodyPreflight,
protection?: { readonly request: object; readonly retainBody: boolean },
): Promise<NetworkBodyCapture> => {
try {
const observed = await withPlaywrightExecutionBoundary(
@@ -140,8 +159,27 @@ export const captureNetworkBody = async (
limit_bytes: maxBodyBytes,
observed_bytes: observed.length,
};
const bodyAllocationCost = secrets.redactionAllocationCost(observed.length);
const base64AllocationCost = Math.ceil(observed.length / 3) * 4 * 8 + 2;
let sourceBody = observed;
if (protection !== undefined) {
sourceBody = secrets.admitResponseSecrets(
protection.request,
observed,
protection.retainBody
? {
maxBytes: options.maxBodyBytes,
maxAllocationBytes: options.maxBodyAllocationBytes,
}
: undefined,
);
if (!protection.retainBody) return { state: "not_requested" };
}
const contentType = source.headers()["content-type"];
const bodyAllocationCost = secrets.redactionAllocationCost(
Math.max(observed.length, sourceBody.length),
options.redactionScope,
);
const base64AllocationCost =
Math.ceil(Math.max(observed.length, sourceBody.length) / 3) * 4 * 8 + 2;
const maxBodyAllocationBytes = options.maxBodyAllocationBytes;
const observedBodyAllocationBytes = Math.max(
bodyAllocationCost,
@@ -156,11 +194,12 @@ export const captureNetworkBody = async (
limit_allocation_bytes: maxBodyAllocationBytes,
observed_allocation_bytes: observedBodyAllocationBytes,
};
const contentType = source.headers()["content-type"];
if (
contentType !== undefined &&
secrets.redactionAllocationCost(contentType.length) >
options.maxHeaderAllocationBytes
secrets.redactionAllocationCost(
contentType.length,
options.redactionScope,
) > options.maxHeaderAllocationBytes
)
return {
state: "unavailable",
@@ -170,16 +209,44 @@ export const captureNetworkBody = async (
limit_allocation_bytes: options.maxHeaderAllocationBytes,
observed_allocation_bytes: secrets.redactionAllocationCost(
contentType.length,
options.redactionScope,
),
};
const admission = preflight?.({
const jsonBody =
protection !== undefined ||
(contentType !== undefined && isJson(mediaTypeEssence(contentType)));
const admission = await preflight?.({
observedBytes: observed.length,
allocationBytes: observedBodyAllocationBytes,
mediaType: contentType,
});
if (admission?.status === "omitted")
return { state: "omitted", reason: admission.reason };
const retained = secrets.redactBytes(observed);
if (protection === undefined && jsonBody)
sourceBody = secrets.redactJsonBody(
observed,
{
maxBytes: options.maxBodyBytes,
maxAllocationBytes: options.maxBodyAllocationBytes,
},
options.redactionScope,
);
const retained = secrets.redactBytes(sourceBody, options.redactionScope);
if (
jsonBody &&
!retained.equals(sourceBody) &&
!safeParseJson(
new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(
retained,
),
).ok
)
return {
state: "unavailable",
reason: "body-read-failed",
message:
"Whole-body secret matching changed JSON syntax; the body was withheld.",
};
if (retained.length > maxBodyBytes)
return {
state: "unavailable",
@@ -202,7 +269,9 @@ export const captureNetworkBody = async (
media_type:
contentType === undefined
? null
: secrets.redactBytes(Buffer.from(contentType)).toString("utf8"),
: secrets
.redactBytes(Buffer.from(contentType), options.redactionScope)
.toString("utf8"),
redacted: !retained.equals(observed),
};
} catch (cause: unknown) {
@@ -214,7 +283,12 @@ export const captureNetworkBody = async (
: cause instanceof AnalysisTimeoutError
? "body-read-timeout"
: "body-read-failed",
message: errorMessage(cause, secrets, options.maxMetadataAllocationBytes),
message: errorMessage(
cause,
secrets,
options.maxMetadataAllocationBytes,
options.redactionScope,
),
};
}
};
src/browser/network/PlaywrightScenarioNetwork.test.ts
+219 -1
View File
@@ -18,6 +18,7 @@ class RequestFixture implements ScenarioNetworkRequest {
headerReads = 0;
bodyReads = 0;
predecessor: ScenarioNetworkRequest | null = null;
responseSize = 64;
constructor(readonly marker: string) {}
method() {
return "POST";
@@ -28,6 +29,9 @@ class RequestFixture implements ScenarioNetworkRequest {
resourceType() {
return "fetch";
}
async sizes() {
return { responseBodySize: this.responseSize };
}
headers(): Record<string, string> {
return {
"content-type": "text/plain",
@@ -72,7 +76,7 @@ class ResponseFixture implements ScenarioNetworkResponse {
status() {
return 200;
}
headers() {
headers(): Record<string, string> {
return {
"content-type": "application/octet-stream",
"set-cookie": "session=transport-secret",
@@ -170,6 +174,185 @@ const bodyText = (event: BrowserScenarioEvent) => {
afterEach(() => vi.useRealTimers());
it.each([true, false])(
"protects source body and headers before the binding is saved (body selected: %s)",
async (responseBody) => {
const { collector, events, secrets } = harness({
request_body: false,
response_body: responseBody,
header_values: true,
});
const request = new RequestFixture("marked");
const secret = "source-token-729d";
const response = new ResponseFixture(
request,
JSON.stringify({ token: secret, public: "kept" }),
);
vi.spyOn(response, "headers").mockReturnValue({
"content-type": "application/json",
"x-derived-token": secret,
});
vi.spyOn(response, "headersArray").mockResolvedValue([
{ name: "X-Derived-Token", value: secret },
]);
secrets.responseProtector("login", [
{ name: "token", json_pointer: "/token", secret: true },
])?.(request);
collector.request(request);
collector.response(response);
collector.finished(request);
await collector.finish();
const content = events.find(
(event) => event.kind === "network-content" && event.phase === "response",
);
if (content?.kind !== "network-content")
throw new Error("Missing response content");
expect(JSON.stringify(content.headers)).not.toContain(secret);
expect(content.headers.state).toBe("captured");
if (responseBody)
expect(JSON.parse(bodyText(content))).toEqual({
token: "[REDACTED:page:login.token]",
public: "kept",
});
else expect(content.body.state).toBe("not_requested");
expect(() => secrets.savedValue("token")).toThrow("No earlier step saved");
secrets.save("login.token", "token", secret, true);
expect(secrets.redact(secret)).toBe("[REDACTED:page:login.token]");
expect(response.bodyReads).toBe(1);
},
);
it.each([
['{"token":"\\u0073ecret-token"}', "/token"],
['{"token":"secret\\/token"}', "/token"],
['{"token":1e3}', "/token"],
['{"token":1000.0}', "/token"],
['{"items":[{"a/b":"\\u0073ecret-token"}]}', "/items/0/a~1b"],
['{"token":"shadowed","token":"\\u0073ecret-token"}', "/token"],
])(
"redacts the selected source JSON token structurally: %s",
async (body, pointer) => {
const { collector, events, secrets } = harness({
request_body: false,
response_body: true,
header_values: false,
});
const request = new RequestFixture("marked");
request.responseSize = Buffer.byteLength(body);
secrets.responseProtector("login", [
{ name: "token", json_pointer: pointer, secret: true },
])?.(request);
collector.request(request);
collector.response(new ResponseFixture(request, body));
collector.finished(request);
await collector.finish();
const content = events.find(
(event) => event.kind === "network-content" && event.phase === "response",
);
expect(content).toBeDefined();
const text = bodyText(content!);
expect(text).not.toMatch(/secret|0073|shadowed|1e3|1000/u);
expect(JSON.parse(text)).toBeDefined();
expect(text).toContain("[REDACTED:page:login.token]");
},
);
it("preserves unselected source spellings while redacting escaped aliases", async () => {
const { collector, events, secrets } = harness({
request_body: false,
response_body: true,
header_values: false,
});
const request = new RequestFixture("marked");
const body =
'{ "token":"secret-token", "mirror":"\\u0073ecret-token", "unknown":1e-400 }';
request.responseSize = Buffer.byteLength(body);
secrets.responseProtector("login", [
{ name: "token", json_pointer: "/token", secret: true },
])?.(request);
collector.request(request);
collector.response(new ResponseFixture(request, body));
collector.finished(request);
await collector.finish();
const content = events.find(
(event) => event.kind === "network-content" && event.phase === "response",
);
const text = bodyText(content!);
expect(text).toContain('"unknown":1e-400');
expect(text).toContain('{ "token":');
expect(text).not.toMatch(/secret|0073/u);
expect(JSON.parse(text).mirror).toBe("[REDACTED:page:login.token]");
});
it("associates sensitive fields by request identity while keeping same-URL unmarked values", async () => {
const { collector, events, secrets } = harness({
request_body: false,
response_body: true,
header_values: false,
});
const marked = new RequestFixture("marked");
const unmarked = new RequestFixture("unmarked");
secrets.responseProtector("login", [
{ name: "token", json_pointer: "/token", secret: true },
])?.(marked);
for (const [request, token] of [
[marked, "private-token-729d"],
[unmarked, "public-unmarked-token"],
] as const) {
collector.request(request);
collector.response(new ResponseFixture(request, JSON.stringify({ token })));
collector.finished(request);
}
await collector.finish();
const bodies = events
.filter(
(event) => event.kind === "network-content" && event.phase === "response",
)
.map((event) => JSON.parse(bodyText(event)));
expect(bodies).toEqual(
expect.arrayContaining([
{ token: "[REDACTED:page:login.token]" },
{ token: "public-unmarked-token" },
]),
);
});
it.each(["gzip", "cache"])(
"withholds marked response content when decoded bounds are unknown: %s",
async (kind) => {
const { collector, events, secrets } = harness();
const request = new RequestFixture("marked");
if (kind === "cache") request.responseSize = 0;
const secret = "private-source-token";
const response = new ResponseFixture(
request,
JSON.stringify({ token: secret }),
);
if (kind === "gzip")
vi.spyOn(response, "headers").mockReturnValue({
"content-type": "application/json",
"content-encoding": "gzip",
});
secrets.responseProtector("login", [
{ name: "token", json_pointer: "/token", secret: true },
])?.(request);
collector.request(request);
collector.response(response);
collector.finished(request);
await collector.finish();
const content = events.find(
(event) => event.kind === "network-content" && event.phase === "response",
);
if (content?.kind !== "network-content")
throw new Error("Missing response content");
expect(content.body.state).toBe("unavailable");
expect(content.headers.state).toBe("unavailable");
expect(response.bodyReads).toBe(0);
expect(response.headerReads).toBe(0);
expect(JSON.stringify(events)).not.toContain(secret);
},
);
it("keeps observed completion independent of rejected completion metadata", async () => {
const { secrets, limits } = harness();
let omit = false;
@@ -994,3 +1177,38 @@ it("stamps when each request was observed, and the browser's own start and durat
expect(failure).not.toHaveProperty("started_at");
expect(failure).not.toHaveProperty("elapsed_ms");
});
it.each([
{ name: "structural expansion", body: '{"token":1}', limit: 16 },
{
name: "shadowed container",
body: '{"token":{},"token":"source-private"}',
limit: 2048,
},
])("withholds marked content for $name", async ({ body, limit }) => {
const { collector, events, secrets } = harness(
selection,
undefined,
"declared-private",
{ max_network_body_bytes: limit },
);
const request = new RequestFixture("marked");
request.responseSize = Buffer.byteLength(body);
secrets.responseProtector("login", [
{ name: "token", json_pointer: "/token", secret: true },
])?.(request);
const response = new ResponseFixture(request, body);
collector.request(request);
collector.response(response);
collector.finished(request);
await collector.finish();
expect(events).toContainEqual(
expect.objectContaining({
kind: "network-content",
phase: "response",
body: expect.objectContaining({ state: "unavailable" }),
headers: expect.objectContaining({ state: "unavailable" }),
}),
);
expect(response.bodyReads).toBe(1);
});
src/browser/network/PlaywrightScenarioNetwork.ts
+119 -69
View File
@@ -10,6 +10,10 @@ import type {
BrowserScenarioEvent,
} from "../../domain/browserScenarioCaptureValues.js";
import type { BrowserScenarioSecrets } from "../BrowserScenarioSecrets.js";
import {
readBoundedResponseBody,
MAX_SAVED_RESPONSE_BYTES,
} from "../BrowserScenarioResponseValues.js";
import {
captureNetworkBody,
captureNetworkHeaders,
@@ -28,6 +32,7 @@ export interface ScenarioNetworkRequest extends NetworkHeadersSource {
failure(): { errorText: string } | null;
/** Browser resource timing: `startTime` in epoch milliseconds (0 until known), `responseEnd` after it (-1 when unknown). */
timing?(): { readonly startTime: number; readonly responseEnd: number };
sizes?(): Promise<{ readonly responseBodySize: number }>;
}
/** The browser's own start and duration for a finished or failed request, as far as it reported them. */
@@ -198,33 +203,12 @@ export class PlaywrightScenarioNetwork {
this.options.selection.request_body ||
this.options.selection.header_values
)
this.collect(
this.collect({
transaction,
sequence,
"request",
request,
(headers) =>
this.options.selection.request_body
? captureNetworkBody(
async () => request.postDataBuffer(),
"request",
request,
this.options.secrets,
this.readOptions,
(body) =>
this.options.admitContentBody({
transactionId: transaction.id,
sourceEventSequence: sequence,
phase: "request",
headers,
observedBytes: body.observedBytes,
allocationBytes: body.allocationBytes,
mediaType: body.mediaType,
}),
)
: Promise.resolve({ state: "not_requested" }),
this.options.selection.header_values,
);
phase: "request",
source: request,
});
}
/** Observe response metadata without starting a potentially unfinished body read. */
@@ -411,6 +395,22 @@ export class PlaywrightScenarioNetwork {
return [...this.gaps];
}
/** Source identity and readiness for the retained-event owner's final privacy pass. */
sourcePrivacy(): ReadonlyMap<string, boolean> {
return new Map(
[...this.transactions].flatMap(([request, transaction]) =>
this.options.secrets.hasResponseProtection(request)
? [
[
transaction.id,
this.options.secrets.responseSecretsReady(request),
] as const,
]
: [],
),
);
}
omittedTransactionRange(): {
readonly first_transaction_id: string;
readonly last_transaction_id: string;
@@ -607,62 +607,112 @@ export class PlaywrightScenarioNetwork {
)
return;
transaction.responseScheduled = true;
this.collect(
this.collect({
transaction,
response.sequence,
"response",
response.source,
(headers) =>
this.options.selection.response_body
? unavailable === undefined
? captureNetworkBody(
() => response.source.body(),
"response",
response.source,
this.options.secrets,
this.readOptions,
(body) =>
this.options.admitContentBody({
transactionId: transaction.id,
sourceEventSequence: response.sequence,
phase: "response",
headers,
observedBytes: body.observedBytes,
allocationBytes: body.allocationBytes,
mediaType: body.mediaType,
}),
)
: Promise.resolve(unavailable)
: Promise.resolve({ state: "not_requested" }),
this.options.selection.header_values,
);
sequence: response.sequence,
phase: "response",
source: response.source,
...(unavailable === undefined ? {} : { unavailable }),
});
}
private collect(
transaction: Transaction,
sequence: number,
phase: "request" | "response",
source: NetworkHeadersSource,
body: (headers: BrowserNetworkHeaders) => Promise<NetworkBodyCapture>,
includeHeaders: boolean,
input: {
readonly transaction: Transaction;
readonly sequence: number;
} & (
| { readonly phase: "request"; readonly source: ScenarioNetworkRequest }
| {
readonly phase: "response";
readonly source: ScenarioNetworkResponse;
readonly unavailable?: BrowserNetworkBody;
}
),
): void {
const { transaction, sequence, phase, source } = input;
const collect = async (): Promise<void> => {
const retainedHeaders = includeHeaders
? await captureNetworkHeaders(
source,
this.options.secrets,
this.readOptions,
)
: { state: "not_requested" as const };
const retainedBody = await body(retainedHeaders);
if (retainedBody.state === "omitted") {
const includeBody =
phase === "request"
? this.options.selection.request_body
: this.options.selection.response_body;
const includeHeaders = this.options.selection.header_values;
const request =
input.phase === "response" ? input.source.request() : undefined;
const protection =
request !== undefined &&
this.options.secrets.hasResponseProtection(request)
? { request, retainBody: includeBody }
: undefined;
const protectedResponse = protection !== undefined;
const privacyOnly = protectedResponse && !includeBody && includeHeaders;
const readOptions = privacyOnly
? { ...this.readOptions, maxBodyBytes: MAX_SAVED_RESPONSE_BYTES }
: this.readOptions;
let headers: Promise<BrowserNetworkHeaders> | undefined;
const getHeaders = (): Promise<BrowserNetworkHeaders> =>
(headers ??= includeHeaders
? captureNetworkHeaders(source, this.options.secrets, readOptions)
: Promise.resolve({ state: "not_requested" }));
// Existing reads keep their order. Marked responses establish secret
// spellings before either their headers or their body can be retained.
if (!protectedResponse && includeHeaders) await getHeaders();
const unavailable =
input.phase === "response" ? input.unavailable : undefined;
const observedBody: NetworkBodyCapture =
unavailable ??
(includeBody || privacyOnly
? await captureNetworkBody(
async () => {
if (input.phase === "request")
return input.source.postDataBuffer();
if (!protectedResponse) return input.source.body();
const bounded = await readBoundedResponseBody(
input.source,
Math.min(readOptions.maxBodyBytes, MAX_SAVED_RESPONSE_BYTES),
);
if (!bounded.ok) throw new Error(bounded.error.message);
return bounded.value;
},
phase,
source,
this.options.secrets,
readOptions,
async (body) =>
this.options.admitContentBody({
transactionId: transaction.id,
sourceEventSequence: sequence,
phase,
headers: await getHeaders(),
observedBytes: body.observedBytes,
allocationBytes: body.allocationBytes,
mediaType: body.mediaType,
}),
protection,
)
: { state: "not_requested" });
if (observedBody.state === "omitted") {
this.options.omitNetwork(
transaction.id,
"network-content",
retainedBody.reason,
observedBody.reason,
);
return;
}
const protectedReadFailed =
protectedResponse &&
(observedBody.state === "unavailable" ||
observedBody.state === "not_exposed");
const retainedHeaders: BrowserNetworkHeaders =
protectedReadFailed && includeHeaders
? {
state: "unavailable",
message:
"Response header values were withheld because marked source values could not be read safely.",
}
: await getHeaders();
const retainedBody = includeBody
? observedBody
: { state: "not_requested" as const };
if (
retainedHeaders.state === "unavailable" ||
retainedBody.state === "unavailable" ||
src/domain/analysisView/EnvironmentStateWriteSimulation.test.ts
+56
View File
@@ -37,6 +37,7 @@ import {
import { updatedFields } from "./environmentStateWriteApply.js";
import {
retainFixed,
retainCreateExemplar,
type EffectContext,
} from "./environmentStateWriteEffects.js";
@@ -3113,6 +3114,61 @@ it("learns omitted fields using the raw ID resolved by the item owner", () => {
});
});
describe("create exemplar retention", () => {
const facts = () => {
const snapshot = { id: "held", payload: "p".repeat(1_000_000) };
const held = Array.from({ length: 1000 }, () => snapshot);
return {
entity: { id: "new", url: "/new" },
call: {
path: "/items",
pathValues: new Map<string, string>(),
body: { payload: "x".repeat(10_000) },
},
idField: "id",
startMs: undefined,
collection: held,
type: held,
};
};
it("charges temporary request facts and shared snapshot references without rebilling owned entity payloads", () => {
let retained = 0;
const context = {
io: {
retain: (bytes: number) => {
retained += bytes;
},
},
} as unknown as EffectContext;
const input = facts();
expect(retainCreateExemplar(context, input, [])).toBe(input);
expect(retained).toBeGreaterThan(28_000);
expect(retained).toBeLessThan(40_000);
});
it("retains no exemplar when every string field is already explained", () => {
const retain = vi.fn();
const context = { io: { retain } } as unknown as EffectContext;
expect(
retainCreateExemplar(context, facts(), [
{ field: "id", rule: "echo" },
{ field: "url", rule: "echo" },
]),
).toBeUndefined();
expect(retain).not.toHaveBeenCalled();
});
it("does not adopt request facts when the retained budget refuses them", () => {
const context = {
io: {
retain: () => {
throw new Error("budget exhausted");
},
},
} as unknown as EffectContext;
expect(() => retainCreateExemplar(context, facts(), [])).toThrow(
"budget exhausted",
);
});
});
it.each([
{
label: "point read with simultaneous unknown action",
src/domain/analysisView/EnvironmentStateWriteSimulation.ts
+1
View File
@@ -224,6 +224,7 @@ export class EnvironmentStateWriteSimulation {
unstamped: new Set(),
kept: new Set(),
fixedWidth: new Map(),
ampersands: new Map(),
linked: link.collection,
});
}
src/domain/analysisView/environmentStateDerivedWrites.test.ts
+589
View File
@@ -20,6 +20,7 @@ import {
UNKNOWN_TAG_UPDATE_ROWS,
} from "../../../tests/fixtures/environmentStateDerivedValues.js";
import {
bookClock,
environmentStateViewOf,
numberedUuid,
} from "../../../tests/fixtures/environmentStateWrites.js";
@@ -42,6 +43,18 @@ const TRIMMED = {
trimmed: true,
} as const;
/** A capture row whose response leaves out one field. */
const omitting = (row: TimedCaptureRow, field: string): TimedCaptureRow => {
const [method, path, body, ...rest] = row;
if (!isObject(body)) throw new Error("Expected an object response");
return [
method,
path,
Object.fromEntries(Object.entries(body).filter(([key]) => key !== field)),
...rest,
];
};
describe("later creates must retain the learned value rules", () => {
it("withholds a later create whose captured composed address is null", () => {
const snapshot = null;
@@ -64,6 +77,33 @@ describe("later creates must retain the learned value rules", () => {
});
});
it.each([
{
name: "composed snapshot address",
tool: "post_bookmarks",
rows: (): TimedCaptureRow[] =>
BOOKMARK_ROWS.slice(0, 6).map((row, index) =>
index === 5 ? omitting(row, "web_archive_snapshot_url") : row,
),
},
{
name: "templated id text",
tool: "post_folders",
rows: (): TimedCaptureRow[] =>
mirrored((id) =>
id === 1656166745726976 ? {} : { idText: String(id) },
),
},
])(
"withholds a later create whose response omits the $name the plan rebuilds",
({ tool, rows }) => {
expect(planOf(rows(), tool)).toMatchObject({
served: false,
reason: "unexplained-change",
});
},
);
it.each([
{
name: "a string integer becomes a UUID",
@@ -346,6 +386,32 @@ describe("values an application composes or generates", () => {
});
});
it("serves folder creates whose address follows a generated uid and the title's slug", () => {
const generated = FOLDER_ROWS.map((row): TimedCaptureRow => {
const [method, path, body, status, sent] = row;
if (method !== "POST" || !isObject(sent) || body === undefined)
return row;
const { uid: _uid, ...title } = sent;
return [method, path, body, status!, title];
});
expect(planOf(generated, "post_folders")).toMatchObject({
served: true,
body: { accepted: ["title"] },
rules: expect.arrayContaining([
{
field: "url",
rule: "composed",
parts: [
{ text: "/dashboards/f/" },
{ from: "uid", as: "value" },
{ text: "/" },
{ from: "title", as: "slug" },
],
},
]),
});
});
it("withholds renames whose address a later call composes when an earlier call left it otherwise", () => {
expect(
planOf(restoredAddressRows(true), "put_folders_by_uuid"),
@@ -441,6 +507,495 @@ const mirrored = (
});
};
/** A folder's slug with `&` spelled as Grafana's Go slugs spell it. */
const wordSlug = (title: string): string =>
title
.replaceAll("&", " and ")
.toLowerCase()
.replace(/[^a-z0-9]+/gu, "-")
.replace(/^-|-$/gu, "");
/** A folder's slug with `&` dropped, as Django's slugify drops it. */
const droppedSlug = (title: string): string =>
wordSlug(title.replaceAll("&", " "));
/** When the seeded folder was made, well before the capture. */
const SEEDED = "2026-10-01T09:00:00Z";
/** Grafana's time-based folder ids, as the recorded folders hold them. */
const FOLDER_IDS = [1655476044521472, 1656166527623168, 1656166745726976];
interface CreatedFolder {
readonly uid: string;
readonly title: string;
/** Whether the call sent the uid, or the application generated it. */
readonly sent: boolean;
/** The address's last segment, when the application spells it otherwise than the title's slug. */
readonly tail?: string;
}
/** A seeded folder and folder creates, each read back and listed, whose addresses join the uid and `spell` of the title. */
const folderCreates = (
created: readonly CreatedFolder[],
spell: (title: string) => string,
seed: { readonly uid: string; readonly title: string } = {
uid: numberedUuid(101),
title: "Lighthouse Rota",
},
): TimedCaptureRow[] => {
const entity = (
folder: {
readonly uid: string;
readonly title: string;
readonly tail?: string;
},
index: number,
at: number,
): JsonObject => ({
id: FOLDER_IDS[index]!,
uid: folder.uid,
title: folder.title,
url: `/dashboards/f/${folder.uid}/${folder.tail ?? spell(folder.title)}`,
created: bookClock(at),
updated: bookClock(at),
});
const held: JsonObject[] = [
{ ...entity(seed, 0, 0), created: SEEDED, updated: SEEDED },
];
const rows: TimedCaptureRow[] = [
["GET", "/api/folders", [...held]],
["GET", `/api/folders/${seed.uid}`, held[0]!],
];
created.forEach((folder, index) => {
// The application stamps a create with the time of its call, the row's position.
const answered = entity(folder, index + 1, rows.length);
held.push(answered);
rows.push(
[
"POST",
"/api/folders",
answered,
200,
folder.sent
? { uid: folder.uid, title: folder.title }
: { title: folder.title },
],
["GET", `/api/folders/${folder.uid}`, answered],
["GET", "/api/folders", [...held]],
);
});
return rows;
};
const addressParts = (ampersand?: "and") => [
{ text: "/dashboards/f/" },
{ from: "uid", as: "value" },
{ text: "/" },
{
from: "title",
as: "slug",
...(ampersand === undefined ? {} : { ampersand }),
},
];
describe("slug spellings across creates", () => {
it("learns that a slug spells & as the word from the first create that sends one", () => {
const rows = folderCreates(
[
{ uid: numberedUuid(102), title: "Alpha Harbour", sent: false },
{ uid: numberedUuid(103), title: "Bravo Tide & Wind", sent: false },
],
wordSlug,
);
expect(planOf(rows, "post_folders")).toMatchObject({
served: true,
rules: expect.arrayContaining([
{ field: "url", rule: "composed", parts: addressParts("and") },
]),
});
});
it("keeps a slug that drops &, and refuses a later create that spells it as the word", () => {
const dropped = folderCreates(
[
{ uid: numberedUuid(102), title: "Alpha & Harbour", sent: false },
{ uid: numberedUuid(103), title: "Bravo Tide", sent: false },
],
droppedSlug,
);
expect(planOf(dropped, "post_folders")).toMatchObject({
served: true,
rules: expect.arrayContaining([
{ field: "url", rule: "composed", parts: addressParts() },
]),
});
const mixed = folderCreates(
[
{ uid: numberedUuid(102), title: "Alpha & Harbour", sent: false },
{
uid: numberedUuid(103),
title: "Bravo Tide & Wind",
sent: false,
tail: "bravo-tide-and-wind",
},
],
droppedSlug,
);
expect(planOf(mixed, "post_folders")).toMatchObject({ served: false });
});
it("keeps a dropped & that an accepted create showed when a later one spells it as the word", () => {
const tag = (id: number, name: string, slug: string) => ({
id,
name,
slug,
});
const seed = tag(1, "Lamp Oil", "lamp-oil");
const created = [
tag(2, "Alpha Notes", "alpha-notes"),
tag(3, "Bravo & Tide", "bravo-tide"),
tag(4, "Charlie & Wind", "charlie-and-wind"),
];
const rows: TimedCaptureRow[] = [
["GET", "/api/tags", [seed]],
["GET", "/api/tags/1", seed],
];
created.forEach((entity, index) =>
rows.push(
["POST", "/api/tags", entity, 201, { name: entity.name }],
["GET", `/api/tags/${String(entity.id)}`, entity],
["GET", "/api/tags", [seed, ...created.slice(0, index + 1)]],
),
);
const plan = planOf(rows, "post_tags");
expect(plan.rules).toContainEqual({
field: "slug",
rule: "slug",
from: "name",
});
expect(plan.served).toBe(false);
});
});
/** A category whose slug the application derives from its name. */
const category = (id: number, name: string, slug: string) => ({
id,
name,
slug,
});
/** Two category renames, each read back, after reading the `held` categories; a PUT sends the whole object back. */
const renameRows = (
method: "PATCH" | "PUT",
held: readonly ReturnType<typeof category>[],
renames: readonly ReturnType<typeof category>[],
): TimedCaptureRow[] => {
const current = new Map(held.map((entity) => [entity.id, entity]));
const rows: TimedCaptureRow[] = [
["GET", "/api/categories", [...held]],
...held.map((entity): TimedCaptureRow => [
"GET",
`/api/categories/${entity.id}`,
entity,
]),
];
for (const renamed of renames) {
const before = current.get(renamed.id)!;
rows.push(
[
method,
`/api/categories/${renamed.id}`,
renamed,
200,
method === "PATCH"
? { name: renamed.name }
: { ...before, name: renamed.name },
],
["GET", `/api/categories/${renamed.id}`, renamed],
);
current.set(renamed.id, renamed);
}
rows.push(["GET", "/api/categories", [...current.values()]]);
return rows;
};
describe("slug spellings across updates", () => {
it.each(["PATCH", "PUT"] as const)(
"applies held spelling to the initial %s rule without needing a later update",
(method) => {
for (const held of [
[category(1, "Salt & Pepper", "salt-and-pepper")],
[
category(1, "Lamp Oil", "lamp-oil"),
category(2, "Salt & Pepper", "salt-and-pepper"),
],
]) {
const plan = planOf(
renameRows(method, held, [
category(1, "Harbour Notes", "harbour-notes"),
]),
`${method.toLowerCase()}_categories_by_id`,
);
expect(plan).toMatchObject({
served: true,
rules: expect.arrayContaining([
{ field: "slug", rule: "slug", from: "name", ampersand: "and" },
]),
});
}
},
);
it.each(["PATCH", "PUT"] as const)(
"withholds an initial %s that contradicts held slug spelling",
(method) => {
const plan = planOf(
renameRows(
method,
[category(1, "Salt & Pepper", "salt-and-pepper")],
[category(1, "Tide & Wind", "tide-wind")],
),
`${method.toLowerCase()}_categories_by_id`,
);
expect(plan.served).toBe(false);
},
);
it.each(["PATCH", "PUT"] as const)(
"withholds an initial %s with contradictory held slug conventions",
(method) => {
const plan = planOf(
renameRows(
method,
[
category(1, "Lamp Oil", "lamp-oil"),
category(2, "Salt & Pepper", "salt-and-pepper"),
category(3, "Tide & Wind", "tide-wind"),
],
[category(1, "Harbour Notes", "harbour-notes")],
),
`${method.toLowerCase()}_categories_by_id`,
);
expect(plan.served).toBe(false);
},
);
it.each(["PATCH", "PUT"] as const)(
"learns that an update's slug spells & as the word from a later %s that renames to one",
(method) => {
const rows = renameRows(
method,
[
category(1, "Harbour Notes", "harbour-notes"),
category(2, "Lamp Oil", "lamp-oil"),
],
[
category(1, "Quay Notes", "quay-notes"),
category(2, "Tide & Wind", "tide-and-wind"),
],
);
expect(
planOf(rows, `${method.toLowerCase()}_categories_by_id`),
).toMatchObject({
served: true,
rules: expect.arrayContaining([
{ field: "slug", rule: "slug", from: "name", ampersand: "and" },
]),
});
},
);
it.each(["PATCH", "PUT"] as const)(
"keeps a slug that spells & as the word when a later %s renames to a name without one",
(method) => {
const rows = renameRows(
method,
[
category(1, "Harbour Notes", "harbour-notes"),
category(2, "Lamp Oil", "lamp-oil"),
],
[
category(1, "Tide & Wind", "tide-and-wind"),
category(2, "Quay Notes", "quay-notes"),
],
);
expect(
planOf(rows, `${method.toLowerCase()}_categories_by_id`),
).toMatchObject({
served: true,
rules: expect.arrayContaining([
{ field: "slug", rule: "slug", from: "name", ampersand: "and" },
]),
});
},
);
it.each([
[
"a held category",
[
category(1, "Harbour Notes", "harbour-notes"),
category(3, "Salt & Pepper", "salt-pepper"),
],
[category(1, "Quay Notes", "quay-notes")],
],
[
"an earlier rename",
[
category(1, "Harbour Notes", "harbour-notes"),
category(3, "Lamp Oil", "lamp-oil"),
],
[category(1, "Salt & Pepper", "salt-pepper")],
],
])(
"refuses a later rename that spells & as the word after %s dropped it",
(_, held, earlier) => {
const rows = renameRows("PATCH", held, [
...earlier,
category(held[1]!.id, "Tide & Wind", "tide-and-wind"),
]);
expect(planOf(rows, "patch_categories_by_id")).toMatchObject({
served: false,
});
},
);
});
describe("slug spellings the held entities show", () => {
it("spells & as the held entities do when the creates slug none", () => {
const rows = folderCreates(
[
{ uid: numberedUuid(102), title: "Alpha Harbour", sent: false },
{ uid: numberedUuid(103), title: "Bravo Tide", sent: false },
],
wordSlug,
{ uid: numberedUuid(101), title: "Lamp & Oil" },
);
expect(planOf(rows, "post_folders")).toMatchObject({
served: true,
rules: expect.arrayContaining([
{ field: "url", rule: "composed", parts: addressParts("and") },
]),
});
});
it("learns a slug field's & spelling from held tags, and no slug rule they contradict", () => {
const tagRows = (held: string, created: string, slug: string) => {
const seed = { id: 1, name: held, slug: wordSlug(held) };
const tag = { id: 2, name: created, slug };
return [
["GET", "/api/tags", [seed]],
["GET", "/api/tags/1", seed],
["POST", "/api/tags", tag, 201, { name: created }],
["GET", "/api/tags/2", tag],
["GET", "/api/tags", [seed, tag]],
] as TimedCaptureRow[];
};
expect(
planOf(
tagRows("Salt & Pepper", "Harbour Notes", "harbour-notes"),
"post_tags",
).rules,
).toContainEqual({
field: "slug",
rule: "slug",
from: "name",
ampersand: "and",
});
expect(
planOf(tagRows("Salt & Pepper", "Tide & Wind", "tide-wind"), "post_tags")
.rules,
).not.toContainEqual(
expect.objectContaining({ field: "slug", rule: "slug" }),
);
});
it("learns no slug rule from held tags that spell & both ways", () => {
const held = [
{ id: 1, name: "Salt & Pepper", slug: "salt-and-pepper" },
{ id: 3, name: "Fish & Chips", slug: "fish-chips" },
];
const tag = { id: 2, name: "Harbour Notes", slug: "harbour-notes" };
const rows: TimedCaptureRow[] = [
["GET", "/api/tags", held],
["GET", "/api/tags/1", held[0]!],
["GET", "/api/tags/3", held[1]!],
["POST", "/api/tags", tag, 201, { name: tag.name }],
["GET", "/api/tags/2", tag],
["GET", "/api/tags", [...held, tag]],
];
expect(planOf(rows, "post_tags").rules).not.toContainEqual(
expect.objectContaining({ field: "slug", rule: "slug" }),
);
});
it("learns no & spelling from a held slug neither spelling gives", () => {
const rows = (created: string, slug: string): TimedCaptureRow[] => {
const seed = { id: 1, name: "Salt & Pepper", slug: "condiments" };
const tag = { id: 2, name: created, slug };
return [
["GET", "/api/tags", [seed]],
["GET", "/api/tags/1", seed],
["POST", "/api/tags", tag, 201, { name: created }],
["GET", "/api/tags/2", tag],
["GET", "/api/tags", [seed, tag]],
];
};
expect(
planOf(rows("Tide & Wind", "tide-and-wind"), "post_tags").rules,
).not.toContainEqual(
expect.objectContaining({ field: "slug", rule: "slug" }),
);
expect(
planOf(rows("Harbour Notes", "harbour-notes"), "post_tags").rules,
).toContainEqual({ field: "slug", rule: "slug", from: "name" });
});
});
describe("ambiguous addresses and templates across creates", () => {
it("learns an address the first create cannot tell apart, its uid being its title's slug, from the second", () => {
const rows = folderCreates(
[
{ uid: "alpha-harbour", title: "Alpha Harbour", sent: true },
{ uid: "bravo-store", title: "Bravo Tide Notes", sent: true },
],
wordSlug,
{ uid: "lighthouse-rota", title: "Lighthouse Rota" },
);
expect(planOf(rows, "post_folders")).toMatchObject({
served: true,
body: { accepted: ["uid", "title"], fixed: [] },
rules: expect.arrayContaining([
{ field: "url", rule: "composed", parts: addressParts() },
]),
});
});
it("withholds creates whose template a later create answers otherwise", () => {
const rows = folderCreates(
[
{
uid: numberedUuid(102),
title: "Alpha Harbour",
sent: false,
tail: "inbox",
},
{
uid: numberedUuid(103),
title: "Bravo Tide",
sent: false,
tail: "outbox",
},
],
wordSlug,
);
expect(planOf(rows, "post_folders")).toMatchObject({
served: false,
reason: "unexplained-change",
});
});
});
describe("generated integers another field repeats", () => {
it("withholds folder creates a later call answers with a UUID where the exemplar generated an integer", () => {
const later = 1656166745726976;
@@ -515,3 +1070,37 @@ describe("generated integers another field repeats", () => {
});
});
});
describe("slugs the held entities' spellings refuse", () => {
const SALT = { id: 1, name: "Salt & Pepper", slug: "salt-and-pepper" };
const FISH = { id: 3, name: "Fish & Chips", slug: "fish-chips" };
it.each([
["spell & both ways", [SALT, FISH], "Harbour Notes", "harbour-notes"],
["spell & otherwise than the create", [SALT], "Tide & Wind", "tide-wind"],
])(
"accepts a created tag's name only as recorded when held tags %s",
(_case, held, name, slug) => {
const tag = { id: 2, name, slug };
const rows: TimedCaptureRow[] = [
["GET", "/api/tags", held],
...held.map((entity): TimedCaptureRow => [
"GET",
`/api/tags/${String(entity.id)}`,
entity,
]),
["POST", "/api/tags", tag, 201, { name }],
["GET", "/api/tags/2", tag],
["GET", "/api/tags", [...held, tag]],
];
const plan = planOf(rows, "post_tags");
expect(plan.rules).not.toContainEqual(
expect.objectContaining({ field: "slug", rule: "slug" }),
);
// The slug kept as recorded fits only the recorded name.
expect(plan.body).not.toBeNull();
expect(plan.body?.accepted).not.toContain("name");
expect(plan.body?.fixed).toContainEqual({ member: "name", sent: name });
},
);
});
src/domain/analysisView/environmentStateView.ts
+9 -2
View File
@@ -262,7 +262,12 @@ const fieldRuleSchema = z.discriminatedUnion("rule", [
}),
z.strictObject({ ...ruled, rule: z.literal("copy"), from: z.string() }),
z.strictObject({ ...ruled, rule: z.literal("path"), from: z.string() }),
z.strictObject({ ...ruled, rule: z.literal("slug"), from: z.string() }),
z.strictObject({
...ruled,
rule: z.literal("slug"),
from: z.string(),
ampersand: z.literal("and").exactOptional(),
}),
z.strictObject({ ...ruled, rule: z.literal("template"), from: z.string() }),
z.strictObject({
...ruled,
@@ -271,9 +276,11 @@ const fieldRuleSchema = z.discriminatedUnion("rule", [
.array(
z.union([
z.strictObject({ text: z.string().min(1) }),
z.strictObject({ from: z.string(), as: z.literal("value") }),
z.strictObject({
from: z.string(),
as: z.enum(["value", "slug"]),
as: z.literal("slug"),
ampersand: z.literal("and").exactOptional(),
}),
z.strictObject({
from: z.string(),
src/domain/analysisView/environmentStateWriteApply.ts
+1 -1
View File
@@ -282,7 +282,7 @@ const derive = (
if (rule.rule === "slug") {
const from = ownMember(sent, rule.from);
if (typeof from === "string")
setOwnMember(written, rule.field, slugOf(from));
setOwnMember(written, rule.field, slugOf(from, rule.ampersand));
} else if (rule.rule === "template") {
const value = templated(observed, written, rule);
if (value !== undefined) setOwnMember(written, rule.field, value);
src/domain/analysisView/environmentStateWriteCalls.ts
+36 -5
View File
@@ -4,6 +4,10 @@ import type { JsonValue } from "../jsonValue.js";
import { ownMember, same, type JsonObject } from "./environmentStateRules.js";
import { storedKinds } from "./environmentStateWriteApply.js";
import { composedValue } from "./environmentStateWriteComposed.js";
import {
ampersandSpelled,
type AmpersandSpelling,
} from "./environmentStateWriteCreateCalls.js";
import type { explainUpdate } from "./environmentStateWriteExplain.js";
import {
alikeRules,
@@ -104,14 +108,29 @@ export interface UpdateCall {
readonly kept: readonly string[];
}
/** Ambiguous saves may echo an already-derived slug or hold a clock in its own second. Keep the demonstrated derivation; call evidence still has to reproduce it. */
/** How a later update's entity and the plan's calls and held entities showed `&` in the slugs it learns. */
export interface UpdateSpelling {
readonly entity: JsonObject;
readonly ampersands: ReadonlyMap<string, AmpersandSpelling>;
}
/** Ambiguous saves may echo an already-derived slug, hold a clock in its own second, or slug no `&`. Keep the demonstrated derivation; call evidence still has to reproduce it. */
export const reconciledUpdateRule = (
held: FieldRule,
later: FieldRule,
kept: ReadonlySet<string>,
spelling?: UpdateSpelling,
): FieldRule | undefined => {
const alike = alikeRules(held, later, kept);
if (alike !== undefined) return alike;
if (spelling !== undefined) {
const spelled = ampersandSpelled(
held,
{ rule: later, entity: spelling.entity },
spelling.ampersands.get(held.field),
);
if (spelled !== undefined) return spelled;
}
for (const [derived, ambiguous] of [
[held, later],
[later, held],
@@ -142,10 +161,18 @@ export const contradictsEarlierCalls = (
): boolean => {
const ruled = new Map(draft.plan.rules.map((rule) => [rule.field, rule]));
return rules.some((rule) => {
const held = ruled.get(rule.field);
// A slug that echoes, or that spells `&` as the word where the plan's drops it, must reproduce every earlier call; one that slugged no `&` keeps the plan's spelling.
if (
!(unsentChange(rule) || regenerates(rule) || rule.rule === "clock") ||
(ruled.has(rule.field) &&
!(rule.rule === "slug" && ruled.get(rule.field)?.rule === "echo"))
(held !== undefined &&
!(
rule.rule === "slug" &&
(held.rule === "echo" ||
(held.rule === "slug" &&
held.ampersand === undefined &&
rule.ampersand !== undefined))
))
)
return false;
// Compute a structured rule's digest once, rather than once per earlier call.
@@ -181,7 +208,10 @@ export const contradictsEarlierCalls = (
return (
rule.rule === "slug" &&
(typeof from === "string"
? !sameObserved(observation, observedValue(slugOf(from)))
? !sameObserved(
observation,
observedValue(slugOf(from, rule.ampersand)),
)
: !kept.includes(rule.field))
);
});
@@ -220,6 +250,7 @@ export const conflictsWithPlan = (
plan: Pick<StateWritePlan, "body" | "rules">,
explained: ReturnType<typeof explainUpdate>,
kept: ReadonlySet<string>,
spelling?: UpdateSpelling,
): boolean => {
const body = plan.body!;
const fixed = new Map(body.fixed.map((entry) => [entry.member, entry.sent]));
@@ -239,7 +270,7 @@ export const conflictsWithPlan = (
const held = rules.get(rule.field);
return (
held !== undefined &&
reconciledUpdateRule(held, rule, kept) === undefined
reconciledUpdateRule(held, rule, kept, spelling) === undefined
);
})
);
src/domain/analysisView/environmentStateWriteChanges.ts
+33 -12
View File
@@ -10,8 +10,9 @@ import type { FieldRule, WriteCall } from "./environmentStateWriteRules.js";
import {
integerOf,
readClock,
slugOf,
slugSpellings,
sortedStrings,
type Ampersand,
} from "./environmentStateWriteValues.js";
import { freshUuid, renewedRule } from "./environmentStateWriteGenerated.js";
import { composedChange } from "./environmentStateWriteComposed.js";
@@ -24,6 +25,20 @@ import {
stampRule,
} from "./environmentStateWriteStamps.js";
/** The member a slug spells, and how it spells `&`. */
export interface SlugSource {
readonly from: string;
readonly ampersand?: Ampersand;
}
/** The slug rule a field follows from its source. */
export const slugRule = (field: string, source: SlugSource): FieldRule => ({
field,
rule: "slug",
from: source.from,
...(source.ampersand === undefined ? {} : { ampersand: source.ampersand }),
});
/** What explaining an update reads: the entity before and after, the call and its method. */
export interface UpdateEvidence {
readonly before: JsonObject;
@@ -51,7 +66,7 @@ export interface UpdateEvidence {
export const changeRule = (
evidence: UpdateEvidence,
field: string,
slugs: ReadonlyMap<string, string>,
slugs: ReadonlyMap<string, SlugSource>,
): FieldRule | undefined => {
const before = ownMember(evidence.before, field);
const after = ownMember(evidence.after, field)!;
@@ -59,8 +74,8 @@ export const changeRule = (
return { field, rule: "clock", format: readClock(after)!.format };
if (freshUuid(evidence, field)) return { field, rule: "unique" };
// A renamed slug is spelled like a token, so the slug comes first.
const member = typeof after === "string" ? slugs.get(after) : undefined;
if (member !== undefined) return { field, rule: "slug", from: member };
const source = typeof after === "string" ? slugs.get(after) : undefined;
if (source !== undefined) return slugRule(field, source);
if (freshToken(evidence, field)) return { field, rule: "unique" };
const nested = nestedClockRule(field, { before, after }, evidence);
if (nested !== undefined) return nested;
@@ -145,16 +160,22 @@ export const sameSecondClocks = (evidence: UpdateEvidence): string[] =>
const sentSlugSource = (
evidence: UpdateEvidence,
member: string,
): string | undefined => {
): SlugSource | undefined => {
const stored = ownMember(evidence.after, member);
if (typeof stored !== "string") return undefined;
return Object.entries(evidence.call.body).find(
([other, value]) =>
for (const [other, value] of Object.entries(evidence.call.body))
if (
other !== member &&
typeof value === "string" &&
same(ownMember(evidence.after, other), value) &&
slugOf(value) === stored,
)?.[0];
same(ownMember(evidence.after, other), value)
)
for (const { slug, ampersand } of slugSpellings(value))
if (slug === stored)
return {
from: other,
...(ampersand === undefined ? {} : { ampersand }),
};
return undefined;
};
/** Whether a call sent back the clock the entity held, as a whole-object write does, and the application stamped it anew near the call. */
@@ -192,8 +213,8 @@ export const sentRule = (
rule: "clock",
format: readClock(ownMember(evidence.after, member))!.format,
};
const from = sentSlugSource(evidence, member);
if (from !== undefined) return { field: member, rule: "slug", from };
const source = sentSlugSource(evidence, member);
if (source !== undefined) return slugRule(member, source);
return freshUuid(evidence, member)
? { field: member, rule: "unique" }
: {
src/domain/analysisView/environmentStateWriteComposed.test.ts
+74
View File
@@ -300,3 +300,77 @@ it(
expect(reads).toBe(n + 2);
},
);
describe("slug spellings", () => {
it("spells & in a slug part as the word only when the part says so", () => {
const parts = (ampersand?: "and") => [
{ text: "/dashboards/f/" },
{ from: "uid", as: "value" as const },
{ text: "/" },
{
from: "title",
as: "slug" as const,
...(ampersand === undefined ? {} : { ampersand }),
},
];
const folder = { uid: "u1", title: "Tide & Wind" };
expect(composedValue(parts("and"), folder)).toBe(
"/dashboards/f/u1/tide-and-wind",
);
expect(composedValue(parts(), folder)).toBe("/dashboards/f/u1/tide-wind");
});
it("reads an address whose uid is its title's slug as held entities spell it", () => {
const held = [
{
uid: "a1b2c3",
title: "Lighthouse Rota",
url: "/dashboards/f/a1b2c3/lighthouse-rota",
},
];
const created = {
uid: "scratch-pier",
title: "Scratch Pier",
url: "/dashboards/f/scratch-pier/scratch-pier",
};
const rules: FieldRule[] = [
{ field: "uid", rule: "echo" },
{ field: "title", rule: "echo" },
];
expect(createdComposer(created, rules, held)("url")).toEqual({
field: "url",
rule: "composed",
parts: [
{ text: "/dashboards/f/" },
{ from: "uid", as: "value" },
{ text: "/" },
{ from: "title", as: "slug" },
],
});
expect(
createdComposer(created, rules, [
{
...held[0]!,
uid: "lighthouse-rota",
url: "/dashboards/f/lighthouse-rota/lighthouse-rota",
},
])("url"),
).toBeUndefined();
});
it("reads no address that more sources spell than it can tell apart", () => {
const fields = Object.fromEntries(
Array.from({ length: 40 }, (_, index) => [
`alias${String(index)}`,
"harbour-notes",
]),
);
const rules: FieldRule[] = Object.keys(fields).map((field) => ({
field,
rule: "echo",
}));
const created = { ...fields, url: "/notes/harbour-notes/harbour-notes" };
const held = [{ ...fields, url: "/notes/harbour-notes/harbour-notes" }];
expect(createdComposer(created, rules, held)("url")).toBeUndefined();
});
});
src/domain/analysisView/environmentStateWriteComposed.ts
+82 -33
View File
@@ -13,6 +13,7 @@ import {
joinsBefore,
readClock,
slugOf,
slugSpellings,
zoneMinutesForFormat,
} from "./environmentStateWriteValues.js";
import {
@@ -47,7 +48,7 @@ export const composedPartText = (
const value = ownMember(fields, part.from);
if (part.as === "clock") return clockDigits(value, part.pattern);
if (part.as === "value") return scalarText(value);
return typeof value === "string" ? slugOf(value) : undefined;
return typeof value === "string" ? slugOf(value, part.ampersand) : undefined;
};
/** A composed string spelled from written fields, or undefined when a part has nothing to spell. */
@@ -74,11 +75,17 @@ const candidatesOf = (
const text = scalarText(value);
if (text !== undefined && text.length >= shortest)
candidates.push({ token: text, part: { from: field, as: "value" } });
if (typeof value === "string") {
const slug = slugOf(value);
if (slug !== value && slug.length >= COPY_MIN_LENGTH)
candidates.push({ token: slug, part: { from: field, as: "slug" } });
}
if (typeof value === "string")
for (const { slug, ampersand } of slugSpellings(value))
if (slug !== value && slug.length >= COPY_MIN_LENGTH)
candidates.push({
token: slug,
part: {
from: field,
as: "slug",
...(ampersand === undefined ? {} : { ampersand }),
},
});
if (readClock(value) !== undefined)
for (const pattern of CLOCK_PATTERNS) {
const digits = clockDigits(value, pattern);
@@ -123,36 +130,65 @@ const matchesAt = (
return [];
};
/** Parts with each slug of a source holding no `&` spelling it as the word, or undefined when no such slug part exists. */
const ampersandWord = (
parts: readonly ComposedPart[],
entity: JsonObject,
): ComposedPart[] | undefined => {
let changed = false;
const spelled = parts.map((part): ComposedPart => {
if (!("as" in part) || part.as !== "slug" || part.ampersand !== undefined)
return part;
const source = ownMember(entity, part.from);
if (typeof source === "string" && source.includes("&")) return part;
changed = true;
return { ...part, ampersand: "and" };
});
return changed ? spelled : undefined;
};
/** More ways to read one string than this leave it unexplained. */
const MAX_SEGMENTATIONS = 8;
/**
* Literal text around the longest delimited candidate at each position, left to right; undefined when none shows, or
* when one text comes from two sources serving could tell apart. `alike` names parts serving fills with one value.
* Literal text around the longest delimited candidate at each position, left to right: one reading per choice where
* sources serving could tell apart spell the same text, which other entities then tell apart. Empty when no candidate
* shows, or the choices exceed `MAX_SEGMENTATIONS`. `alike` names parts serving fills with one value.
*/
const segmented = (
const segmentations = (
text: string,
index: CandidateIndex,
alike: (left: SourcePart, right: SourcePart) => boolean,
): ComposedPart[] | undefined => {
const parts: ComposedPart[] = [];
): ComposedPart[][] => {
let readings: ComposedPart[][] = [[]];
let literal = "";
let shown = false;
for (let at = 0; at < text.length;) {
const matches = matchesAt(text, at, index);
const match = matches[0];
if (match === undefined) {
if (matches.length === 0) {
literal += text[at];
at += 1;
continue;
}
if (matches.some((other) => !alike(match.part, other.part)))
return undefined;
if (literal !== "") parts.push({ text: literal });
// Stop at the cap before comparing or allocating more, however many sources spell the text.
const choices: SourcePart[] = [];
for (const { part } of matches)
if (!choices.some((choice) => alike(choice, part))) {
choices.push(part);
if (readings.length * choices.length > MAX_SEGMENTATIONS) return [];
}
const lead: ComposedPart[] = literal === "" ? [] : [{ text: literal }];
readings = readings.flatMap((parts) =>
choices.map((choice) => [...parts, ...lead, choice]),
);
literal = "";
parts.push(match.part);
shown = true;
at += match.token.length;
at += matches[0]!.token.length;
}
if (literal !== "") parts.push({ text: literal });
return shown ? parts : undefined;
if (!shown) return [];
return literal === ""
? readings
: readings.map((parts) => [...parts, { text: literal }]);
};
const samePart = (left: SourcePart, right: SourcePart): boolean =>
@@ -160,7 +196,10 @@ const samePart = (left: SourcePart, right: SourcePart): boolean =>
left.as === right.as &&
(left.as !== "clock" ||
right.as !== "clock" ||
left.pattern === right.pattern);
left.pattern === right.pattern) &&
(left.as !== "slug" ||
right.as !== "slug" ||
left.ampersand === right.ampersand);
/** How many entities holding the field and what its parts read spell it as the parts do, or undefined when one holds it otherwise, a non-string value included. */
const heldSpellings = (
@@ -227,12 +266,20 @@ export const createdComposer = (
: [],
),
);
const parts = segmented(text, index, alike);
return parts === undefined ||
parts.some((part) => "from" in part && part.from === field) ||
!((heldSpellings(parts, field, held) ?? 0) > 0)
? undefined
: { field, rule: "composed", parts };
const fits = segmentations(text, index, alike).flatMap((parts) => {
if (parts.some((part) => "from" in part && part.from === field))
return [];
// A slug whose source here holds no `&` spells it as the held entities do.
const spelled = [parts, ampersandWord(parts, entity)].find(
(candidate) =>
candidate !== undefined &&
(heldSpellings(candidate, field, held) ?? 0) > 0,
);
return spelled === undefined ? [] : [spelled];
});
return fits.length === 1
? { field, rule: "composed", parts: fits[0]! }
: undefined;
};
};
@@ -270,10 +317,12 @@ export const composedChange = (
),
),
);
const parts = segmented(text, index, samePart);
return parts === undefined ||
composedValue(parts, before) !== ownMember(before, field) ||
heldSpellings(parts, field, entity.others) === undefined
? undefined
: { field, rule: "composed", parts };
const fits = segmentations(text, index, samePart).filter(
(parts) =>
composedValue(parts, before) === ownMember(before, field) &&
heldSpellings(parts, field, entity.others) !== undefined,
);
return fits.length === 1
? { field, rule: "composed", parts: fits[0]! }
: undefined;
};
src/domain/analysisView/environmentStateWriteCreate.ts
+86 -26
View File
@@ -42,6 +42,7 @@ import {
refuse,
requestBodyPlan,
retainFixed,
retainCreateExemplar,
searchFrom,
startOf,
UNKNOWN,
@@ -53,6 +54,7 @@ import {
import {
createdInteger,
explainCreate,
type CreateEvidence,
insertCandidates,
insertIndex,
randomIdentities,
@@ -77,7 +79,11 @@ import {
fittedClockFormat,
noteFixedWidths,
} from "./environmentStateWriteClockFormats.js";
import { mergedCreatePlan } from "./environmentStateWriteCreateCalls.js";
import {
mergedCreatePlan,
noteAmpersands,
noteHeldAmpersands,
} from "./environmentStateWriteCreateCalls.js";
import {
containsDelimited,
generatedIntegerDigits,
@@ -270,27 +276,77 @@ const refineCreate = (
linkedUnexplained(explained.rules, evidence.observed.object, held)
)
return false;
const merged = mergedCreatePlan(
{ rules: draft.plan.rules, body: draft.plan.body! },
draft.observed!,
const later = laterOf(evidence, explained);
const widened =
mergedCreatePlan(
{ rules: draft.plan.rules, body: draft.plan.body! },
draft.observed!,
later,
earlierOf(draft),
) ?? reexplained(draft, explained.rules, later);
if (widened === undefined) return false;
retainFixed(context, widened.body.fixed.slice(draft.plan.body!.fixed.length));
draft.plan.rules = widened.rules;
draft.plan.body = widened.body;
draft.exemplarInput = undefined;
noteAmpersands(draft.ampersands, explained.rules, evidence.observed.object);
return true;
};
/** What the plan's earlier calls showed, for merging a later one. */
const earlierOf = (draft: WriteDraft) => ({
kept: draft.kept,
clockOf: (field: string) => clockEvidence(draft, field),
ampersands: draft.ampersands,
});
/** A later create as the plan merge reads it. */
const laterOf = (
evidence: CreateCallEvidence,
explained: ReturnType<typeof explainCreate>,
) => ({
entity: evidence.observed.object,
body: evidence.call.body,
rules: explained.rules,
contract: {
...explained.body,
...(evidence.envelope === undefined ? {} : { envelope: evidence.envelope }),
},
});
/**
* The plan its exemplar gives again with the composed rules a later call learned for fields the exemplar left
* unexplained, such as an address spelled from an id that equals the title's slug, merged with that call. Undefined
* once a later call widened the plan, or unless the rules spell the exemplar's own values.
*/
const reexplained = (
draft: WriteDraft,
learned: readonly FieldRule[],
later: ReturnType<typeof laterOf>,
) => {
const input = draft.exemplarInput;
if (input === undefined) return undefined;
const hints = learned.filter(
(rule) =>
rule.rule === "composed" &&
!draft.plan.rules.some((held) => held.field === rule.field),
);
if (hints.length === 0) return undefined;
const again = explainCreate({ ...input, hints });
if (!hints.every((hint) => again.rules.includes(hint))) return undefined;
const envelope = draft.plan.body!.envelope;
return mergedCreatePlan(
{
entity: evidence.observed.object,
body: evidence.call.body,
rules: explained.rules,
contract: {
...explained.body,
...(evidence.envelope === undefined
? {}
: { envelope: evidence.envelope }),
rules: again.rules,
body: {
...again.body,
...(envelope === undefined ? {} : { envelope }),
},
},
{ kept: draft.kept, clockOf: (field) => clockEvidence(draft, field) },
draft.observed!,
later,
earlierOf(draft),
);
if (merged === undefined) return false;
retainFixed(context, merged.body.fixed.slice(draft.plan.body!.fixed.length));
draft.plan.rules = merged.rules;
draft.plan.body = merged.body;
return true;
};
type CreateCallEvidence = {
@@ -310,7 +366,7 @@ const explainedCall = (
const type = draft.plan.target!;
const members = fieldsOf(context.state.members(type, evidence.collection));
const held = fieldsOf(context.state.ofType(type));
const explained = explainCreate({
const input: CreateEvidence = {
entity: evidence.observed.object,
call: evidence.call,
idField: context.idField(type),
@@ -318,8 +374,8 @@ const explainedCall = (
collection: members,
type: held,
named: (id) => context.state.named(id),
});
return { explained, members, held };
};
return { explained: explainCreate(input), members, held, input };
};
/** Build a create plan from its exemplar's observation. */
@@ -331,7 +387,7 @@ const buildCreate = (
): boolean => {
const type = draft.plan.target!;
const idField = context.idField(type);
const { explained, members, held } = explainedCall(
const { explained, members, held, input } = explainedCall(
context,
draft,
step,
@@ -398,6 +454,9 @@ const buildCreate = (
insert: draft.candidates[0]!,
});
draft.observed = evidence.observed.object;
draft.exemplarInput = retainCreateExemplar(context, input, explained.rules);
noteHeldAmpersands(draft.ampersands, explained.rules, held);
noteAmpersands(draft.ampersands, explained.rules, evidence.observed.object);
for (const entity of held) noteFixedWidths(draft.fixedWidth, entity);
return true;
};
@@ -486,12 +545,13 @@ const followsCreateRules = (
draft.plan.rules.every((rule) => {
const value = ownMember(created.fields, rule.field);
const exemplar = ownMember(draft.observed, rule.field);
if (rule.rule === "composed" || rule.rule === "reference")
// Serving always rebuilds a composed or templated field, so a response that leaves it out contradicts the plan.
if (rule.rule === "composed" || rule.rule === "template")
return value === ownMember(created.observed, rule.field);
if (rule.rule === "reference")
return (
!Object.hasOwn(created.observed, rule.field) ||
value === ownMember(created.observed, rule.field) ||
(rule.rule === "reference" &&
same(value, ownMember(created.observed, rule.field)))
same(value, ownMember(created.observed, rule.field))
);
if (rule.rule === "clock")
return (
src/domain/analysisView/environmentStateWriteCreateCalls.ts
+147
View File
@@ -2,11 +2,13 @@ import type { JsonValue } from "../jsonValue.js";
import { ownMember, same, type JsonObject } from "./environmentStateRules.js";
import { storedKinds } from "./environmentStateWriteApply.js";
import { fittedClockFormat } from "./environmentStateWriteClockFormats.js";
import { composedValue } from "./environmentStateWriteComposed.js";
import {
alikeRules,
type FieldRule,
type WriteBody,
} from "./environmentStateWriteRules.js";
import { slugOf, type Ampersand } from "./environmentStateWriteValues.js";
type Default = { readonly member: string; readonly value: JsonValue };
@@ -14,8 +16,13 @@ type Default = { readonly member: string; readonly value: JsonValue };
export interface EarlierEvidence {
readonly kept: ReadonlySet<string>;
readonly clockOf: (field: string) => JsonValue | undefined;
/** How the plan's accepted calls and the type's held entities spelled `&` in each field's slug; absent where none showed one. */
readonly ampersands?: ReadonlyMap<string, AmpersandSpelling>;
}
/** How a field's slug showed `&`: dropped, as the word, or in a way neither spelling gives. */
export type AmpersandSpelling = "dropped" | Ampersand | "unknown";
/** The plan's rule for a field and a later call's agree: equal, or clocks one spelling covers given what earlier calls showed. */
const agreeing = (
held: FieldRule,
@@ -24,10 +31,18 @@ const agreeing = (
readonly field: string;
readonly later: JsonValue | undefined;
readonly earlier: EarlierEvidence;
readonly exemplar: JsonObject;
readonly laterEntity: JsonObject;
},
): FieldRule | undefined => {
const alike = alikeRules(held, later, values.earlier.kept);
if (alike !== undefined) return alike;
const spelled = ampersandSpelled(
held,
{ rule: later, entity: values.laterEntity },
values.earlier.ampersands?.get(values.field),
);
if (spelled !== undefined) return spelled;
if (held.rule !== "clock" || later.rule !== "clock") return undefined;
const format = fittedClockFormat(
held.format,
@@ -37,6 +52,134 @@ const agreeing = (
return format === undefined ? undefined : { ...held, format };
};
/** A rule with its slugs spelled the default way. */
const unspelled = (rule: FieldRule): FieldRule => {
if (rule.rule === "slug") {
const { ampersand: _ampersand, ...plain } = rule;
return plain;
}
if (rule.rule !== "composed") return rule;
return {
...rule,
parts: rule.parts.map((part) =>
"as" in part && part.as === "slug"
? { from: part.from, as: "slug" as const }
: part,
),
};
};
/** Whether a call's entity holds `&` in a field the rule slugs, which shows how it spells one. */
const showsAmpersand = (rule: FieldRule, entity: JsonObject): boolean =>
(rule.rule === "slug"
? [rule.from]
: rule.rule === "composed"
? rule.parts.flatMap((part) =>
"as" in part && part.as === "slug" ? [part.from] : [],
)
: []
).some((field) => {
const value = ownMember(entity, field);
return typeof value === "string" && value.includes("&");
});
/**
* Two calls' rules that differ only in how a slug spells `&`: the one that spells it as the word, when the call
* spelling it otherwise slugged no `&`, and no call the plan accepted nor entity the type held dropped one (`shown`).
*/
export const ampersandSpelled = (
held: FieldRule,
later: { readonly rule: FieldRule; readonly entity: JsonObject },
shown: AmpersandSpelling | undefined,
): FieldRule | undefined => {
const plainHeld = unspelled(held);
if (!same(plainHeld, unspelled(later.rule))) return undefined;
const heldSpells = !same(held, plainHeld);
if (heldSpells === !same(later.rule, unspelled(later.rule))) return undefined;
if (heldSpells)
return showsAmpersand(later.rule, later.entity) ? undefined : held;
return shown === "dropped" || shown === "unknown" ? undefined : later.rule;
};
/** How a rule's slug spells `&`, for a rule that slugs a field. */
const slugSpelling = (rule: FieldRule): AmpersandSpelling | undefined => {
if (rule.rule === "slug") return rule.ampersand ?? "dropped";
if (rule.rule !== "composed") return undefined;
for (const part of rule.parts)
if ("as" in part && part.as === "slug") return part.ampersand ?? "dropped";
return undefined;
};
/** Note how a call that slugged `&` spelled it in each field its own rules slug. */
export const noteAmpersands = (
ampersands: Map<string, AmpersandSpelling>,
rules: readonly FieldRule[],
entity: JsonObject,
): void => {
for (const rule of rules) {
const spelling = slugSpelling(rule);
if (spelling !== undefined && showsAmpersand(rule, entity))
ampersands.set(rule.field, spelling);
}
};
/** Exact spelling observations in held entities; absence and contradictory spellings remain distinct. */
export const heldAmpersandSpellings = (
rule: FieldRule,
held: readonly JsonObject[],
): ReadonlySet<AmpersandSpelling> => {
const observed = new Set<AmpersandSpelling>();
if (slugSpelling(rule) === undefined) return observed;
const plain = unspelled(rule);
for (const entity of held) {
if (!showsAmpersand(rule, entity)) continue;
const stored = ownMember(entity, rule.field);
if (stored === undefined) continue;
const source =
rule.rule === "slug" ? ownMember(entity, rule.from) : undefined;
const spelled = (spelling: Ampersand | undefined): string | undefined =>
rule.rule === "slug"
? typeof source === "string"
? slugOf(source, spelling)
: undefined
: plain.rule === "composed"
? composedValue(
plain.parts.map((part) =>
"as" in part && part.as === "slug" && spelling !== undefined
? { ...part, ampersand: spelling }
: part,
),
entity,
)
: undefined;
observed.add(
spelled(undefined) === stored
? "dropped"
: spelled("and") === stored
? "and"
: "unknown",
);
}
return observed;
};
/** Retain the constraints later rule refinement needs from the same owned observations. */
export const noteHeldAmpersands = (
ampersands: Map<string, AmpersandSpelling>,
rules: readonly FieldRule[],
held: readonly JsonObject[],
): void => {
for (const rule of rules) {
const observed = heldAmpersandSpellings(rule, held);
const known = ampersands.get(rule.field);
if (observed.has("dropped")) ampersands.set(rule.field, "dropped");
else if (observed.has("unknown") && known !== "dropped")
ampersands.set(rule.field, "unknown");
else if (observed.has("and") && known === undefined)
ampersands.set(rule.field, "and");
}
};
/** Defaults joined, undefined when two calls showed one member left out at different values. */
const joinedDefaults = (
held: readonly Default[],
@@ -80,6 +223,8 @@ const addedReference = (
field,
later: later.entity[field],
earlier: calls.earlier,
exemplar: calls.exemplar,
laterEntity: later.entity,
});
return rule === undefined ? undefined : { rule };
};
@@ -113,6 +258,8 @@ const mergedField = (
field,
later: later.entity[field],
earlier: calls.earlier,
exemplar,
laterEntity: later.entity,
});
return rule === undefined ? undefined : { rule };
}
src/domain/analysisView/environmentStateWriteEffects.ts
+60 -4
View File
@@ -51,6 +51,12 @@ import {
} from "./environmentStateWriteObserve.js";
import { plannedBody, restampedFields } from "./environmentStateWriteStamps.js";
import { readClock } from "./environmentStateWriteValues.js";
import type { CreateEvidence } from "./environmentStateWriteExplain.js";
import {
noteAmpersands,
noteHeldAmpersands,
type AmpersandSpelling,
} from "./environmentStateWriteCreateCalls.js";
import {
fittedClockFormat,
matchesClockFormat,
@@ -98,6 +104,10 @@ export interface WriteDraft {
readonly kept: Set<string>;
/** A value a held entity or an accepted call showed each clock field at a fixed width in, which keeps it untrimmed. */
readonly fixedWidth: Map<string, JsonValue>;
/** What explained a create's exemplar, kept until a later call widens the plan, which it may then explain again. */
exemplarInput?: CreateEvidence | undefined;
/** How the calls a create plan accepted and the entities its type held spelled `&` in each slugged field. */
readonly ampersands: Map<string, AmpersandSpelling>;
}
/** The value a draft fits a clock field against: one showing it at a fixed width, else the exemplar's. */
@@ -399,7 +409,10 @@ const refineUpdate = (
const shown = later.get(rule.field);
return shown === undefined
? rule
: (reconciledUpdateRule(rule, shown, draft.kept) ?? rule);
: (reconciledUpdateRule(rule, shown, draft.kept, {
entity: evidence.after.object,
ampersands: draft.ampersands,
}) ?? rule);
}),
...evidence.explained.rules.filter((rule) => !ruled.has(rule.field)),
];
@@ -413,6 +426,40 @@ export const retainFixed = (
if (fixed.length > 0) context.io.retain(64 + jsonTextCodeUnits(fixed) * 2);
};
/** Retain only exemplars that may acquire a composed rule; state already owns and charges their entity facts. */
export const retainCreateExemplar = (
context: EffectContext,
input: CreateEvidence,
rules: readonly FieldRule[],
): CreateEvidence | undefined => {
if (
!Object.entries(input.entity).some(
([field, value]) =>
typeof value === "string" &&
!rules.some((rule) => rule.field === field),
)
)
return undefined;
// Snapshot arrays borrow immutable entity fields whose state charges last
// throughout simulation. Their containers and the request reader's temporary
// facts need their own retained charge before the draft adopts them.
const references = [...new Set([input.collection, input.type])].reduce(
(count, items) => count + items.length,
0,
);
context.io.retain(
192 +
references * 8 +
jsonTextCodeUnits(input.call.body) * 2 +
input.call.path.length * 2 +
[...input.call.pathValues].reduce(
(bytes, [key, value]) => bytes + 64 + (key.length + value.length) * 2,
0,
),
);
return input;
};
/** An update's targets, and its recorded outcome: the entity takes what its observation shows, attributed to no tool. */
const outcomes = (
context: EffectContext,
@@ -476,8 +523,13 @@ const buildUpdate = (
rules: evidence.explained.rules,
});
draft.observed = evidence.after.object;
for (const entity of context.state.ofType(draft.plan.target!))
noteFixedWidths(draft.fixedWidth, entity.fields);
const held = context.state.ofType(draft.plan.target!);
for (const entity of held) noteFixedWidths(draft.fixedWidth, entity.fields);
noteHeldAmpersands(
draft.ampersands,
evidence.explained.rules,
held.map((entity) => entity.fields),
);
};
/**
@@ -596,6 +648,7 @@ export const updateEntity = (
draft.plan.rules = rules;
noteFixedWidths(draft.fixedWidth, after.object);
buildUpdate(context, draft, step, { after, explained, idField });
noteAmpersands(draft.ampersands, explained.rules, after.object);
// Serving reads the plan's observation, so the gate does too; generated values are this call's.
const written = updatedFields(
draft.plan,
@@ -740,4 +793,7 @@ const contradictsPlan = (
): boolean =>
contradictsEarlierCalls(draft, explained.rules) ||
contradictsGeneratedClock(before, after, draft.plan.rules, explained) ||
conflictsWithPlan(draft.plan, explained, draft.kept);
conflictsWithPlan(draft.plan, explained, draft.kept, {
entity: after,
ampersands: draft.ampersands,
});
src/domain/analysisView/environmentStateWriteExplain.ts
+110 -23
View File
@@ -1,3 +1,4 @@
import { heldAmpersandSpellings } from "./environmentStateWriteCreateCalls.js";
import type { JsonValue } from "../jsonValue.js";
import { compareUnicodeCodePoints as compare } from "../unicodeCodePointOrder.js";
import {
@@ -26,11 +27,14 @@ import {
isUuid,
readClock,
sequenceNumber,
slugOf,
slugSpellings,
sortedStrings,
type ClockFormat,
} from "./environmentStateWriteValues.js";
import { createdComposer } from "./environmentStateWriteComposed.js";
import {
composedValue,
createdComposer,
} from "./environmentStateWriteComposed.js";
import { referenceRule } from "./environmentStateWriteReferences.js";
import {
fittedClockFormat,
@@ -60,6 +64,8 @@ import {
heldClock,
sameSecondClocks,
sentRule,
slugRule,
type SlugSource,
type UpdateEvidence,
} from "./environmentStateWriteChanges.js";
@@ -74,6 +80,8 @@ export interface CreateEvidence {
readonly type: readonly JsonObject[];
/** The entities of any type the state holds under an id. */
readonly named?: (id: string) => readonly StateEntity[];
/** Composed rules another call of the tool learned, which explain a field here when they spell its value. */
readonly hints?: readonly FieldRule[];
}
/** What explaining a create looks values up in, built once per request body. */
@@ -81,9 +89,24 @@ interface BodyIndex {
/** Each body string long enough to show a copy, at its shallowest pointer (breadth-first, members in order). */
readonly copies: ReadonlyMap<string, string>;
/** Each slug a top-level string member slugs to, other than its own value, from the first such member. */
readonly slugs: ReadonlyMap<string, string>;
readonly slugs: ReadonlyMap<string, SlugSource>;
}
/** Add a string's slugs to an index, keeping the first member that spells each; `own` keeps one equal to the string. */
const addSlugs = (
slugs: Map<string, SlugSource>,
member: string,
sent: string,
own = false,
): void => {
for (const { slug, ampersand } of slugSpellings(sent))
if ((own || slug !== sent) && !slugs.has(slug))
slugs.set(slug, {
from: member,
...(ampersand === undefined ? {} : { ampersand }),
});
};
/** Index a request body once, so explaining each field looks values up rather than walking the body again. */
const indexBody = (body: JsonObject): BodyIndex => {
const copies = new Map<string, string>();
@@ -109,12 +132,9 @@ const indexBody = (body: JsonObject): BodyIndex => {
pointer: `${next.pointer}${pointerTo(key)}`,
});
}
const slugs = new Map<string, string>();
const slugs = new Map<string, SlugSource>();
for (const [member, sent] of Object.entries(body))
if (typeof sent === "string") {
const slug = slugOf(sent);
if (slug !== sent && !slugs.has(slug)) slugs.set(slug, member);
}
if (typeof sent === "string") addSlugs(slugs, member, sent);
return { copies, slugs };
};
@@ -165,10 +185,8 @@ const requestRule = (
if (text !== undefined && text === pathValue)
return { field, rule: "path", from: name };
const slugged = typeof value === "string" && value.length >= 3;
const member = slugged ? index.slugs.get(value) : undefined;
return member === undefined
? undefined
: { field, rule: "slug", from: member };
const source = slugged ? index.slugs.get(value) : undefined;
return source === undefined ? undefined : slugRule(field, source);
};
/** A clock's format, trimmed when the type's values of the field show its fraction trimmed and none shows a fixed width. */
@@ -290,6 +308,36 @@ const bodyOf = (
});
/** Explain every field of a created entity: echoed, copied, path, slug, generated, template or constant (unlisted). */
/**
* A slug rule as the held entities spell `&`: as the word when the call slugged none and they spell it so, and no rule
* when they spell it both ways, or otherwise than the call did or in a way neither spelling gives. Other rules pass
* unchanged.
*/
const heldSlugRule = (
rule: FieldRule | undefined,
sent: JsonObject,
held: readonly JsonObject[],
): FieldRule | undefined => {
if (rule?.rule !== "slug") return rule;
const shown = heldAmpersandSpellings(rule, held);
const source = ownMember(sent, rule.from);
if (typeof source === "string" && source.includes("&"))
return shown.has("unknown") ||
shown.has(rule.ampersand === "and" ? "dropped" : "and")
? undefined
: rule;
if (shown.has("and") && shown.has("dropped")) return undefined;
return shown.has("and") && !shown.has("unknown")
? { ...rule, ampersand: "and" }
: rule;
};
/** Whether a composed string spells a field besides the one a template rebuilds it around, such as a title's slug beside a generated id. */
const spellsMore = (composed: FieldRule, template: FieldRule): boolean =>
composed.rule === "composed" &&
template.rule === "template" &&
composed.parts.some((part) => "from" in part && part.from !== template.from);
export const explainCreate = (
evidence: CreateEvidence,
): {
@@ -300,9 +348,15 @@ export const explainCreate = (
const ruled: FieldRule[] = [];
const unexplained: string[] = [];
const index = indexBody(evidence.call.body);
// The member each slug the held entities' `&` spellings refused was slugged from.
const refusedSlugs = new Map<string, string>();
for (const [field, value] of Object.entries(evidence.entity)) {
const requested = requestRule(evidence, index, field, value);
const held = heldSlugRule(requested, evidence.call.body, evidence.type);
if (requested?.rule === "slug" && held === undefined)
refusedSlugs.set(field, requested.from);
const rule =
requestRule(evidence, index, field, value) ??
held ??
generatedRule(evidence, field, value) ??
(evidence.named === undefined
? undefined
@@ -330,10 +384,26 @@ export const explainCreate = (
const constants: string[] = [];
const composed = createdComposer(evidence.entity, rules, evidence.type);
for (const field of unexplained) {
const template =
templateRule(evidence.entity, field, sources) ?? composed(field);
if (template === undefined) constants.push(field);
else rules.push(template);
const hinted = evidence.hints?.find(
(rule) =>
rule.field === field &&
rule.rule === "composed" &&
composedValue(rule.parts, evidence.entity) ===
ownMember(evidence.entity, field),
);
if (hinted !== undefined) {
rules.push(hinted);
continue;
}
const template = templateRule(evidence.entity, field, sources);
const spelled = composed(field);
const rule =
spelled !== undefined &&
(template === undefined || spellsMore(spelled, template))
? spelled
: template;
if (rule === undefined) constants.push(field);
else rules.push(rule);
}
const tokens = [...evidence.call.pathValues.values()].filter(
(value) => value.length > 0,
@@ -358,6 +428,11 @@ export const explainCreate = (
evidence.named ?? (() => []),
),
...instantMembers(evidence.call.body, evidence.entity, constants),
// A refused slug kept as recorded is right only for the name it was slugged from.
...constants.flatMap((field) => {
const from = refusedSlugs.get(field);
return from === undefined ? [] : [from];
}),
]);
const pinned = unreferenced(
rules,
@@ -392,12 +467,11 @@ export const explainCreate = (
const echoedSlugs = (
body: JsonObject,
echoed: ReadonlySet<string>,
): Map<string, string> => {
const slugs = new Map<string, string>();
): Map<string, SlugSource> => {
const slugs = new Map<string, SlugSource>();
for (const member of echoed) {
const sent = ownMember(body, member);
if (typeof sent === "string" && !slugs.has(slugOf(sent)))
slugs.set(slugOf(sent), member);
if (typeof sent === "string") addSlugs(slugs, member, sent, true);
}
return slugs;
};
@@ -431,6 +505,7 @@ export const explainUpdate = (
readonly unstamped: string[];
} => {
const { body } = evidence.call;
const knownEntities = [evidence.before, ...evidence.others];
const rules: FieldRule[] = [];
const echoed = new Set<string>();
const unexplained: string[] = [];
@@ -456,7 +531,15 @@ export const explainUpdate = (
continue;
}
if (!Object.hasOwn(evidence.after, member)) continue;
const rule = sentRule(evidence, member, held);
const rule = heldSlugRule(
sentRule(evidence, member, held),
body,
knownEntities,
);
if (rule === undefined) {
unexplained.push(member);
continue;
}
rules.push(rule);
if (rule.rule === "echo") echoed.add(member);
else if (rule.rule !== "constant") generated.add(member);
@@ -484,7 +567,11 @@ export const explainUpdate = (
if (kept !== undefined) rules.push(kept);
continue;
}
const rule = changeRule(evidence, field, slugs);
const rule = heldSlugRule(
changeRule(evidence, field, slugs),
body,
knownEntities,
);
if (rule === undefined) unexplained.push(field);
else rules.push(rule);
}
src/domain/analysisView/environmentStateWriteRules.ts
+14 -2
View File
@@ -11,6 +11,7 @@ import {
} from "./environmentStateRules.js";
import {
sortedStrings,
type Ampersand,
type ClockFormat,
} from "./environmentStateWriteValues.js";
import {
@@ -89,7 +90,13 @@ export type FieldRule =
| { readonly field: string; readonly rule: "echo"; readonly sorted?: true }
| { readonly field: string; readonly rule: "copy"; readonly from: string }
| { readonly field: string; readonly rule: "path"; readonly from: string }
| { readonly field: string; readonly rule: "slug"; readonly from: string }
/** A slug of the sent member, spelling `&` as the word when `ampersand` says so. */
| {
readonly field: string;
readonly rule: "slug";
readonly from: string;
readonly ampersand?: Ampersand;
}
| { readonly field: string; readonly rule: "template"; readonly from: string }
/** A string rebuilt from literal text and other written fields, in order. */
| {
@@ -195,7 +202,12 @@ export const keptOrders = (body: JsonObject, stored: JsonObject): string[] =>
/** One part of a composed string: literal text, or another written field as written, slugged, or as a clock's calendar digits. */
export type ComposedPart =
| { readonly text: string }
| { readonly from: string; readonly as: "value" | "slug" }
| { readonly from: string; readonly as: "value" }
| {
readonly from: string;
readonly as: "slug";
readonly ampersand?: Ampersand;
}
| {
readonly from: string;
readonly as: "clock";
src/domain/analysisView/environmentStateWriteValues.ts
+13 -2
View File
@@ -305,14 +305,25 @@ const fillMask = (mask: string, value: number): string => {
};
/** Lower case, with every run of other characters joined by one `-` and none at either end. */
export const slugOf = (text: string): string =>
text
/** How a slug spells `&`: dropped when absent, as Django does, or as the word, as Go's gosimple/slug and JavaScript's slugify do. */
export type Ampersand = "and";
export const slugOf = (text: string, ampersand?: Ampersand): string =>
(ampersand === "and" ? text.replaceAll("&", " and ") : text)
.normalize("NFKD")
.replace(/[̀-ͯ]/gu, "")
.toLowerCase()
.replace(/[^a-z0-9]+/gu, "-")
.replace(/^-+|-+$/gu, "");
/** A text's slugs, each with the spelling that gives it: also with `&` as the word when the text holds one. */
export const slugSpellings = (
text: string,
): readonly { readonly slug: string; readonly ampersand?: Ampersand }[] =>
text.includes("&")
? [{ slug: slugOf(text) }, { slug: slugOf(text, "and"), ampersand: "and" }]
: [{ slug: slugOf(text) }];
const isAlphanumeric = (character: string | undefined): boolean =>
character !== undefined && /[\p{L}\p{N}]/u.test(character);
src/domain/browserScenario.ts
+75 -1
View File
@@ -14,7 +14,10 @@ import {
type BrowserScenarioAction,
type BrowserScenarioUrl,
type BrowserScenarioValue,
SAVED_URL_PLACEHOLDER,
savedPlaceholderNames,
} from "./browserScenarioValues.js";
import type { JsonValue } from "./jsonValue.js";
export const browserScenarioInputSchema = z.strictObject({
browser: browserScenarioBrowserSchema.describe(
@@ -61,6 +64,47 @@ const secretReferencesInValue = (
value: BrowserScenarioValue | undefined,
): readonly string[] => (value?.source === "secret" ? [value.secret_id] : []);
/** A request step's header values, with Basic credentials as their two parts. */
const headerValues = (
action: Extract<BrowserScenarioAction, { action: "request" }>,
) =>
action.headers.flatMap(({ value }) =>
value.source === "basic_credentials" ? value.parts : [value],
);
/** Every string in a JSON value, at any depth. */
const jsonStrings = (value: JsonValue | undefined): readonly string[] => {
if (typeof value === "string") return [value];
if (Array.isArray(value)) return value.flatMap(jsonStrings);
if (value !== null && typeof value === "object")
return Object.values(value).flatMap(jsonStrings);
return [];
};
/** The saved values a step sends: `{saved:name}` in a request path, JSON string or goto URL, and saved header values and credential parts. */
const sentSavedNames = (action: BrowserScenarioAction): readonly string[] => {
if (action.action === "goto")
return savedPlaceholderNames(action.destination.url, SAVED_URL_PLACEHOLDER);
if (action.action !== "request") return [];
return [
...[action.path, ...jsonStrings(action.json)].flatMap((text) =>
savedPlaceholderNames(text),
),
...headerValues(action).flatMap((value) =>
value.source === "saved" ? [value.name] : [],
),
];
};
/** The names a step saves: a request's `save`, or a UI step's `save_response`. */
const savedNames = (action: BrowserScenarioAction): readonly string[] =>
action.action === "request"
? action.save.map(({ name }) => name)
: (action.action === "click" || action.action === "press") &&
action.save_response !== undefined
? action.save_response.save.map(({ name }) => name)
: [];
const destinationSecretReferences = (
destination: BrowserScenarioUrl,
): readonly string[] =>
@@ -76,13 +120,31 @@ const actionSecretReferences = (
if (action.action === "request")
return [
...action.query.flatMap(({ value }) => secretReferencesInValue(value)),
...action.headers.flatMap(({ value }) =>
...headerValues(action).flatMap((value) =>
value.source === "secret" ? [value.secret_id] : [],
),
];
return [];
};
/** Each saved value a step sends was saved by an earlier step. */
const validateSavedValues = (
scenario: ScenarioShape,
context: z.RefinementCtx,
): void => {
const saved = new Set<string>();
scenario.actions.forEach((action, index) => {
for (const name of new Set(sentSavedNames(action)))
if (!saved.has(name))
addIssue(
context,
["actions", index],
`Step ${action.step_id} sends saved value ${name}, which no earlier step saves`,
);
for (const name of savedNames(action)) saved.add(name);
});
};
/** Request steps stay on the start origin and send a body only with methods that carry one. */
const validateRequestSteps = (
scenario: ScenarioShape,
@@ -132,6 +194,17 @@ const validateRequestSteps = (
["actions", index, "headers", position, "value", "value"],
"A header value cannot carry line breaks or control characters",
);
// Evidence records literals as written, so a password must come from a reference.
const password =
header.value.source === "basic_credentials"
? header.value.parts[1]
: undefined;
if (password?.source === "literal" && password.value !== "")
addIssue(
context,
["actions", index, "headers", position, "value", "parts", 1],
"Basic credentials take their password from a declared secret, a saved value, a cookie or local storage, not a literal",
);
});
});
};
@@ -198,6 +271,7 @@ const validateBrowserScenario = (
);
validateRequestSteps(scenario, context);
validateSavedValues(scenario, context);
const references = scenario.actions.flatMap(actionSecretReferences);
references.push(...destinationSecretReferences(scenario.start_url));
references.push(...validateStorage(scenario));
src/domain/browserScenarioValues.ts
+98 -2
View File
@@ -138,6 +138,93 @@ const browserScenarioPageValueSchemas = [
}),
] as const;
/** A path on the start origin: one leading slash, never `//` or `/\`. Written without look-around or reserved class syntax, so every JSON Schema regex dialect reads it. */
const START_ORIGIN_PATH = /^\x2F(?:[^\x2F\x5C]|$)/u;
/** `{saved:name}` as written, which sends a saved value in a request path or a JSON body string. */
export const SAVED_PLACEHOLDER = /\{saved:([A-Za-z][A-Za-z0-9._-]*)\}/gu;
/**
* `{saved:name}` in a goto destination's URL, which the scenario schema parsed, so it may spell the braces and the colon
* percent-encoded. Other text keeps an encoded spelling as written.
*/
export const SAVED_URL_PLACEHOLDER =
/(?:\{|%7[Bb])saved(?::|%3[Aa])([A-Za-z][A-Za-z0-9._-]*)(?:\}|%7[Dd])/gu;
/** The saved value names a text's placeholders send, as `placeholder` spells them. */
export const savedPlaceholderNames = (
text: string,
placeholder: RegExp = SAVED_PLACEHOLDER,
): readonly string[] =>
[...text.matchAll(placeholder)].map(([, name]) => name!);
/** A value an earlier step saved from a JSON response. */
const savedValueSchema = z.strictObject({
source: z.literal("saved"),
name: scenarioIdentifierSchema,
});
/** A literal, declared secret or saved value a request step sends. */
export const browserScenarioRequestValueSchema = z.discriminatedUnion(
"source",
[...browserScenarioValueSchema.options, savedValueSchema],
);
export type BrowserScenarioRequestValue = z.infer<
typeof browserScenarioRequestValueSchema
>;
/**
* A part of Basic credentials: a literal, declared secret or saved value, or a value read from the page, as a frontend
* builds such a header from a login token. The page values carry no descriptions here, since each part repeats them.
*/
const basicCredentialPartSchema = z.discriminatedUnion("source", [
...browserScenarioRequestValueSchema.options,
z.strictObject({ source: z.literal("cookie"), name: z.string().min(1) }),
z.strictObject({
source: z.literal("local_storage"),
key: z.string().min(1),
json_pointer: z
.string()
.regex(/^(?:\/.*)?$/u)
.optional(),
}),
]);
export type BrowserScenarioCredentialPart = z.infer<
typeof basicCredentialPartSchema
>;
/** A username and password sent as the base64 of `username:password`, as HTTP Basic auth spells them; redacted like a declared secret. */
const basicCredentialsSchema = z.strictObject({
source: z.literal("basic_credentials"),
parts: z
.array(basicCredentialPartSchema)
.length(2)
.describe("Username, then password."),
});
/** A string or number a step keeps from a JSON response, by name, for later steps. */
const saveEntrySchema = z.strictObject({
name: scenarioIdentifierSchema,
json_pointer: z.string().regex(/^(?:\/.*)?$/u),
secret: z.boolean().default(false),
});
export type BrowserScenarioSaveEntry = z.infer<typeof saveEntrySchema>;
/** Values a UI step keeps from a newly observed matching response; matching does not prove causality. */
const saveResponseSchema = z
.strictObject({
method: z.enum(BROWSER_SCENARIO_REQUEST_METHODS),
path: z
.string()
.regex(START_ORIGIN_PATH)
.describe("`{*}` matches one segment; the query is ignored."),
save: z.array(saveEntrySchema).min(1),
})
.describe(
"Save values from a newly observed matching 2xx JSON response; matching does not prove the UI caused it. Multiple observed matches fail.",
);
export type BrowserScenarioSaveResponse = z.infer<typeof saveResponseSchema>;
const requestHeaderSchema = z.strictObject({
name: z
.string()
@@ -149,6 +236,8 @@ const requestHeaderSchema = z.strictObject({
value: z.discriminatedUnion("source", [
...browserScenarioValueSchema.options,
...browserScenarioPageValueSchemas,
savedValueSchema,
basicCredentialsSchema,
]),
prefix: z
.string()
@@ -284,6 +373,7 @@ export const browserScenarioActionSchema = z.discriminatedUnion("action", [
locator: locatorSchema,
button: z.enum(["left", "middle", "right"]).default("left"),
click_count: z.int().positive().default(1),
save_response: saveResponseSchema.optional(),
}),
z.strictObject({
...stepBase,
@@ -296,6 +386,7 @@ export const browserScenarioActionSchema = z.discriminatedUnion("action", [
action: z.literal("press"),
locator: locatorSchema,
key: z.string().min(1),
save_response: saveResponseSchema.optional(),
}),
z.strictObject({
...stepBase,
@@ -320,8 +411,7 @@ export const browserScenarioActionSchema = z.discriminatedUnion("action", [
method: z.enum(BROWSER_SCENARIO_REQUEST_METHODS),
path: z
.string()
// Preserve the prefix grammar without look-around or reserved class syntax.
.regex(/^\x2F(?:[^\x2F\x5C]|$)/u)
.regex(START_ORIGIN_PATH)
.describe(
"Path, and optional literal query, on the start URL's origin, such as /api/folders; it must resolve to that origin.",
),
@@ -332,6 +422,12 @@ export const browserScenarioActionSchema = z.discriminatedUnion("action", [
.describe(
"JSON request body, sent with Content-Type application/json unless a header sets one; not allowed for GET or HEAD.",
),
save: z
.array(saveEntrySchema)
.default([])
.describe(
"Values to save from the JSON response. Later steps send one as `{saved:name}` in a request path, goto URL or JSON string, or as a header value.",
),
expected_status: z
.array(
z
src/domain/environmentBundleExport.ts
+1
View File
@@ -105,6 +105,7 @@ export const environmentBuildReportSchema = z.strictObject({
"name-from-file",
"credential-named-members",
"credential-scan-incomplete",
"credential-named-saved-values",
]),
message: z.string(),
pointers: z
src/generatedPackageMetadata.ts
+1 -1
View File
@@ -6,5 +6,5 @@ export const PACKAGE_METADATA = {
serverSdkVersion: "2.3.1",
clientSdkVersion: "2.3.1",
coreSdkVersion: "2.3.1",
skillVersion: "49",
skillVersion: "50",
} as const;