Files
rea/src/domain/analysisView/environmentStateView.ts
T
538017d1d2 feat(environment): apply evidence-gated write effects to the served state (#5600)
* feat(environment): render more read tools from the state and describe responses

Keep main's environment serving and fold in coverage for non-entity reads,
response summaries, and MCP tool annotations.

Squashed rebase of #5352 onto main.

* fix(environment): keep rendered documents and quoted samples inside their budgets

A recorded body of exponential numbers can render past 256 KiB and turn a
working read into a state size error, so that document stays a recorded replay.
Description samples are measured after JSON quoting, and the replay-fallback
catalog check uses a tool count the longer response descriptions still separate.

Co-authored-by: morluto <[email protected]>

* feat(environment): apply evidence-gated write effects to the served state

Serve write tools whose later reads confirm the effect, and keep main's
state-file ownership and part-wise reply measurement.

Squashed rebase of #5600 onto the rebased #5352 tip.

* fix(environment): bound and validate write state history

* feat(environment): render more read tools from the state and describe responses

Keep main's environment serving and fold in coverage for non-entity reads,
response summaries, and MCP tool annotations.

Squashed rebase of #5352 onto main.

* fix(environment): keep rendered documents and quoted samples inside their budgets

A recorded body of exponential numbers can render past 256 KiB and turn a
working read into a state size error, so that document stays a recorded replay.
Description samples are measured after JSON quoting, and the replay-fallback
catalog check uses a tool count the longer response descriptions still separate.

Co-authored-by: morluto <[email protected]>

* feat(environment): apply evidence-gated write effects to the served state

Serve write tools whose later reads confirm the effect, and keep main's
state-file ownership and part-wise reply measurement.

Squashed rebase of #5600 onto the rebased #5352 tip.

* fix(environment): bound and validate write state history

* fix(environment): bound additional collection history

* fix(environment): credit sent fields, negative sequences, and encoded targets

A later overwrite copied surviving derived fields into the sent-member
claim, so a members-only check rejected an update whose sent field still
matched. Serving floored sequences at zero and stored missing history as
zero, so a collection sequence learned from negative ranks skipped the
next value. An absence after a delete forgot the resolved collection, and
an encoded read could not observe an update spelled with another
collection encoding.

Regression tests cover each case.

Co-authored-by: morluto <[email protected]>

* test(environment): cover resolved encoded update identities

* fix: validate timed update observations

* fix(environment): share write overlap and check clocks on earlier calls

Observation search skips a read begun before the write ended. That
comparison now uses the same overlap as the check path.

A clock learned on a later call was not checked against earlier calls
that kept the field. Serving would stamp those calls too, so the plan
stays unserved. A clock learned earlier is still refused when a later
call does not advance it.

Regression tests cover an overlapping empty update and a published_at
clock that one recorded call leaves unchanged.

Co-authored-by: morluto <[email protected]>

* fix: preserve partial update evidence

* fix(environment): let an unmodelled write explain only the list members that differ

A failed whole-list read now reports which members alone account for the
difference. An unmodelled write excuses the read's writes only when it came
after each such member's last check and pending writes, so an older change
elsewhere no longer hides a newer write the list shows undone.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 7b6aab9d7191e600051693f2c192afcfc743d083)

* fix(environment): explain added fields, tie creates to serving's collection, and bound write descriptions

An update explains a field the entity lacked before when the observing
view had shown the entity, so a server-filled field is not dropped. A
create is tied to the held collection serving picks for its path. Write
descriptions and body refusals quote member names and sample them within
a byte bound. The request body is indexed once per explanation, so copied
and slugged fields no longer rescan it.

Replays 07aca54e onto the restacked branch. Its rule that a read begun
before a write ended teaches nothing is already here (fb6785dc), so only
its test is kept.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(environment): state how list reads weigh unmodelled writes, added update fields and quoted members

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* refactor(environment): keep the observing view helpers with the observations

shownBefore and the item-or-list fields check move to the observe module,
keeping the write effects within the line limit.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): resolve unmodelled writes onto the held collection

An unmodelled path kept the request's collection spelling. An archive under
the plain spelling of a collection held encoded marked only that plain
collection unknown, so a later update was blamed for the archive's fields
and dropped. Name that entity the way a modelled write does.

Co-authored-by: morluto <[email protected]>

* fix(environment): confirm no write through volatile fields or overwritten replacements, and bound the served clock

A members-only check no longer compares volatile fields, which render
from the read itself. A document replacement credits the members it
wrote or removed at its object, so a later merge that overwrites them
takes its confirmation away. A recorded time past the range of a date
gives no served clock instead of throwing.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): leave unknown entities unchecked by failed reads, volatile fields and unchecked calls

A failed item read that is no absence leaves an unknown entity unknown, so
a later usable read still checks its writes. A read showing an unknown
entity again checks no write on a volatile field. An update of an unknown
entity takes its recorded outcome and adds nothing to a built plan. A later
update that does not answer with its entity is observed only by a read
whose view shows every learned clock; without one it is unmodelled.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): fill a create's path fields from the collection serving selects

The gate built a later create's path fields from the recorded request path,
while serving fills them from the held collection it selects for that path,
so a read showing the other spelling's parent could confirm a create serving
cannot reproduce. Rules still learn from the recorded path.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): name unmodelled writes as modelled ones, count fields updates write, and keep later members' types

An unmodelled write below an item path names its entity through itemTarget,
and one at or above a collection resolves the held collection, as modelled
writes do, so an encoded spelling no longer hides the entity it changed.
An update's answer counts the fields its rules write, other than increments,
as available, so a field only the update writes no longer makes it
incomplete-entities. A member a later call first stores keeps that call's
JSON type in the body's kinds, which the body check falls back to.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): let a document replacement overwrite everything under its object

A replacement claims only the members it wrote or removed, but it now
overwrites every earlier effect under its object, so a merge that wrote
the object it replaces loses its confirmation.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): start the served clock from the records whose times a date can hold

A record timed past the range of a date no longer leaves the served clock
without a start, which the store read as zero and stamped writes in 1970;
the clock starts after the latest record a date can hold.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-22 14:15:35 +08:00

326 lines
10 KiB
TypeScript

import { z } from "zod";
import type { AnalysisError } from "../analysisErrorBase.js";
import { jsonValueSchema } from "../jsonValue.js";
import { ok, type Result } from "../result.js";
import type {
AnalysisViewParent,
AnalysisViewRequest,
UnsignedAnalysisView,
} from "./analysisView.js";
import { EnvironmentStateAccumulator } from "./EnvironmentStateAccumulator.js";
import {
groupCaptureRecords,
templateRulesSchema,
} from "./environmentManifestView.js";
export const environmentStateViewRequestSchema = z.strictObject({
kind: z.literal("environment-state"),
template_rules: templateRulesSchema.describe(
"Opt-in template rules as for environment-manifest, each listed at most once. Use the rules of the manifest the state accompanies so tool names match.",
),
});
const jsonObjectFieldsSchema = z.record(z.string(), jsonValueSchema);
const checkedSchema = z
.array(z.number().int().nonnegative())
.describe(
"Latest init record per request line that the view reproduced from the state.",
);
const entityViewFields = {
tool: z.string(),
type: z.string(),
pointer: z
.string()
.describe(
"JSON pointer to the entity object or array within the response: the root or one member, or for an item the element of a one-element array there (/0 or /<member>/0).",
),
fields: z
.array(z.string())
.describe("Ordered fields the view shows of each entity."),
required_fields: z.array(z.string()),
checked_record_ordinals: checkedSchema,
};
const viewSchema = z.discriminatedUnion("shape", [
z.strictObject({
...entityViewFields,
shape: z.literal("item"),
envelope_ordinal: z
.number()
.int()
.nonnegative()
.describe(
"Record whose response, with the entity object replaced, every item response shares.",
),
}),
z.strictObject({
...entityViewFields,
shape: z.literal("list"),
membership: z
.enum(["all", "recorded"])
.describe(
"all: every entity of the collection in state order; recorded: the entities each recorded response listed.",
),
counters: z
.array(z.string())
.describe("Envelope members that count the listed entities."),
}),
z.strictObject({
tool: z.string(),
shape: z.literal("document"),
checked_record_ordinals: checkedSchema,
}),
z.strictObject({
tool: z.string(),
shape: z.literal("recorded"),
reason: z.enum([
"query-body",
"no-init-recording",
"body-unusable",
"no-identity",
"path-varies",
"query-varies",
"fields-differ",
"envelope-varies",
"list-differs",
"rendered-oversize",
]),
}),
]);
const ordinalSchema = z.number().int().nonnegative();
const clockFormatSchema = z.discriminatedUnion("kind", [
z.strictObject({
kind: z.literal("iso"),
separator: z.enum(["T", " "]),
fraction: z.number().int().nonnegative(),
zone: z.string(),
}),
z.strictObject({ kind: z.literal("epoch"), unit: z.enum(["s", "ms"]) }),
]);
const ruled = { field: z.string() };
const fieldRuleSchema = z.discriminatedUnion("rule", [
z.strictObject({
...ruled,
rule: z.literal("identity"),
generator: z.enum(["sequence", "uuid", "token"]),
}),
z.strictObject({
...ruled,
rule: z.literal("sequence"),
scope: z.enum(["collection", "type"]),
}),
z.strictObject({ ...ruled, rule: z.literal("unique") }),
z.strictObject({
...ruled,
rule: z.literal("clock"),
format: clockFormatSchema,
}),
z.strictObject({ ...ruled, rule: z.literal("echo") }),
z.strictObject({ ...ruled, rule: z.literal("copy"), from: z.string() }),
z.strictObject({ ...ruled, rule: z.literal("path"), from: z.string() }),
z.strictObject({ ...ruled, rule: z.literal("slug"), from: z.string() }),
z.strictObject({ ...ruled, rule: z.literal("template"), from: z.string() }),
z.strictObject({ ...ruled, rule: z.literal("increment") }),
z.strictObject({
...ruled,
rule: z.literal("constant"),
value: jsonValueSchema,
}),
]);
const insertRuleSchema = z.union([
z.strictObject({ rule: z.enum(["first", "last"]) }),
z.strictObject({
rule: z.literal("sorted"),
field: z.string(),
direction: z.enum(["ascending", "descending"]),
}),
]);
const writePlanSchema = z.strictObject({
tool: z.string(),
effect: z.enum(["create", "update", "delete", "document", "none"]),
target: z
.string()
.nullable()
.describe(
"The entity type an entity effect writes, or the document view a document write changes.",
),
served: z.boolean(),
reason: z
.enum([
"no-state-target",
"no-call",
"body-unusable",
"no-observation",
"unexplained-change",
"contradicted",
"side-effect",
"unconfirmed",
"incomplete-entities",
])
.nullable(),
exemplar_ordinal: ordinalSchema
.nullable()
.describe("The first successful recorded call the plan was built from."),
observation: z
.strictObject({ ordinal: ordinalSchema, pointer: z.string() })
.nullable()
.describe(
"The record whose response showed the written entity or document, and a JSON pointer to it.",
),
response: z
.strictObject({
ordinal: ordinalSchema,
shape: z.enum(["entity", "identity", "empty", "recorded"]),
pointer: z.string().nullable(),
})
.nullable(),
body: z
.strictObject({
accepted: z.array(z.string()),
fixed: z.array(
z.strictObject({ member: z.string(), sent: jsonValueSchema }),
),
required: z.array(z.string()),
identity: z.string().nullable(),
kinds: z.array(
z.strictObject({
member: z.string(),
kind: z.enum(["array", "boolean", "number", "object", "string"]),
}),
),
})
.nullable()
.describe(
"Body members the application stored as sent (accepted), members accepted only with their recorded value (fixed), members every call sends (required), and the JSON type a later call stored an accepted member as when the observation does not show it (kinds).",
),
rules: z
.array(fieldRuleSchema)
.describe(
"How written fields get their values; unlisted fields of a created entity keep the observed value.",
),
collection: z
.string()
.nullable()
.describe(
"For a create whose recorded values are tied to its collection, the only collection it writes to.",
),
insert: insertRuleSchema.nullable(),
merge: z.enum(["merge", "replace"]).nullable(),
checked_record_ordinals: z
.array(ordinalSchema)
.describe("Later reads that reproduced from the state after this write."),
contradicted_by: ordinalSchema.nullable(),
});
export const environmentStateFields = {
origin: z
.string()
.describe("Primary origin, as the environment manifest selects it."),
name: z.string(),
timeline: z
.enum(["recorded-start-time", "record-order"])
.describe(
"Records are ordered by recorded start time, or by record order when any start time is missing.",
),
types: z.array(
z.strictObject({
collection_template: z.string(),
id_field: z.string(),
id_basis: z
.enum(["path-parameter", "field-name"])
.describe(
"path-parameter: item responses repeat the requested path value in this field; field-name: inferred from listed items as the first identity name, else the first field ending in Id or _id shown distinct across at least two items, that every listed item carries distinctly.",
),
init_before_ordinal: z
.number()
.int()
.nonnegative()
.nullable()
.describe(
"First recorded write whose path root relates to the type; state comes from records that finished before it.",
),
volatile_fields: z
.array(z.string())
.describe(
"Fields whose recorded values changed with no recorded write between; the state keeps the latest.",
),
}),
),
entities: z
.array(
z.strictObject({
type: z.string(),
collection: z.string(),
id: z.string(),
fields: jsonObjectFieldsSchema,
view_fields: z.array(
z.strictObject({ tool: z.string(), fields: jsonObjectFieldsSchema }),
),
record_ordinals: z.array(z.number().int().nonnegative()),
}),
)
.describe(
"Entities in collection order: type, collection, then first sighting.",
),
documents: z.array(
z.strictObject({
tool: z.string(),
path: z.string(),
value: jsonValueSchema,
record_ordinal: z.number().int().nonnegative(),
}),
),
views: z.array(viewSchema),
writes: z
.array(writePlanSchema)
.describe(
"One plan per tool whose method may change state: its modelled effect, and whether the capture's later reads confirmed it (served).",
),
write_clock: z
.string()
.nullable()
.describe(
"The instant a served clock starts from: the end of the capture's last timed record.",
),
unknowns: z.array(
z.strictObject({
kind: z.enum([
"authorization-model",
"response-semantics",
"state-transition",
]),
statement: z.string(),
}),
),
};
/** Extract the init state a capture's reads observed and each read tool's render plan; types and identity are inferences. */
export const projectEnvironmentStateView = (
parent: AnalysisViewParent,
view: Extract<AnalysisViewRequest, { kind: "environment-state" }>,
accumulator = new EnvironmentStateAccumulator(undefined, view.template_rules),
): Result<UnsignedAnalysisView, AnalysisError> => {
const grouped = groupCaptureRecords(parent, view.kind, view.template_rules, {
accumulator,
noun: "state",
limitations: [
"Entity types, identity fields, list membership and counters are inferences from recorded response bodies; the state does not establish server-side schemas, derived values or values copied outside entity objects.",
],
});
if (!grouped.ok) return grouped;
const state = accumulator.finishState();
if (!state.ok) return state;
return ok({
kind: "environment-state",
view,
parent_evidence_id: parent.evidenceId,
parent_operation: grouped.value.operation,
parent_digest: parent.evidenceId.slice(3),
artifact: {
path: grouped.value.artifact.path,
sha256: grouped.value.artifact.sha256,
},
...state.value,
});
};