diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index 64b0bdad23..1eb5366684 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -2,7 +2,7 @@ name: reverse-engineer-anything description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. metadata: - version: "53" + version: "54" --- # REA diff --git a/.agents/skills/reverse-engineer-anything/references/environments.md b/.agents/skills/reverse-engineer-anything/references/environments.md index 9df5486c88..7d2ededf46 100644 --- a/.agents/skills/reverse-engineer-anything/references/environments.md +++ b/.agents/skills/reverse-engineer-anything/references/environments.md @@ -155,8 +155,9 @@ locators to exactly one element. with varied free text. After each write, and before the next one, read the object back by id and then its list: create 1, read 1, list, create 2, read 2, list; never several writes before their reads. -4. Leave more than a second between a create and its first edit - (`wait_for_timeout` with `duration_ms: 1100`). +4. Leave more than a second between two creates and between a create and its + first edit (`wait_for_timeout` with `duration_ms: 1100`), so lists sorted + by creation time show them in order. 5. Update both objects and read each back, changing the fields a person would change (a name, an email address, a status, a due date) rather than only a description: a served update accepts only the members a recorded diff --git a/docs/specs/agent-environments/write-varied-calls/plan.md b/docs/specs/agent-environments/write-varied-calls/plan.md new file mode 100644 index 0000000000..93e9255730 --- /dev/null +++ b/docs/specs/agent-environments/write-varied-calls/plan.md @@ -0,0 +1,9 @@ +# Implementation plan + +1. **Update members.** `refinedBody` keeps only the required members a later update also sent. +2. **Numbered unless sent.** `sent` on sequence rules and their schema; the create merge adopts it for an echoed member + and a sequence from a call that left the member out; serving and replay take the sent value when there is one. +3. **Cleared by a PUT.** `explainUpdate` records a `PUT`'s left-out member cleared to an empty value as a default; + update refinement keeps defaults from later calls; `updatedFields` applies them to members a call leaves out. +4. **Update clocks.** `restampedFields` respells clocks from the first later read as it does nested clocks. +5. **Skill, tests and docs.** diff --git a/docs/specs/agent-environments/write-varied-calls/spec.md b/docs/specs/agent-environments/write-varied-calls/spec.md new file mode 100644 index 0000000000..4458a29f52 --- /dev/null +++ b/docs/specs/agent-environments/write-varied-calls/spec.md @@ -0,0 +1,56 @@ +# Serve writes whose calls send different members + +Issue: https://github.com/morluto/rea/issues/4933 + +## Outcome + +A fresh recording round of seven applications, main against #7332, showed the recording skill now varies what the two +creates and two updates of a type send, as it asks. Where the second call left out a member the first sent, REA +withheld the write: + +- **Updates.** An update plan required every member its first call sent and never narrowed that, so EspoCRM's second + contact update, which changed only an email address, was refused in both recordings of the branch. +- **Members numbered when left out.** Linkding stores a bundle's `order` as sent, or one past the others when a call + leaves it out. One create showed the first and another the second, which the create merge took as a disagreement. + +The same recordings and the 130 earlier captures showed two more update gaps: + +- **Whole-object PUTs.** Paperless clears a tag's, correspondent's or document type's `owner` when a `PUT` leaves it + out, and Homebox an item's `assetId`, so those updates were `unexplained-change`. +- **Update clocks read at another precision.** Vikunja answers a task update with `done_at` to the nanosecond while its + reads spell it to the second, so the read after the update contradicted it. + +The round also found two creates of EspoCRM contacts in one second, which a list sorted newest first shows in an order +the application breaks ties in. The skill now spaces creates. + +## Behaviour + +- **Update members.** A later update narrows the plan's required members to those every call sent. +- **Numbered unless sent.** A member one create sent and stored as sent, and another left out and the application + numbered one past the others, is a `sequence` with `sent`, in either order: serving stores the value a call sends, + else the next number, and a later call that sends it must have stored what it sent. The next number follows the + largest value any call took, a deleted entity's included, in replay as in serving. +- **Cleared by a PUT.** A field a `PUT` left out that the application cleared to an empty value (`""`, `null`, `false`, + `[]` or `{}`) is that member's default (`body.defaults`) when every call that left it out stored that value, as its + answer shows it or, where the answer leaves the member out, the read after it: serving stores it for an update that + leaves the member out. Calls that left it out and stored it otherwise withhold the plan. +- **Update clocks.** The state holds an update's clock as the first read of the entity after it spells the same instant + at a coarser precision in the same zone and separator, as it already does for nested clocks: replay and serving store + every update's clocks in the spelling of the read the plan keeps as `observation.read`, so a later update whose read + spells them otherwise contradicts the plan, while answers keep the update's own. A create's clocks stay as answered, + as serving makes them. +- **Skill.** Recordings leave more than a second between two creates, as between a create and its first edit. + +Plans and views that need none of this are unchanged. + +## Not covered + +- Vikunja numbers tasks per project. Creates into projects the recording did not list cannot be numbered, so its task + creates stay withheld. +- Linkding renumbers bundles' `order` after a delete, which no rule models. +- Homebox item creates expand sent tag ids into tag objects and pin `parentId`; they stay withheld. + +## Acceptance + +- No write tool of the recorded captures goes from served to withheld. +- The writes the diagnosis named are served where their recordings show nothing else against them. diff --git a/docs/specs/agent-environments/write-varied-calls/tasks.md b/docs/specs/agent-environments/write-varied-calls/tasks.md new file mode 100644 index 0000000000..446d670a0d --- /dev/null +++ b/docs/specs/agent-environments/write-varied-calls/tasks.md @@ -0,0 +1,11 @@ +# Tasks + +- [x] Narrow an update plan's required members to those every call sent. +- [x] Serve a member the value a call sends, else the next number, when calls showed both. +- [x] Learn and serve the value a `PUT` stores for a member it leaves out. +- [x] Hold an update's clock as the read after it spells it. +- [x] Ask recordings to space creates. +- [x] Add view, serving and unit tests. +- [x] Document the rules. +- [x] Measure the recorded captures. +- [x] Run static, formatting, documentation and test gates. diff --git a/docs/specs/agent-environments/write-varied-calls/verification.md b/docs/specs/agent-environments/write-varied-calls/verification.md new file mode 100644 index 0000000000..506b581e1f --- /dev/null +++ b/docs/specs/agent-environments/write-varied-calls/verification.md @@ -0,0 +1,95 @@ +# Verification + +Executed on macOS with Node v24.22.1 on this branch, on #7332 at `7d01a5d5` (main `5aa23c3d`), and on Ubuntu 24.04 VMs +with snap Chromium for the recording round. + +## Diagnosis + +Claude Code (`claude-sonnet-5-5`) built an environment of each of seven applications from the same prompt, once with +REA main `5aa23c3d` (skill 50) and once with #7332 at `7d01a5d5` (skill 53). Each bundle's write tasks ran with +`fireworks_ai/gpt-oss-120b` and `fireworks_ai/glm-5p3-flash`, which scored alike: + +| Application | Write tasks | Main | #7332 | +| ----------- | ----------- | ------- | ------- | +| Miniflux | 3 | 3/3 | 3/3 | +| Grafana | 2 | 2/2 | 2/2 | +| Vikunja | 3 + 2 | 0/3 0/2 | 1/3 2/2 | +| Paperless | 3 | 1/3 | 2/3 | +| Linkding | 2 | 0/2 | 2/2 | +| Homebox | 2 | 2/2 | 0/2 | +| EspoCRM | 3 | 1/3 | 0/3 | +| Total | 20 | 9/20 | 12/20 | + +Served writes were 14 and 25. Recorded again, Homebox and EspoCRM gave main 2/2 and 1/3 and #7332 2/2 and 3/3: the +first losses were the agents' runs, not REA. Built with both versions, none of the 18 recordings served fewer writes +with #7332 (44 and 53 in all). + +Replaying #7332's recordings with failing checks and refusals logged gave the causes this branch takes: EspoCRM's +second contact update left out `addressCity` and was refused for lacking a required member (both recordings); Linkding's +second bundle create left `order` to the application, which the merge took as disagreeing with the first create's +echo; Vikunja's task update stamped `done_at` to the nanosecond where the read after it showed seconds. Over the +earlier captures, 14 withheld updates were `PUT`s whose application cleared a member they left out (Paperless `owner`, +Homebox `assetId`). + +## Gates + +- `npm run -s check:fast`, `format:check`, `scan:todos`, `docs:check`, `verify:test-discovery`, `mcp-catalog:check` and + `jscpd`: passed. `npx knip` reports `staticFileUrlPath`, `ObjcClassImportBindingCoverage` and + `AndroidManifestComponentInventory`, as main `5aa23c3d` does. +- `npx vitest run --project domain --project services`: 11,694 passed, 8 failed (`appleApplication.test.ts`), which fail + the same way on main `5aa23c3d`. +- `npm run -s test:focused --` the environment build, export and serve CLI boundaries, the analysis-view CLI and MCP + boundaries and the environment serve MCP boundary: 80 passed. `outputSchemaProfile`, `toolSchemaValidity`, + `mcpSchemaValidation`, `outputSchemaSemanticConstraints` and `analysisViewMcp`: 54 passed; the compact catalog stays + under its frame. +- The seven new view and serving tests fail on #7332 at `7d01a5d5` and pass here. + +## Write plans over recorded captures + +The 130 captures #7332 measured and the round's 18 recordings, built by #7332 at `7d01a5d5` and by this branch: 694 +plans, 386 served and 398. No write goes from served to withheld and every view is unchanged. Newly served: + +- Paperless correspondent, document type and tag updates in four recordings, whose `PUT` clears `owner`. +- EspoCRM's contact update and Linkding's bundle create in #7332's recordings of the round. +- Grafana's folder update and delete in a fresh recording whose second update sent fewer members. +- Homebox's item update in the published capture, whose `PUT` clears `assetId`. + +Vikunja's task update reads back confirmed, but stays withheld: the task creates before it are not modelled (per-project +numbers), so their entities carry the creates' answer spellings into later reads. + +## Review round + +- Greptile's first review: a `sequence` with `sent` now accepts a value a later create sent, whichever create came + first; a default must agree with every update that left its member out, else the plan is withheld; and serving + stores an update's clocks as the update plan's read-back spells them (`observation.read`). Their four tests fail on + `6300ec80`. #7332's made-object clock check, merged in, has its test there. +- Gates as above: `domain` + `services` 11,699 passed, 8 failed (main's `appleApplication.test.ts`); focused boundaries + 80 passed; the five MCP schema files 54 passed. The census stays at 694 plans and 398 served, with no plan or view + changing. + +Greptile's review after main `21b19270` was merged in (`467ac78f`) raised three more, fixed: + +- A member a call left out and the application stored a fixed value for (`body.defaults`) is no longer turned into a + `sequence` with `sent` by a later call that left it out and got the next number. +- Serving stores an update's clock at a coarser read's precision from the answer's own digits, so an answer to the + nanosecond read back to the microsecond keeps its microseconds. +- A `PUT` that names a `reference` field's entity by its member supplies that field: it is not checked against, or + given, the default a `PUT` that left the field out showed, whichever of the two calls came first. + +Their tests fail on `467ac78f`, and the reference case with the clearing call first on `7030ca4d`. Gates pass but +`knip`, whose one finding main `21b19270` reports too; `domain` + `services` 12,046 passed; focused boundaries 82 passed; the five MCP schema files 55 passed. Over the 148 captures, main +serves 380 writes and this branch 392, with no plan or view changing from `467ac78f`. + +Greptile's review of `b56f8924` raised two more on numbered members, fixed: a member a call sent empty (`""`) is no +longer filled in for a call that leaves it out when a `sequence` with `sent` numbers it; and a later create the +application numbered at or below a value an earlier create sent (9 sent and deleted, then 2) no longer turns the echo +into a numbered `sequence`, since serving counts past every value a call sent. Their tests fail on `b56f8924`; gates +pass but `knip` (main's finding), `domain` + `services` 12,048 passed, and the census is unchanged. Greptile's next +review found that check counting values sent in other collections, which withheld a create numbered within its own +collection; the check now counts them in the sequence's scope, and its test fails on `d4922b3d`. + +## Not verified + +- No recording round ran with this branch, so its write tasks are not measured. +- The skill's spacing of creates. +- Vikunja per-project task numbers, Linkding's renumbering after a delete and Homebox item creates, which stay withheld. diff --git a/docs/web-network-captures.md b/docs/web-network-captures.md index 2b75fedcb7..3f6573262c 100644 --- a/docs/web-network-captures.md +++ b/docs/web-network-captures.md @@ -798,7 +798,14 @@ clocks and own members, and a count only the later call shows starts at the first object's value; a later create's made object must hold clocks its own request stamped, in the rule's format. Members either call sent are accepted, only those every call sent are required, and an echoed member a call left out takes the value that call stored -(`body.defaults`). A member a create sent empty (`""`, `null`, `false`, `[]` or +(`body.defaults`). A member one call sent and stored as sent, and another left +out and the application numbered one past the others, takes the value a call +sends, else that number (a `sequence` with `sent`), whichever call came first, +unless a call that left it out stored a fixed value (its `body.defaults`) or +the number a later call was given is not above every value an earlier create +sent in the sequence's scope (the type, or the later call's collection); an empty value a call sent is not applied to a call that leaves it out; +a later call that sends it must have stored what it sent, and the next number +follows the largest value any call took, a deleted entity's included. A member a create sent empty (`""`, `null`, `false`, `[]` or `{}`), as a form sends a field left as it was, is not required: a call that leaves it out is applied as sending that value (`body.blank`), unless a call that left it out showed a default, which it then keeps. A call may send it as any @@ -861,13 +868,25 @@ is unchanged) must be a clock, a nested clock, an increment, a slug, a fresh `un value, a `stamp` of the acting user, a value `derived` from the one member it stores nowhere, a `reference` to the entity a sent member names (kept by a call that names the entity the field already repeats), a `composed` string whose parts read members it stored as sent or fields it left as they were and spell the value before the call too, or for -an item action a `set` of a scalar to the value it showed, or the plan is +an item action a `set` of a scalar to the value it showed, or, for a `PUT`, a +member it left out that the application cleared to an empty value, which calls +that leave it out then store (`body.defaults`) as long as every call that left +it out stored that value (a call naming a `reference` field's entity by its +member does not leave the field out), or the plan is `unexplained-change`; an increment, slug, stamp, derived value or set must hold for every call of the tool, and an action is served only once a call found what it sets already set. A write is served only with the query its first call sent, which serving applies to a call that -leaves a parameter out and the catalog advertises as required. Every member the first call sent is required, except by `PATCH` and -`POST` updates and members a create sent empty. +leaves a parameter out and the catalog advertises as required. Every member every call sent is required, except by `PATCH` and +`POST` updates and members a create sent empty. The state holds an update's +clock as the first read of the entity after it spells the same instant at a +coarser precision in the same zone and separator: replay and serving store every +update's clocks in the spelling of the read the plan keeps as +`observation.read`, so a later update whose read spells them otherwise +contradicts the plan, while answers keep the update's own; a create's clocks +stay as answered. A `PUT` default must agree with what every call that left the +member out stored, as its answer shows it or, where the answer leaves it out, +the read after the call. A read on a line `query` answers is checked against the query evaluated over the replayed state. It covers the collection only when every parameter sorts or diff --git a/src/application/environment/environmentStateBulkAnswer.ts b/src/application/environment/environmentStateBulkAnswer.ts index 434cc9fdd9..b0d0894a6b 100644 --- a/src/application/environment/environmentStateBulkAnswer.ts +++ b/src/application/environment/environmentStateBulkAnswer.ts @@ -37,6 +37,7 @@ import { observedEntity, pathValues, sortedOrder, + storedUpdate, unloadable, heldEntity, updateTarget, @@ -210,12 +211,16 @@ export const answerBulk = ( ); if (!answer.ok) return answer; if (update) { + const stored = written.value.map((entity) => ({ + ...entity, + fields: storedUpdate(serving, plan, entity.fields), + })); const updated = serving.store.updateAll( - written.value, + stored, sortedOrder( serving, plan, - written.value.map((after, index) => ({ + stored.map((after, index) => ({ before: targets.value[index]!.entity, after, })), diff --git a/src/application/environment/environmentStateVariedCallsAnswer.test.ts b/src/application/environment/environmentStateVariedCallsAnswer.test.ts new file mode 100644 index 0000000000..5137265a18 --- /dev/null +++ b/src/application/environment/environmentStateVariedCallsAnswer.test.ts @@ -0,0 +1,123 @@ +import { describe, expect, it } from "vitest"; + +import { + DONE_CLOCK_ROWS, + FEWER_MEMBERS_ROWS, + NUMBERED_THEN_BLANK_ROWS, + NUMBERED_UNLESS_SENT_ROWS, + OWNER_CLEARED_THEN_NAMED_ROWS, + OWNER_NAMED_THEN_CLEARED_ROWS, + PUT_CLEARS_ROWS, +} from "../../../tests/fixtures/environmentStateVariedCalls.js"; +import { servedFrom } from "../../../tests/fixtures/environmentStateServing.js"; + +describe("served writes whose calls send different members", () => { + it("numbers a created bundle that leaves its order out, and keeps one a call sends", () => { + const bundles = servedFrom(NUMBERED_UNLESS_SENT_ROWS); + expect( + bundles.answer("post_bundles", { body: { name: "Echo" } }).body, + ).toMatchObject({ name: "Echo", order: 2 }); + expect( + bundles.answer("post_bundles", { body: { name: "Foxtrot", order: 9 } }) + .body, + ).toMatchObject({ + name: "Foxtrot", + order: 9, + }); + }); + + it("numbers past an order a call sent, even once that bundle is gone", () => { + const bundles = servedFrom(NUMBERED_UNLESS_SENT_ROWS); + bundles.answer("post_bundles", { body: { name: "Echo" } }); + bundles.answer("post_bundles", { body: { name: "Foxtrot", order: 9 } }); + bundles.store.removeAll( + bundles.store + .ofType("/api/bundles") + .filter((entity) => entity.fields["order"] === 9), + ); + expect( + bundles.answer("post_bundles", { body: { name: "Golf" } }).body, + ).toMatchObject({ name: "Golf", order: 10 }); + }); + + it("clears a member a PUT leaves out, as the application did, and keeps one it sends", () => { + const tags = servedFrom(PUT_CLEARS_ROWS); + expect( + tags.answer("put_tags_by_id", { id: "2", body: { name: "Receipts" } }) + .body, + ).toMatchObject({ + name: "Receipts", + owner: null, + }); + expect(tags.answer("get_tags_by_id", { id: "2" }).body).toMatchObject({ + owner: null, + }); + expect( + tags.answer("put_tags_by_id", { + id: "1", + body: { name: "Mail", owner: 3 }, + }).body, + ).toMatchObject({ + name: "Mail", + owner: 3, + }); + }); + + it.each([ + ["named first", OWNER_NAMED_THEN_CLEARED_ROWS], + ["cleared first", OWNER_CLEARED_THEN_NAMED_ROWS], + ])( + "clears a referenced owner a PUT leaves out, and stores the one a PUT names by its id: %s", + (_order, rows) => { + const tags = servedFrom(rows); + expect( + tags.answer("put_tags_by_id", { id: "1", body: { name: "Receipts" } }) + .body, + ).toMatchObject({ name: "Receipts", owner: null }); + expect( + tags.answer("put_tags_by_id", { + id: "2", + body: { name: "Mail", ownerId: 4 }, + }).body, + ).toMatchObject({ name: "Mail", owner: { id: 4, name: "Bo" } }); + }, + ); + + it("answers an update with its clock spelled as the recorded answer and stores it as reads spell it", () => { + const tasks = servedFrom(DONE_CLOCK_ROWS); + const answered = tasks.answer("patch_tasks_by_id", { + id: "1", + body: { done: true }, + }).body as { done_at: string }; + expect(answered.done_at).toMatch(/\.\d{9}Z$/u); + const read = tasks.answer("get_tasks_by_id", { id: "1" }).body as { + done_at: string; + }; + expect(read.done_at).toMatch(/:\d{2}Z$/u); + }); + + it("serves an update that sends only the member every recorded update sent", () => { + const contacts = servedFrom(FEWER_MEMBERS_ROWS); + expect( + contacts.answer("put_contacts_by_id", { + id: "1", + body: { email: "ada.f@example.test" }, + }).body, + ).toMatchObject({ name: "Ada Finch", email: "ada.f@example.test" }); + expect( + contacts.refusal("put_contacts_by_id", { + id: "2", + body: { city: "Kumasi" }, + }), + ).toMatchObject({ + message: expect.stringContaining("email"), + }); + }); + + it("numbers a created bundle that leaves its order out, though a recorded create sent it empty", () => { + const bundles = servedFrom(NUMBERED_THEN_BLANK_ROWS); + expect( + bundles.answer("post_bundles", { body: { name: "Echo" } }).body, + ).toMatchObject({ name: "Echo", order: "2" }); + }); +}); diff --git a/src/application/environment/environmentStateWriteAnswer.ts b/src/application/environment/environmentStateWriteAnswer.ts index fb997d84f0..895512e342 100644 --- a/src/application/environment/environmentStateWriteAnswer.ts +++ b/src/application/environment/environmentStateWriteAnswer.ts @@ -23,6 +23,7 @@ import { nestedClockValue, nextMadeCount, unenveloped, + readSpelledClocks, } from "../../domain/analysisView/environmentStateWriteStamps.js"; import { anyLinkedSource, @@ -41,6 +42,7 @@ import { generatedIntegerDigits, seededInteger, seededLike, + sequenceNumber, spelledLike, startedSequence, } from "../../domain/analysisView/environmentStateWriteValues.js"; @@ -230,6 +232,12 @@ export const generator = ( ): { readonly forTarget: (target: GeneratedTarget) => GenerateValue; readonly taken: Map; + /** Count a number the call sent for a sequence, as replay does, so later generated ones follow it. */ + readonly sentSequence: ( + target: string, + rule: Extract, + value: JsonValue | undefined, + ) => void; } => { const { store } = serving; const write = store.pendingWrite(); @@ -239,6 +247,11 @@ export const generator = ( taken.set(key, value); return value; }; + const sequenceKey = ( + rule: Extract, + target: string, + ) => + `${request.tool.name} ${rule.field} ${rule.scope === "collection" ? target : ""}`; const ofType = store.ofType(request.plan.target ?? ""); // A sequence never reuses a value the state started with, even after its entity was deleted. const started = store.startedWith(request.plan.target ?? ""); @@ -291,7 +304,7 @@ export const generator = ( spelledLike( exemplar, sequence( - `${request.tool.name} ${rule.field} ${rule.scope === "collection" ? target.name : ""}`, + sequenceKey(rule, target.name), startedSequence( nextHeld( rule.scope === "collection" @@ -361,7 +374,17 @@ export const generator = ( ); } }; - return { forTarget: (target) => (rule) => generate(target, rule), taken }; + return { + forTarget: (target) => (rule) => generate(target, rule), + taken, + sentSequence: (target, rule, value) => { + const number = sequenceNumber(value); + if (number === undefined) return; + const key = sequenceKey(rule, target); + // commitWrite keeps the larger of this and the values taken before. + taken.set(key, Math.max(taken.get(key) ?? number, number)); + }, + }; }; /** A write's answer on its recorded response's status and content type: a rendered body, none, or the recorded one. */ @@ -556,6 +579,17 @@ const answerCreate = ( idField, ); if (!fields.ok) return err(refused(tool, fields.error)); + for (const rule of plan.rules) + if ( + rule.rule === "sequence" && + rule.sent === true && + Object.hasOwn(call.body ?? {}, rule.field) + ) + values.sentSequence( + collection, + rule, + ownMember(fields.value, rule.field), + ); const field = unloadableMember(fields.value); if (field !== undefined) return err(unloadable(tool, `Field ${field}`)); const id = scalarText(ownMember(fields.value, idField)); @@ -665,7 +699,10 @@ const answerUpdate = ( }, ); if (!answer.ok) return answer; - const after = updatedEntity(entity, written.value); + const after = updatedEntity( + entity, + storedUpdate(serving, plan, written.value), + ); const updated = serving.store.updateAll( [after], sortedOrder(serving, plan, [{ before: entity, after }]), @@ -675,6 +712,20 @@ const answerUpdate = ( return answer; }; +/** An update's fields as the state stores them: its clocks spelled as the plan's read-back spelled them. */ +export const storedUpdate = ( + serving: StateServing, + plan: StateWritePlan, + written: JsonObject, +): JsonObject => + readSpelledClocks( + plan.rules, + written, + plan.observation?.read === undefined + ? undefined + : recordedObject(serving, plan.observation.read), + ); + /** The collection's members in the order a sorted collection holds them after these writes; empty when the plan keeps them in place. */ export const sortedOrder = ( serving: StateServing, diff --git a/src/domain/analysisView/EnvironmentStateWriteSimulation.ts b/src/domain/analysisView/EnvironmentStateWriteSimulation.ts index 597e4d0ca3..488deec68b 100644 --- a/src/domain/analysisView/EnvironmentStateWriteSimulation.ts +++ b/src/domain/analysisView/EnvironmentStateWriteSimulation.ts @@ -297,6 +297,7 @@ export class EnvironmentStateWriteSimulation { sequences: new Map(), starts: new Map(), createCollections: [], + sentLargest: new Map(), steady: new Set(), unstamped: new Set(), kept: new Set(), diff --git a/src/domain/analysisView/environmentStateVariedCalls.test.ts b/src/domain/analysisView/environmentStateVariedCalls.test.ts new file mode 100644 index 0000000000..f157859c3b --- /dev/null +++ b/src/domain/analysisView/environmentStateVariedCalls.test.ts @@ -0,0 +1,248 @@ +import { describe, expect, it } from "vitest"; + +import { + timedCaptureEvidence, + type TimedCaptureRow, +} from "../../../tests/fixtures/analysisView.js"; +import { + CLEARED_THEN_OMITTED_KEPT_ROWS, + DEFAULT_THEN_NUMBERED_ROWS, + DONE_CLOCK_ROWS, + DONE_MICRO_ROWS, + NUMBERED_THEN_BLANK_ROWS, + OWNER_CLEARED_THEN_NAMED_ROWS, + SENT_DELETED_THEN_NUMBERED_ROWS, + SENT_IN_ANOTHER_PROJECT_ROWS, + OWNER_NAMED_THEN_CLEARED_ROWS, + DONE_MIXED_ROWS, + DONE_ZONE_ROWS, + OMITTED_KEPT_THEN_CLEARED_ROWS, + FEWER_MEMBERS_ROWS, + NUMBERED_UNLESS_SENT_ROWS, + NUMBERED_THEN_SENT_ROWS, + SENT_THEN_DELETED_ROWS, + KEPT_THEN_CLEARED_ROWS, + CLEARED_THEN_KEPT_ROWS, + PUT_CLEARS_ROWS, +} from "../../../tests/fixtures/environmentStateVariedCalls.js"; +import { environmentStateViewOf } from "../../../tests/fixtures/environmentStateWrites.js"; + +const planOf = (rows: readonly TimedCaptureRow[], tool: string) => { + const plan = environmentStateViewOf(timedCaptureEvidence(rows)).writes.find( + (candidate) => candidate.tool === tool, + ); + if (plan === undefined) throw new Error(`No plan for ${tool}`); + return plan; +}; + +describe("writes whose calls send different members", () => { + it("serves an update a later call sends fewer members to, requiring only those every call sent", () => { + const plan = planOf(FEWER_MEMBERS_ROWS, "put_contacts_by_id"); + expect(plan).toMatchObject({ served: true }); + expect(plan.body?.required).toEqual(["email"]); + expect(plan.body?.accepted).toEqual( + expect.arrayContaining(["name", "city", "email"]), + ); + }); + + it("serves a create whose member one call sent and another left for the application to number", () => { + const plan = planOf(NUMBERED_UNLESS_SENT_ROWS, "post_bundles"); + expect(plan).toMatchObject({ served: true }); + expect(plan.rules).toContainEqual({ + field: "order", + rule: "sequence", + scope: "collection", + sent: true, + }); + expect(plan.body?.required).toEqual(["name"]); + }); + + it("serves a create whose first call left a member to be numbered and whose second sent one", () => { + const plan = planOf(NUMBERED_THEN_SENT_ROWS, "post_bundles"); + expect(plan).toMatchObject({ served: true }); + expect(plan.rules).toContainEqual({ + field: "order", + rule: "sequence", + scope: "collection", + sent: true, + }); + }); + + it.each([ + ["kept, then cleared", KEPT_THEN_CLEARED_ROWS], + ["cleared, then kept", CLEARED_THEN_KEPT_ROWS], + ])( + "withholds a PUT whose calls left a member out and stored it otherwise (%s)", + (_order, rows) => { + expect(planOf(rows, "put_tags_by_id")).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }, + ); + + it("confirms an update whose read shows the clock it stamped to the second", () => { + expect(planOf(DONE_CLOCK_ROWS, "patch_tasks_by_id")).toMatchObject({ + served: true, + }); + }); + + it("serves no update whose clock the read after it spells in another zone, which serving cannot store", () => { + // The read spells the answer's instant at the second, but as +00:00: serving would keep the answer's spelling. + expect(planOf(DONE_ZONE_ROWS, "patch_tasks_by_id")).toMatchObject({ + served: false, + }); + }); + + it.each([ + [ + "answered without the owner its read shows kept, then cleared", + OMITTED_KEPT_THEN_CLEARED_ROWS, + ], + [ + "cleared, then answered without the owner its read shows kept", + CLEARED_THEN_OMITTED_KEPT_ROWS, + ], + ])( + "withholds a PUT default an earlier or later read contradicts: %s", + (_order, rows) => { + expect(planOf(rows, "put_tags_by_id")).toMatchObject({ served: false }); + }, + ); + + it("serves a PUT that clears a member it left out, as a default", () => { + const plan = planOf(PUT_CLEARS_ROWS, "put_tags_by_id"); + expect(plan).toMatchObject({ served: true }); + expect(plan.body?.defaults).toEqual([{ member: "owner", value: null }]); + }); + + it("recognizes a sent array when learning a later omitted-member default", () => { + const a = { id: 1, name: "A", tags: ["x"] }; + const b = { id: 2, name: "B", tags: ["y"] }; + const changedA = { ...a, name: "AA" }; + const changedB = { ...b, name: "BB", tags: [] }; + const rows: readonly TimedCaptureRow[] = [ + ["GET", "/api/tags", [a, b]], + ["GET", "/api/tags/1", a], + ["GET", "/api/tags/2", b], + ["PUT", "/api/tags/1", changedA, 200, { name: "AA", tags: ["x"] }], + ["GET", "/api/tags/1", changedA], + ["PUT", "/api/tags/2", changedB, 200, { name: "BB" }], + ["GET", "/api/tags/2", changedB], + ["GET", "/api/tags", [changedA, changedB]], + ]; + expect(planOf(rows, "put_tags_by_id")).toMatchObject({ served: true }); + }); + + it("withholds a sent sequence value contradicted by its recorded answer", () => { + const rows: readonly TimedCaptureRow[] = [ + ...NUMBERED_UNLESS_SENT_ROWS.map((row) => + row[0] === "GET" && row[1] === "/api/bundles" && Array.isArray(row[2]) + ? ([ + row[0], + row[1], + (row[2] as { id: number; name: string }[]).map( + ({ id, name }) => ({ id, name }), + ), + ] as TimedCaptureRow) + : row, + ), + [ + "POST", + "/api/bundles", + { id: 5, name: "Echo", order: 10 }, + 201, + { name: "Echo", order: 9 }, + ], + [ + "GET", + "/api/bundles", + [ + { id: 1, name: "Alpha" }, + { id: 2, name: "Bravo" }, + { id: 3, name: "Charlie" }, + { id: 4, name: "Delta" }, + { id: 5, name: "Echo" }, + ], + ], + ]; + expect(planOf(rows, "post_bundles")).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }); +}); + +describe("writes replay holds as serving does", () => { + it("serves no create numbered below a sent value it deleted, as serving counts past it", () => { + // Serving keeps the largest order sent, 9, and would number the last create 10. + expect(planOf(SENT_THEN_DELETED_ROWS, "post_bundles")).toMatchObject({ + served: false, + }); + }); + + it("serves no update whose read spells its clock otherwise than the read the plan keeps", () => { + // Serving stores every update's clock as the first read spells it, to the millisecond. + expect(planOf(DONE_MIXED_ROWS, "patch_tasks_by_id")).toMatchObject({ + served: false, + }); + }); +}); + +describe("writes whose calls send different members, alike in replay and serving", () => { + it("serves no create numbered when left out after a call that left it out stored a fixed value", () => { + const plan = planOf(DEFAULT_THEN_NUMBERED_ROWS, "post_bundles"); + expect(plan.rules).not.toContainEqual( + expect.objectContaining({ field: "order", rule: "sequence" }), + ); + expect(plan).toMatchObject({ served: false }); + }); + + it("confirms an update whose read shows the clock it stamped to the microsecond", () => { + expect(planOf(DONE_MICRO_ROWS, "patch_tasks_by_id")).toMatchObject({ + served: true, + }); + }); + + it.each([ + ["named first", OWNER_NAMED_THEN_CLEARED_ROWS], + ["cleared first", OWNER_CLEARED_THEN_NAMED_ROWS], + ])( + "serves a PUT that names a reference by its member and one that leaves it out and clears it: %s", + (_order, rows) => { + const plan = planOf(rows, "put_tags_by_id"); + expect(plan).toMatchObject({ served: true }); + expect(plan.rules).toContainEqual( + expect.objectContaining({ + field: "owner", + rule: "reference", + member: "ownerId", + }), + ); + }, + ); + + it("serves no create numbered below a value an earlier create sent, which serving counts past", () => { + const plan = planOf(SENT_DELETED_THEN_NUMBERED_ROWS, "post_bundles"); + expect(plan.rules).not.toContainEqual( + expect.objectContaining({ field: "order", rule: "sequence" }), + ); + expect(plan).toMatchObject({ served: false }); + }); + + it("serves a create numbered in its own project past what it holds, though another project's create sent more", () => { + const plan = planOf( + SENT_IN_ANOTHER_PROJECT_ROWS, + "post_projects_bundles_by_id", + ); + expect(plan).toMatchObject({ served: true }); + expect(plan.rules).toContainEqual( + expect.objectContaining({ + field: "order", + rule: "sequence", + scope: "collection", + sent: true, + }), + ); + }); +}); diff --git a/src/domain/analysisView/environmentStateView.ts b/src/domain/analysisView/environmentStateView.ts index 1802d850f1..ce9f8dc6cc 100644 --- a/src/domain/analysisView/environmentStateView.ts +++ b/src/domain/analysisView/environmentStateView.ts @@ -247,6 +247,7 @@ const fieldRuleSchema = z.discriminatedUnion("rule", [ rule: z.literal("sequence"), scope: z.enum(["collection", "type"]), start: z.number().int().exactOptional(), + sent: z.literal(true).exactOptional(), }), z.strictObject({ ...ruled, rule: z.literal("unique") }), z.strictObject({ @@ -396,7 +397,7 @@ const writePlanSchema = z.strictObject({ }) .exactOptional() .describe( - "A create's later read of the entity: an item read lends fields the response lacks and its clock spellings, a list read only adds fields the response lacks.", + "A write's later read of the entity: an item read lends a create fields the response lacks, and its clock spellings; a list read only adds fields the response lacks.", ), }) .nullable() diff --git a/src/domain/analysisView/environmentStateWriteApply.ts b/src/domain/analysisView/environmentStateWriteApply.ts index 05a93adefe..4af66a7de3 100644 --- a/src/domain/analysisView/environmentStateWriteApply.ts +++ b/src/domain/analysisView/environmentStateWriteApply.ts @@ -229,8 +229,11 @@ const ruledValue = ( switch (rule.rule) { case "reference": return referencedValue(rule, body, referenced); - case "identity": case "sequence": + return rule.sent === true && Object.hasOwn(body, rule.field) + ? ok(body[rule.field]!) + : generate(rule); + case "identity": case "unique": case "renewed": case "clock": @@ -360,7 +363,21 @@ export const createdFields = ( idField?: string, ): Applied => { if (plan.body === null) return err("The tool has no modelled body."); - const call = withBlanks(plan.body, sentCall); + // A member a `sequence` with `sent` numbers when a call leaves it out takes the next number, not an empty value one sent. + const numbered = new Set( + plan.rules.flatMap((rule) => + rule.rule === "sequence" && rule.sent === true ? [rule.field] : [], + ), + ); + const call = withBlanks( + { + ...plan.body, + blank: (plan.body.blank ?? []).filter( + ({ member }) => !numbered.has(member), + ), + }, + sentCall, + ); const problem = bodyProblem(plan.body, call, { entity: { ...copiedShapes(plan.rules, observed), ...observed }, }); @@ -489,6 +506,18 @@ export const updatedFields = ( setOwnMember(written, rule.field, value.value); } } + // A member the call leaves out takes the default a whole-object update showed; one a reference names by its member is not left out. + for (const { member, value } of plan.body.defaults ?? []) + if ( + !Object.hasOwn(sent, member) && + !plan.rules.some( + (rule) => + rule.rule === "reference" && + rule.field === member && + Object.hasOwn(sent, rule.member), + ) + ) + setOwnMember(written, member, value); const unspelled = derive( plan.rules.filter( (rule) => rule.rule !== "slug" || Object.hasOwn(sent, rule.from), diff --git a/src/domain/analysisView/environmentStateWriteCalls.ts b/src/domain/analysisView/environmentStateWriteCalls.ts index a649471573..2d6ab4087b 100644 --- a/src/domain/analysisView/environmentStateWriteCalls.ts +++ b/src/domain/analysisView/environmentStateWriteCalls.ts @@ -104,7 +104,10 @@ const sameObserved = (left: ObservedValue, right: ObservedValue): boolean => /** The equality facts a modelled update showed, scalar inputs it sent, and fields it kept, which later rules must reproduce. */ export interface UpdateCall { readonly shown: ReadonlyMap; + /** Members its answer left out that the read after it showed, which a default must agree with too. */ + readonly read: ReadonlyMap; readonly sent: ReadonlyMap; + readonly sentMembers: ReadonlySet; readonly kept: readonly string[]; } @@ -228,11 +231,21 @@ export const refinedBody = ( const accepted = later.accepted.filter( (member) => !body.accepted.includes(member), ); + const held = body.defaults ?? []; + const defaults = [ + ...held, + ...(later.defaults ?? []).filter( + (entry) => !held.some((kept) => kept.member === entry.member), + ), + ]; return { ...body, + ...(defaults.length === 0 ? {} : { defaults }), // A later call may be the first to show the body repeating the path's id. identity: body.identity ?? later.identity, accepted: [...body.accepted, ...accepted], + // A member is required only when every call sent it. + required: body.required.filter((member) => later.required.includes(member)), fixed: [ ...body.fixed, ...later.fixed.filter((entry) => !fixed.has(entry.member)), @@ -314,6 +327,42 @@ export const appliedUpdate = ( }; }; +/** Whether defaults disagree with a call that left their member out: a remembered call, or this one, showed it otherwise. */ +export const defaultsDisagree = ( + defaults: WriteBody["defaults"], + calls: { + readonly earlier: readonly UpdateCall[]; + readonly sent?: JsonObject; + readonly after?: JsonObject; + }, + rules: readonly FieldRule[], +): boolean => { + // A call that named a reference's entity by its member supplied the field. + const sources = new Map( + rules.flatMap((rule) => + rule.rule === "reference" ? [[rule.field, rule.member] as const] : [], + ), + ); + const supplied = (sent: (member: string) => boolean, member: string) => + sent(member) || (sources.has(member) && sent(sources.get(member)!)); + return (defaults ?? []).some( + ({ member, value }) => + calls.earlier.some((call) => { + const stored = call.shown.get(member) ?? call.read.get(member); + return ( + !supplied((name) => call.sentMembers.has(name), member) && + stored !== undefined && + !same(stored, observedValue(value)) + ); + }) || + (calls.sent !== undefined && + calls.after !== undefined && + !supplied((name) => Object.hasOwn(calls.sent!, name), member) && + Object.hasOwn(calls.after, member) && + !same(calls.after[member], value)), + ); +}; + /** Keep what a modelled update call showed, which later calls' rules must reproduce, and the fields it found already set. */ export const rememberCall = ( retain: (bytes: number) => void, @@ -328,6 +377,8 @@ export const rememberCall = ( readonly after: JsonObject; readonly sent: JsonObject; readonly kept: readonly string[]; + /** The entity as the read after the call showed it, when one did. */ + readonly read?: JsonObject | undefined; }, /** Clocks the call left as they were in its own second, which show neither a stamp nor a kept value. */ sameSecond: readonly string[], @@ -342,8 +393,29 @@ export const rememberCall = ( ]), ); for (const field of unstamped) shown.delete(field); - const facts = { shown, sent: scalars(call.sent), kept }; - retain(64 + jsonTextCodeUnits([...facts.shown, ...facts.sent, kept]) * 2); + const read = new Map( + Object.entries(call.read ?? {}) + .filter(([field]) => !Object.hasOwn(call.after, field)) + .map(([field, value]) => [field, observedValue(value)]), + ); + const facts = { + shown, + read, + sent: scalars(call.sent), + sentMembers: new Set(Object.keys(call.sent)), + kept, + }; + retain( + 64 + + jsonTextCodeUnits([ + ...facts.shown, + ...facts.read, + ...facts.sent, + [...facts.sentMembers], + kept, + ]) * + 2, + ); draft.updates.push(facts); for (const field of steadyFields(draft.plan.rules, call.before, call.after)) draft.steady.add(field); diff --git a/src/domain/analysisView/environmentStateWriteCreate.ts b/src/domain/analysisView/environmentStateWriteCreate.ts index 694c45d0c5..04a67a121f 100644 --- a/src/domain/analysisView/environmentStateWriteCreate.ts +++ b/src/domain/analysisView/environmentStateWriteCreate.ts @@ -295,8 +295,8 @@ const refineCreate = ( { rules: draft.plan.rules, body: draft.plan.body! }, draft.observed!, later, - earlierOf(draft), - ) ?? reexplained(draft, explained.rules, later); + earlierOf(draft, evidence.collection), + ) ?? reexplained(draft, explained.rules, later, evidence.collection); if (widened === undefined) return false; retainFixed(context, widened.body.fixed.slice(draft.plan.body!.fixed.length)); draft.plan.rules = widened.rules; @@ -307,11 +307,15 @@ const refineCreate = ( }; /** What the plan's earlier calls showed, for merging a later one. */ -const earlierOf = (draft: WriteDraft) => ({ +const earlierOf = (draft: WriteDraft, collection: string) => ({ kept: draft.kept, clockOf: (field: string) => clockEvidence(draft, field), startOf: (field: string) => draft.starts.get(field), createCollections: draft.createCollections, + sentLargest: (field: string, scope: "collection" | "type") => + draft.sentLargest.get( + scope === "collection" ? `${field} ${collection}` : field, + ), ampersands: draft.ampersands, }); @@ -338,6 +342,7 @@ const reexplained = ( draft: WriteDraft, learned: readonly FieldRule[], later: ReturnType, + collection: string, ) => { const input = draft.exemplarInput; if (input === undefined) return undefined; @@ -360,7 +365,7 @@ const reexplained = ( }, draft.observed!, later, - earlierOf(draft), + earlierOf(draft, collection), ); }; @@ -504,13 +509,21 @@ const sequencesOf = ( : [], ); -/** Whether a later call's sequences hold what serving generates: one past the largest value their scope holds or started with and past the plan's last, spelled as the exemplar spells it. */ +/** Whether a later call's sequences hold what serving generates: one past the largest value their scope holds or started with and past the plan's last, spelled as the exemplar spells it; or, for one that takes a sent value, the value the call sent. */ const followsSequences = ( context: EffectContext, draft: WriteDraft, - created: Pick, + created: Pick & { + readonly sent: JsonObject; + readonly observed: JsonObject; + }, ): boolean => sequencesOf(draft, created.collection).every(({ rule, key }) => { + if (rule.sent === true && Object.hasOwn(created.sent, rule.field)) + return ( + !Object.hasOwn(created.observed, rule.field) || + same(ownMember(created.observed, rule.field), created.sent[rule.field]) + ); const scope = [ ...context.state.ofType(created.type), ...context.startedWith(created.type), @@ -550,6 +563,7 @@ const agreesLater = ( ...created, observed: call.observed, startMs: call.startMs, + sent: call.body, }) && !contradictsDefault(draft.plan.rules, call, created.fields) && !unruledDiffers(draft.plan.rules, call.observed, { @@ -598,6 +612,7 @@ const followsCreateRules = ( created: Pick & { readonly observed: JsonObject; readonly startMs: number | undefined; + readonly sent: JsonObject; }, ): boolean => draft.plan.rules.every((rule) => { @@ -706,27 +721,45 @@ const fitClocks = (draft: WriteDraft, fields: JsonObject): void => { noteFixedWidths(draft.fixedWidth, fields); }; -/** Record the values a create's sequences took, which the next call follows. */ +/** Record the values a create's sequences took, which the next call follows; as serving does, the largest stays, so a lower sent value does not lower it. */ const takeSequences = ( draft: WriteDraft, created: Pick, ): void => { - for (const { rule, key } of sequencesOf(draft, created.collection)) { - const value = sequenceNumber( - ownMember(created.fields, rule.field), - ownMember(draft.observed, rule.field), + const take = (key: string, value: number | undefined) => { + if (value === undefined) return; + const taken = draft.sequences.get(key); + draft.sequences.set( + key, + taken === undefined ? value : Math.max(taken, value), + ); + }; + for (const { rule, key } of sequencesOf(draft, created.collection)) + take( + key, + sequenceNumber( + ownMember(created.fields, rule.field), + ownMember(draft.observed, rule.field), + ), ); - if (value !== undefined) draft.sequences.set(key, value); - } for (const rule of draft.plan.rules) { if (rule.rule !== "made-object") continue; const made = ownMember(created.fields, rule.field); for (const { member } of rule.counts) { - const value = isObject(made) - ? sequenceNumber(ownMember(made, member)) - : undefined; - if (value !== undefined) - draft.sequences.set(madeKey(rule.field, member), value); + take( + madeKey(rule.field, member), + isObject(made) ? sequenceNumber(ownMember(made, member)) : undefined, + ); + } + } + for (const rule of draft.plan.rules) { + if (rule.rule !== "echo") continue; + const value = sequenceNumber(ownMember(created.fields, rule.field)); + // By type and by collection, as a sequence adopted later may count either. + for (const key of [rule.field, `${rule.field} ${created.collection}`]) { + const held = draft.sentLargest.get(key); + if (value !== undefined && (held === undefined || value > held)) + draft.sentLargest.set(key, value); } } }; diff --git a/src/domain/analysisView/environmentStateWriteCreateCalls.ts b/src/domain/analysisView/environmentStateWriteCreateCalls.ts index 1383545146..f52b5fdf80 100644 --- a/src/domain/analysisView/environmentStateWriteCreateCalls.ts +++ b/src/domain/analysisView/environmentStateWriteCreateCalls.ts @@ -33,6 +33,11 @@ export interface EarlierEvidence { /** The integer the exemplar showed in an unruled field above every value its type held, where a sequence may start. */ readonly startOf?: (field: string) => number | undefined; readonly createCollections?: readonly string[]; + /** The largest integer an echoed field held on the plan's accepted creates, across the type or in the later call's collection. */ + readonly sentLargest?: ( + field: string, + scope: "collection" | "type", + ) => number | undefined; /** How the plan's accepted calls and the type's held entities spelled `&` in each field's slug; absent where none showed one. */ readonly ampersands?: ReadonlyMap; } @@ -298,6 +303,8 @@ interface FieldCalls { readonly exemplar: JsonObject; readonly later: { readonly entity: JsonObject; readonly body: JsonObject }; readonly sentBefore: ReadonlySet; + /** Members an earlier call left out and the application stored a fixed value for (`body.defaults`). */ + readonly defaulted: ReadonlySet; readonly required: readonly string[]; readonly earlier: EarlierEvidence; } @@ -339,6 +346,42 @@ const shownAlone = ( return sent ? undefined : { rule: own }; }; +/** A member one call sent and stored as sent, and another left out and the application numbered: the value a call sends, else the next number. */ +const sentOrNumbered = ( + before: FieldRule, + after: FieldRule, + field: string, + calls: FieldCalls, + agree: (held: FieldRule) => FieldRule | undefined, +): FieldRule | undefined => { + const laterSent = Object.hasOwn(calls.later.body, field); + if (before.rule === "echo") { + // Serving numbers past every value a call sent, so a later call numbered at or below one cannot be repeated. + const numbered = sequenceNumber(ownMember(calls.later.entity, field)); + const largest = + after.rule === "sequence" + ? calls.earlier.sentLargest?.(field, after.scope) + : undefined; + // An earlier call that left the member out stored a fixed value, so the application does not number it. + return after.rule === "sequence" && + !laterSent && + !calls.defaulted.has(field) && + (largest === undefined || (numbered !== undefined && numbered > largest)) + ? { ...after, sent: true } + : undefined; + } + if (before.rule !== "sequence") return undefined; + if (after.rule === "echo") + return before.sent === true || !calls.sentBefore.has(field) + ? { ...before, sent: true } + : undefined; + if (before.sent !== true || after.rule !== "sequence" || laterSent) + return undefined; + const { sent: _sent, ...numbered } = before; + const merged = agree(numbered); + return merged?.rule === "sequence" ? { ...merged, sent: true } : undefined; +}; + /** One field's rule in the merged plan, and the default it gives a member; undefined when the calls disagree on it. */ const mergedField = ( field: string, @@ -351,13 +394,18 @@ const mergedField = ( if (!inExemplar || !Object.hasOwn(later.entity, field)) return shownAlone(inExemplar, calls); if (before !== undefined && after !== undefined) { - const rule = agreeing(before, after, { + const values = { field, later: later.entity[field], earlier: calls.earlier, exemplar, laterEntity: later.entity, - }); + }; + const rule = + agreeing(before, after, values) ?? + sentOrNumbered(before, after, field, calls, (held) => + agreeing(held, after, values), + ); return rule === undefined ? undefined : { rule }; } // The plan echoes a member this call left out: what it stored there is the member's default. @@ -425,6 +473,9 @@ export const mergedCreatePlan = ( exemplar, later, sentBefore, + defaulted: new Set( + (plan.body.defaults ?? []).map(({ member }) => member), + ), required: plan.body.required, earlier, }); diff --git a/src/domain/analysisView/environmentStateWriteEffects.ts b/src/domain/analysisView/environmentStateWriteEffects.ts index 4ace3c1121..ef4696a2c6 100644 --- a/src/domain/analysisView/environmentStateWriteEffects.ts +++ b/src/domain/analysisView/environmentStateWriteEffects.ts @@ -6,6 +6,7 @@ import { isObject, locateItem, at, + same, type JsonObject, type ParsedRequestBody, requestJson, @@ -34,6 +35,7 @@ import { import { otherFields } from "./environmentStateWriteGenerated.js"; import { contradictsEarlierCalls, + defaultsDisagree, disagreesWithCall, keptFields, rememberCall, @@ -45,11 +47,19 @@ import { } from "./environmentStateWriteCalls.js"; import { laterObservation, + respelledInstant, shownBefore, viewFields, type Observed, + type ObservationSearch, + type ObservedRead, } from "./environmentStateWriteObserve.js"; -import { plannedBody, restampedFields } from "./environmentStateWriteStamps.js"; +import { + plannedBody, + readSpellsCoarser, + readSpelledClocks, + restampedFields, +} from "./environmentStateWriteStamps.js"; import { readClock } from "./environmentStateWriteValues.js"; import type { CreateEvidence } from "./environmentStateWriteExplain.js"; import { @@ -98,6 +108,8 @@ export interface WriteDraft { readonly starts: Map; /** Collections of accepted create calls, used to check late sequence adoption. */ readonly createCollections: string[]; + /** The largest integer each echoed field held on an accepted create, which a sequence adopted later must count past. */ + readonly sentLargest: Map; /** The collection a create writes into when its own path names no state type: the one its returned id was read under. */ readonly linked?: string | undefined; /** Fields an item action's call found already at the value it sets them to. */ @@ -487,6 +499,49 @@ const outcomes = ( return { targets, target, recorded }; }; +/** The item read after an update that spells a clock the update stamped at a coarser precision, which serving stores it as. */ +const spelledReadBack = ( + context: EffectContext, + step: WriteStep, + target: { + readonly type: string; + readonly idField: string; + readonly collection: string; + readonly id: string; + }, + call: { readonly rules: readonly FieldRule[]; readonly after: JsonObject }, +): ObservedRead | undefined => { + const read = laterObservation( + searchFrom(context, step), + itemReads(context, target), + ); + if (read === undefined || context.viewOf(read.ordinal)?.shape !== "item") + return undefined; + const respelled = call.rules.some((rule) => { + if (rule.rule !== "clock") return false; + const [own, shown] = [ + ownMember(call.after, rule.field), + ownMember(read.object, rule.field), + ]; + return respelledInstant(own, shown) && readSpellsCoarser(own, shown); + }); + return respelled + ? { ordinal: read.ordinal, pointer: read.pointer, shape: "item" } + : undefined; +}; + +/** The object the plan's kept read-back shows, which serving spells every update's clocks by. */ +const keptRead = ( + search: ObservationSearch, + read: ObservedRead | undefined, +): JsonObject | undefined => + read === undefined + ? undefined + : search.withBody(read.ordinal, (body) => { + const object = body === undefined ? undefined : at(body, read.pointer); + return isObject(object) ? object : undefined; + }); + /** Build an update plan from its exemplar, or add what a later call showed. */ const buildUpdate = ( context: EffectContext, @@ -639,6 +694,11 @@ export const updateEntity = ( refuse(draft, step, "unexplained-change"); return recorded(); } + // What the read after the call showed, for members its answer left out that a default must agree with. + const read = laterObservation( + searchFrom(context, step), + itemReads(context, { type, idField, collection, id }), + )?.object; // Each call must agree with the plan; one on an unknown entity cannot be checked, so it adds nothing to the plan. if ( built(draft) && @@ -648,6 +708,17 @@ export const updateEntity = ( entity.fields, after.object, explained, + ) || + // A member calls left out is cleared by every one or by none. + // A reference this call or an earlier one showed counts its member as the field's, whichever came first. + defaultsDisagree(explained.body.defaults, { earlier: draft.updates }, [ + ...rules, + ...explained.rules, + ]) || + defaultsDisagree( + draft.plan.body?.defaults, + { earlier: [], sent: body, after: { ...read, ...after.object } }, + [...rules, ...explained.rules], )) ) { if (!unknown) refuse(draft, step, "unexplained-change"); @@ -656,6 +727,21 @@ export const updateEntity = ( draft.plan.rules = rules; noteFixedWidths(draft.fixedWidth, after.object); buildUpdate(context, draft, step, { after, explained, idField }); + // Serving stores the clocks it stamps as the read after the update spells them. + const readBack = spelledReadBack( + context, + step, + { type, idField, collection, id }, + { rules: explained.rules, after: after.object }, + ); + if ( + readBack !== undefined && + draft.plan.observation !== null && + draft.plan.observation.read === undefined + ) + Object.assign(draft.plan, { + observation: { ...draft.plan.observation, read: readBack }, + }); noteAmpersands(draft.ampersands, explained.rules, after.object); // Serving reads the plan's observation, so the gate does too; generated values are this call's. const written = updatedFields( @@ -684,15 +770,20 @@ export const updateEntity = ( rememberCall( context.io.retain, draft, - { before: entity.fields, after: after.object, sent: body, kept }, + { before: entity.fields, after: after.object, sent: body, kept, read }, explained.unstamped, ); - // Response reproducibility is established before the read owner adopts a stored clock spelling. - const stored = restampedFields( - searchFrom(context, step), - itemReads(context, { type, idField, collection, id }), - written.value, + // Response reproducibility is established before the read owner adopts a stored clock spelling; serving spells every update's clocks as the plan's kept read-back does, so replay does too. + const search = searchFrom(context, step); + const stored = readSpelledClocks( draft.plan.rules, + restampedFields( + search, + itemReads(context, { type, idField, collection, id }), + written.value, + draft.plan.rules, + ), + keptRead(search, draft.plan.observation?.read), ); context.io.retain( 64 + diff --git a/src/domain/analysisView/environmentStateWriteExplain.ts b/src/domain/analysisView/environmentStateWriteExplain.ts index 6b2563724e..bc8aa8ad25 100644 --- a/src/domain/analysisView/environmentStateWriteExplain.ts +++ b/src/domain/analysisView/environmentStateWriteExplain.ts @@ -522,6 +522,8 @@ export const explainUpdate = ( const rules: FieldRule[] = []; const echoed = new Set(); const unexplained: string[] = []; + // A whole-object PUT clears a member it leaves out: that value is the member's default. + const defaults: { readonly member: string; readonly value: JsonValue }[] = []; // A member the application replaces with a value it generates is accepted with any value. const generated = new Set(); let identity: string | null = null; @@ -593,8 +595,14 @@ export const explainUpdate = ( body, knownEntities, ); - if (rule === undefined) unexplained.push(field); - else rules.push(rule); + if (rule !== undefined) rules.push(rule); + else if ( + evidence.method === "PUT" && + isBlankValue(ownMember(evidence.after, field)) && + !isBlankValue(ownMember(evidence.before, field)) + ) + defaults.push({ member: field, value: evidence.after[field]! }); + else unexplained.push(field); } rules.push(...pairedStamps(evidence, rules)); // A member a reference reads names an entity, which any held one may be. @@ -614,7 +622,11 @@ export const explainUpdate = ( ); return { rules, - body: { ...shaped, identity }, + body: { + ...shaped, + identity, + ...(defaults.length === 0 ? {} : { defaults }), + }, unexplained, unstamped: sameSecondClocks(evidence), }; diff --git a/src/domain/analysisView/environmentStateWriteRules.ts b/src/domain/analysisView/environmentStateWriteRules.ts index 3378f67be8..7da1aeb34b 100644 --- a/src/domain/analysisView/environmentStateWriteRules.ts +++ b/src/domain/analysisView/environmentStateWriteRules.ts @@ -53,12 +53,13 @@ export type GeneratedRule = readonly rule: "identity"; readonly generator: "sequence" | "uuid" | "token"; } - /** One past the largest value its scope holds, and never below `start`, a value the first create took above them. */ + /** One past the largest value its scope holds, and never below `start`, a value the first create took above them; with `sent`, the value a call sends instead. */ | { readonly field: string; readonly rule: "sequence"; readonly scope: "collection" | "type"; readonly start?: number; + readonly sent?: true; } | { readonly field: string; readonly rule: "unique" } /** The value kept with each UUID at the pointers (`*` for any array index) replaced by a fresh one, as rewritten child rows are. */ diff --git a/src/domain/analysisView/environmentStateWriteStamps.ts b/src/domain/analysisView/environmentStateWriteStamps.ts index ae6f49bef9..ec0347c17a 100644 --- a/src/domain/analysisView/environmentStateWriteStamps.ts +++ b/src/domain/analysisView/environmentStateWriteStamps.ts @@ -20,6 +20,7 @@ import { containsDelimited, isUuid, readClock, + formatClock, sequenceNumber, spelledLike, type ClockFormat, @@ -246,6 +247,44 @@ export const createdNestedClock = ( return stamped ? nestedRule(field, value, members, evidence.type) : undefined; }; +/** Whether an application that answers a clock as `written` stores it as `read` spells it: both ISO, the read at a coarser precision and otherwise alike. Replay and serving respell an update's clocks by this one rule. */ +export const readSpellsCoarser = ( + written: JsonValue | undefined, + read: JsonValue | undefined, +): boolean => { + const [value, stored] = [readClock(written), readClock(read)]; + return ( + value?.format.kind === "iso" && + stored?.format.kind === "iso" && + stored.format.fraction < value.format.fraction && + same({ ...stored.format, fraction: 0 }, { ...value.format, fraction: 0 }) + ); +}; + +/** An update's fields as its application stores them: each clock at the coarser precision its plan's read-back spelled it. */ +export const readSpelledClocks = ( + rules: readonly FieldRule[], + written: JsonObject, + read: JsonObject | undefined, +): JsonObject => { + if (read === undefined) return written; + const fields = { ...written }; + for (const rule of rules) { + if (rule.rule !== "clock") continue; + const answered = ownMember(written, rule.field); + const shown = ownMember(read, rule.field); + if (!readSpellsCoarser(answered, shown)) continue; + const clock = readClock(answered)!; + const spelled = formatClock( + clock.ms, + readClock(shown)!.format, + clock.subMillisecond, + ); + if (spelled !== undefined) setOwnMember(fields, rule.field, spelled); + } + return fields; +}; + /** Whether a clock is one a request starting at `startMs` stamped, in the given format. */ export const stampedBy = ( value: JsonValue | undefined, @@ -666,7 +705,10 @@ export const pinnedEchoes = ( ); }; -/** An update's fields as stored: each nested clock as the first later read of the entity shows it restamped, else as the call showed it. */ +/** + * A write's fields as stored: each nested clock as the first later read of the entity shows it restamped, else as the + * call showed it. + */ export const restampedFields = ( search: ObservationSearch, target: Parameters[1], diff --git a/src/domain/analysisView/environmentStateWriteValues.ts b/src/domain/analysisView/environmentStateWriteValues.ts index b2135f7ae0..ce96e87bcd 100644 --- a/src/domain/analysisView/environmentStateWriteValues.ts +++ b/src/domain/analysisView/environmentStateWriteValues.ts @@ -114,18 +114,20 @@ const isoText = ( return `${pad(year, 4)}-${pad(at.getUTCMonth() + 1, 2)}-${pad(at.getUTCDate(), 2)}${separator}${pad(at.getUTCHours(), 2)}:${pad(at.getUTCMinutes(), 2)}:${pad(at.getUTCSeconds(), 2)}${digits === "" ? "" : `.${digits}`}${zone}`; }; -/** An instant in a recorded format; undefined outside its four-digit ISO year range. Fraction digits past milliseconds are zeros. */ +/** An instant in a recorded format; undefined outside its four-digit ISO year range. Fraction digits past milliseconds are `subMillisecond`'s, else zeros. */ export const formatClock = ( ms: number, format: ClockFormat, + subMillisecond = 0, ): JsonValue | undefined => { if (format.kind === "epoch") return format.unit === "s" ? Math.floor(ms / 1000) : ms; const at = new Date(ms + zoneMinutesForFormat(format.zone) * MINUTE_MS); - const digits = `${pad(at.getUTCMilliseconds(), 3)}${"0".repeat(9)}`.slice( - 0, - format.fraction, - ); + const digits = + `${pad(at.getUTCMilliseconds(), 3)}${pad(subMillisecond, 6)}`.slice( + 0, + format.fraction, + ); return isoText( at, format.separator, diff --git a/src/generatedPackageMetadata.ts b/src/generatedPackageMetadata.ts index 25bb35bb64..5372dbba8b 100644 --- a/src/generatedPackageMetadata.ts +++ b/src/generatedPackageMetadata.ts @@ -6,5 +6,5 @@ export const PACKAGE_METADATA = { serverSdkVersion: "2.3.1", clientSdkVersion: "2.3.1", coreSdkVersion: "2.3.1", - skillVersion: "53", + skillVersion: "54", } as const; diff --git a/tests/fixtures/environmentStateVariedCalls.ts b/tests/fixtures/environmentStateVariedCalls.ts new file mode 100644 index 0000000000..fc0baadde4 --- /dev/null +++ b/tests/fixtures/environmentStateVariedCalls.ts @@ -0,0 +1,466 @@ +import type { TimedCaptureRow } from "./analysisView.js"; +import { bookClock } from "./environmentStateWrites.js"; +import type { JsonValue } from "../../src/domain/jsonValue.js"; + +type Entity = { readonly [key: string]: JsonValue }; + +/** Two contact updates, the second sending only the email the first sent with a name and a city. */ +export const FEWER_MEMBERS_ROWS: readonly TimedCaptureRow[] = (() => { + const contact = ( + id: number, + name: string, + city: string, + email: string, + ): Entity => ({ id, name, city, email }); + const [ada, mina] = [ + contact(1, "Ada Finch", "Porto", "ada@example.test"), + contact(2, "Mina Okafor", "Accra", "mina@example.test"), + ]; + const adaMoved = contact( + 1, + "Ada Finch-Lowe", + "Lisbon", + "ada.lowe@example.test", + ); + const minaMail = { ...mina, email: "mina.o@example.test" }; + return [ + ["GET", "/api/contacts", [ada, mina]], + ["GET", "/api/contacts/1", ada], + ["GET", "/api/contacts/2", mina], + [ + "PUT", + "/api/contacts/1", + adaMoved, + 200, + { + name: "Ada Finch-Lowe", + city: "Lisbon", + email: "ada.lowe@example.test", + }, + ], + ["GET", "/api/contacts/1", adaMoved], + ["PUT", "/api/contacts/2", minaMail, 200, { email: "mina.o@example.test" }], + ["GET", "/api/contacts/2", minaMail], + ["GET", "/api/contacts", [adaMoved, minaMail]], + ]; +})(); + +/** Bundle creates: the first sends its order, the second leaves it to the application, which numbers it next. */ +export const NUMBERED_UNLESS_SENT_ROWS: readonly TimedCaptureRow[] = (() => { + const bundle = (id: number, name: string, order: number): Entity => ({ + id, + name, + order, + }); + const [alpha, bravo] = [bundle(1, "Alpha", 0), bundle(2, "Bravo", 1)]; + const [charlie, delta] = [bundle(3, "Charlie", 5), bundle(4, "Delta", 6)]; + return [ + ["GET", "/api/bundles", [alpha, bravo]], + ["GET", "/api/bundles/1", alpha], + ["GET", "/api/bundles/2", bravo], + ["POST", "/api/bundles", charlie, 201, { name: "Charlie", order: 5 }], + ["GET", "/api/bundles/3", charlie], + ["GET", "/api/bundles", [alpha, bravo, charlie]], + ["POST", "/api/bundles", delta, 201, { name: "Delta" }], + ["GET", "/api/bundles/4", delta], + ["GET", "/api/bundles", [alpha, bravo, charlie, delta]], + ]; +})(); + +/** Tasks marked done: each answer stamps `done_at` to the nanosecond, the read after it to the second. */ +export const DONE_CLOCK_ROWS: readonly TimedCaptureRow[] = (() => { + const task = (id: number, title: string): Entity => ({ + id, + title, + done: false, + done_at: "0001-01-01T00:00:00Z", + }); + const [trim, oil] = [task(1, "Trim the wicks"), task(2, "Order lamp oil")]; + const done = (entity: Entity, at: number, fraction: string): Entity => ({ + ...entity, + done: true, + done_at: bookClock(at).replace("Z", fraction), + }); + return [ + ["GET", "/api/tasks", [trim, oil]], + ["GET", "/api/tasks/1", trim], + ["GET", "/api/tasks/2", oil], + [ + "PATCH", + "/api/tasks/1", + done(trim, 3, ".001234567Z"), + 200, + { done: true }, + ], + ["GET", "/api/tasks/1", done(trim, 3, "Z")], + ["PATCH", "/api/tasks/2", done(oil, 5, ".002345678Z"), 200, { done: true }], + ["GET", "/api/tasks/2", done(oil, 5, "Z")], + ]; +})(); + +/** Tasks marked done, each answer to the nanosecond: the first read spells `done_at` to the millisecond, the second to the second. */ +export const DONE_MIXED_ROWS: readonly TimedCaptureRow[] = DONE_CLOCK_ROWS.map( + (row, index) => + index === 4 && row[0] === "GET" + ? [ + row[0], + row[1], + { + ...(row[2] as Entity), + done_at: String((row[2] as Entity)["done_at"]).replace( + "Z", + ".001Z", + ), + }, + ] + : row, +); + +/** Bundles numbered unless an order is sent: 2 left out, 9 and 4 sent, the 9 deleted, then one left out the application numbers 5. */ +export const SENT_THEN_DELETED_ROWS: readonly TimedCaptureRow[] = (() => { + const bundle = (id: number, name: string, order: number): Entity => ({ + id, + name, + order, + }); + const seeds = [bundle(1, "Alpha", 0), bundle(2, "Bravo", 1)]; + const [charlie, delta, echo, foxtrot] = [ + bundle(3, "Charlie", 2), + bundle(4, "Delta", 9), + bundle(5, "Echo", 4), + bundle(6, "Foxtrot", 5), + ]; + return [ + ["GET", "/api/bundles", seeds], + ["GET", "/api/bundles/1", seeds[0]!], + ["POST", "/api/bundles", charlie, 201, { name: "Charlie" }], + ["GET", "/api/bundles/3", charlie], + ["GET", "/api/bundles", [...seeds, charlie]], + ["POST", "/api/bundles", delta, 201, { name: "Delta", order: 9 }], + ["GET", "/api/bundles/4", delta], + ["GET", "/api/bundles", [...seeds, charlie, delta]], + ["POST", "/api/bundles", echo, 201, { name: "Echo", order: 4 }], + ["GET", "/api/bundles/5", echo], + ["GET", "/api/bundles", [...seeds, charlie, delta, echo]], + ["DELETE", "/api/bundles/4", null, 204], + ["GET", "/api/bundles", [...seeds, charlie, echo]], + ["POST", "/api/bundles", foxtrot, 201, { name: "Foxtrot" }], + ["GET", "/api/bundles/6", foxtrot], + ["GET", "/api/bundles", [...seeds, charlie, echo, foxtrot]], + ]; +})(); + +/** Tasks marked done: each answer stamps `done_at` to the millisecond in `Z`, the read after it to the second in `+00:00`. */ +export const DONE_ZONE_ROWS: readonly TimedCaptureRow[] = (() => { + const task = (id: number, title: string): Entity => ({ + id, + title, + done: false, + done_at: "0001-01-01T00:00:00Z", + }); + const [trim, oil] = [task(1, "Trim the wicks"), task(2, "Order lamp oil")]; + const done = (entity: Entity, at: number, suffix: string): Entity => ({ + ...entity, + done: true, + done_at: bookClock(at).replace("Z", suffix), + }); + return [ + ["GET", "/api/tasks", [trim, oil]], + ["GET", "/api/tasks/1", trim], + ["GET", "/api/tasks/2", oil], + ["PATCH", "/api/tasks/1", done(trim, 3, ".123Z"), 200, { done: true }], + ["GET", "/api/tasks/1", done(trim, 3, "+00:00")], + ["PATCH", "/api/tasks/2", done(oil, 5, ".234Z"), 200, { done: true }], + ["GET", "/api/tasks/2", done(oil, 5, "+00:00")], + ]; +})(); + +/** Tag PUTs: one leaves the owner out and the application clears it, the other sends it and keeps it. */ +export const PUT_CLEARS_ROWS: readonly TimedCaptureRow[] = (() => { + const tag = (id: number, name: string, owner: number | null): Entity => ({ + id, + name, + owner, + }); + const [inbox, bills] = [tag(1, "Inbox", 3), tag(2, "Bills", 3)]; + const [triage, paid] = [tag(1, "Triage", null), tag(2, "Paid", 3)]; + return [ + ["GET", "/api/tags", [inbox, bills]], + ["GET", "/api/tags/1", inbox], + ["GET", "/api/tags/2", bills], + ["PUT", "/api/tags/1", triage, 200, { name: "Triage" }], + ["GET", "/api/tags/1", triage], + ["PUT", "/api/tags/2", paid, 200, { name: "Paid", owner: 3 }], + ["GET", "/api/tags/2", paid], + ["GET", "/api/tags", [triage, paid]], + ]; +})(); + +/** Bundle creates the other way round: the first leaves its order to the application, the second sends one. */ +export const NUMBERED_THEN_SENT_ROWS: readonly TimedCaptureRow[] = (() => { + const bundle = (id: number, name: string, order: number): Entity => ({ + id, + name, + order, + }); + const [alpha, bravo] = [bundle(1, "Alpha", 0), bundle(2, "Bravo", 1)]; + const [charlie, delta] = [bundle(3, "Charlie", 2), bundle(4, "Delta", 9)]; + return [ + ["GET", "/api/bundles", [alpha, bravo]], + ["GET", "/api/bundles/1", alpha], + ["GET", "/api/bundles/2", bravo], + ["POST", "/api/bundles", charlie, 201, { name: "Charlie" }], + ["GET", "/api/bundles/3", charlie], + ["GET", "/api/bundles", [alpha, bravo, charlie]], + ["POST", "/api/bundles", delta, 201, { name: "Delta", order: 9 }], + ["GET", "/api/bundles/4", delta], + ["GET", "/api/bundles", [alpha, bravo, charlie, delta]], + ]; +})(); + +/** Two name-only tag PUTs, one keeping the owner and one clearing it, in the given order. */ +const ownerPuts = (clearsFirst: boolean): readonly TimedCaptureRow[] => { + const tag = (id: number, name: string, owner: number | null): Entity => ({ + id, + name, + owner, + }); + const [inbox, bills] = [tag(1, "Inbox", 3), tag(2, "Bills", 3)]; + const [triage, paid] = [ + tag(1, "Triage", clearsFirst ? null : 3), + tag(2, "Paid", clearsFirst ? 3 : null), + ]; + return [ + ["GET", "/api/tags", [inbox, bills]], + ["GET", "/api/tags/1", inbox], + ["GET", "/api/tags/2", bills], + ["PUT", "/api/tags/1", triage, 200, { name: "Triage" }], + ["GET", "/api/tags/1", triage], + ["PUT", "/api/tags/2", paid, 200, { name: "Paid" }], + ["GET", "/api/tags/2", paid], + ["GET", "/api/tags", [triage, paid]], + ]; +}; + +/** A name-only PUT that keeps the owner, then one that clears it. */ +export const KEPT_THEN_CLEARED_ROWS = ownerPuts(false); + +/** Two name-only tag PUTs where one answers only `id` and `name` and its read shows the owner kept, and the other clears it. */ +const omittedOwnerPuts = (omitsFirst: boolean): readonly TimedCaptureRow[] => { + const tag = (id: number, name: string, owner: number | null): Entity => ({ + id, + name, + owner, + }); + const [inbox, bills] = [tag(1, "Inbox", 3), tag(2, "Bills", 3)]; + const kept = (entity: Entity, name: string) => ({ ...entity, name }); + const answered = (entity: Entity) => ({ + id: entity["id"]!, + name: entity["name"]!, + }); + const [triage, paid] = omitsFirst + ? [kept(inbox, "Triage"), tag(2, "Paid", null)] + : [tag(1, "Triage", null), kept(bills, "Paid")]; + const answer = (entity: Entity, omits: boolean) => + omits ? answered(entity) : entity; + return [ + ["GET", "/api/tags", [inbox, bills]], + ["GET", "/api/tags/1", inbox], + ["GET", "/api/tags/2", bills], + ["PUT", "/api/tags/1", answer(triage, omitsFirst), 200, { name: "Triage" }], + ["GET", "/api/tags/1", triage], + ["PUT", "/api/tags/2", answer(paid, !omitsFirst), 200, { name: "Paid" }], + ["GET", "/api/tags/2", paid], + ["GET", "/api/tags", [triage, paid]], + ]; +}; + +/** A name-only PUT answered without the owner its read shows kept, then one that clears it. */ +export const OMITTED_KEPT_THEN_CLEARED_ROWS = omittedOwnerPuts(true); + +/** A name-only PUT that clears the owner, then one answered without the owner its read shows kept. */ +export const CLEARED_THEN_OMITTED_KEPT_ROWS = omittedOwnerPuts(false); + +/** A name-only PUT that clears the owner, then one that keeps it. */ +export const CLEARED_THEN_KEPT_ROWS = ownerPuts(true); + +/** Bundle creates: the first sends order 5, the second leaves it out and the application stores 5, the third leaves it out and gets 6. */ +export const DEFAULT_THEN_NUMBERED_ROWS: readonly TimedCaptureRow[] = (() => { + const bundle = (id: number, name: string, order: number): Entity => ({ + id, + name, + order, + }); + const [alpha, bravo] = [bundle(1, "Alpha", 0), bundle(2, "Bravo", 1)]; + const [charlie, delta, echo] = [ + bundle(3, "Charlie", 5), + bundle(4, "Delta", 5), + bundle(5, "Echo", 6), + ]; + return [ + ["GET", "/api/bundles", [alpha, bravo]], + ["GET", "/api/bundles/1", alpha], + ["GET", "/api/bundles/2", bravo], + ["POST", "/api/bundles", charlie, 201, { name: "Charlie", order: 5 }], + ["GET", "/api/bundles/3", charlie], + ["GET", "/api/bundles", [alpha, bravo, charlie]], + ["POST", "/api/bundles", delta, 201, { name: "Delta" }], + ["GET", "/api/bundles/4", delta], + ["GET", "/api/bundles", [alpha, bravo, charlie, delta]], + ["POST", "/api/bundles", echo, 201, { name: "Echo" }], + ["GET", "/api/bundles/5", echo], + ["GET", "/api/bundles", [alpha, bravo, charlie, delta, echo]], + ]; +})(); + +/** Tasks marked done, each answer to the nanosecond and each read after it to the microsecond. */ +export const DONE_MICRO_ROWS: readonly TimedCaptureRow[] = DONE_CLOCK_ROWS.map( + (row, index) => { + const answer = DONE_CLOCK_ROWS[index - 1]?.[2] as Entity | undefined; + return row[0] === "GET" && (index === 4 || index === 6) + ? [ + row[0], + row[1], + { + ...(row[2] as Entity), + done_at: String(answer!["done_at"]).replace( + /(\.\d{6})\d{3}Z$/u, + "$1Z", + ), + }, + ] + : row; + }, +); + +/** Tag PUTs whose owner is a user object, in the given order: one names the owner by `ownerId`, the other leaves it out and the application clears it. */ +const ownerReferencePuts = ( + clearsFirst: boolean, +): readonly TimedCaptureRow[] => { + const [ada, bo] = [ + { id: 3, name: "Ada" }, + { id: 4, name: "Bo" }, + ]; + const tag = (id: number, name: string, owner: Entity | null): Entity => ({ + id, + name, + owner, + }); + const [inbox, bills] = [tag(1, "Inbox", bo), tag(2, "Bills", bo)]; + const named = tag(clearsFirst ? 2 : 1, clearsFirst ? "Paid" : "Triage", ada); + const cleared = tag( + clearsFirst ? 1 : 2, + clearsFirst ? "Triage" : "Paid", + null, + ); + const put = (entity: Entity): TimedCaptureRow[] => [ + [ + "PUT", + `/api/tags/${String(entity["id"])}`, + entity, + 200, + entity["owner"] === null + ? { name: entity["name"]! } + : { name: entity["name"]!, ownerId: 3 }, + ], + ["GET", `/api/tags/${String(entity["id"])}`, entity], + ]; + const [first, second] = clearsFirst ? [cleared, named] : [named, cleared]; + return [ + ["GET", "/api/users", [ada, bo]], + ["GET", "/api/users/3", ada], + ["GET", "/api/users/4", bo], + ["GET", "/api/tags", [inbox, bills]], + ["GET", "/api/tags/1", inbox], + ["GET", "/api/tags/2", bills], + ...put(first), + ...put(second), + [ + "GET", + "/api/tags", + [first, second].sort((a, b) => Number(a["id"]) - Number(b["id"])), + ], + ]; +}; + +/** A PUT naming the owner by `ownerId`, then one that leaves it out and clears it. */ +export const OWNER_NAMED_THEN_CLEARED_ROWS = ownerReferencePuts(false); + +/** A PUT that leaves the owner out and clears it, then one naming it by `ownerId`. */ +export const OWNER_CLEARED_THEN_NAMED_ROWS = ownerReferencePuts(true); + +/** Bundle creates with a string order: the first leaves it out and the application numbers it "2", the second sends "" and stores it. */ +export const NUMBERED_THEN_BLANK_ROWS: readonly TimedCaptureRow[] = (() => { + const bundle = (id: number, name: string, order: string): Entity => ({ + id, + name, + order, + }); + const [alpha, bravo] = [bundle(1, "Alpha", "0"), bundle(2, "Bravo", "1")]; + const [charlie, delta] = [bundle(3, "Charlie", "2"), bundle(4, "Delta", "")]; + return [ + ["GET", "/api/bundles", [alpha, bravo]], + ["GET", "/api/bundles/1", alpha], + ["GET", "/api/bundles/2", bravo], + ["POST", "/api/bundles", charlie, 201, { name: "Charlie" }], + ["GET", "/api/bundles/3", charlie], + ["GET", "/api/bundles", [alpha, bravo, charlie]], + ["POST", "/api/bundles", delta, 201, { name: "Delta", order: "" }], + ["GET", "/api/bundles/4", delta], + ["GET", "/api/bundles", [alpha, bravo, charlie, delta]], + ]; +})(); + +/** Bundle creates: the first sends order 9 and is deleted, the second leaves it out and the application numbers it 2. */ +export const SENT_DELETED_THEN_NUMBERED_ROWS: readonly TimedCaptureRow[] = + (() => { + const bundle = (id: number, name: string, order: number): Entity => ({ + id, + name, + order, + }); + const [alpha, bravo] = [bundle(1, "Alpha", 0), bundle(2, "Bravo", 1)]; + const [charlie, delta] = [bundle(3, "Charlie", 9), bundle(4, "Delta", 2)]; + return [ + ["GET", "/api/bundles", [alpha, bravo]], + ["GET", "/api/bundles/1", alpha], + ["GET", "/api/bundles/2", bravo], + ["POST", "/api/bundles", charlie, 201, { name: "Charlie", order: 9 }], + ["GET", "/api/bundles/3", charlie], + ["GET", "/api/bundles", [alpha, bravo, charlie]], + ["DELETE", "/api/bundles/3", null, 204], + ["GET", "/api/bundles", [alpha, bravo]], + ["POST", "/api/bundles", delta, 201, { name: "Delta" }], + ["GET", "/api/bundles/4", delta], + ["GET", "/api/bundles", [alpha, bravo, delta]], + ]; + })(); + +/** Bundles numbered per project: a create in project 1 sends order 9, one in project 2 leaves it out and gets 1. */ +export const SENT_IN_ANOTHER_PROJECT_ROWS: readonly TimedCaptureRow[] = (() => { + const bundle = (id: number, name: string, order: number): Entity => ({ + id, + name, + order, + }); + const [alpha, bravo] = [bundle(1, "Alpha", 0), bundle(2, "Bravo", 0)]; + const [charlie, delta] = [bundle(3, "Charlie", 9), bundle(4, "Delta", 1)]; + return [ + ["GET", "/api/projects/1/bundles", [alpha]], + ["GET", "/api/projects/1/bundles/1", alpha], + ["GET", "/api/projects/2/bundles", [bravo]], + ["GET", "/api/projects/2/bundles/2", bravo], + [ + "POST", + "/api/projects/1/bundles", + charlie, + 201, + { name: "Charlie", order: 9 }, + ], + ["GET", "/api/projects/1/bundles/3", charlie], + ["GET", "/api/projects/1/bundles", [alpha, charlie]], + ["POST", "/api/projects/2/bundles", delta, 201, { name: "Delta" }], + ["GET", "/api/projects/2/bundles/4", delta], + ["GET", "/api/projects/2/bundles", [bravo, delta]], + ]; +})();