Files
rea/src/domain/analysisView/environmentStateWriteValues.ts
T
131fbe0754 feat(environment): serve writes whose calls send different members (#7539)
* feat(environment): serve the writes callers make without recorded form fields

- A create member a recording sent empty ("", null, false, [] or {}) is no
  longer required; a call that leaves it out is applied as sending that value
  (body.blank).
- List views render envelope members that list the listed entities' ids
  (Paperless's `all`), so creates in such collections are confirmed.
- Updates move their entity in collections kept sorted by a field they write,
  and a failed whole-collection read after an update can teach that order.
- A later create that stores an unruled field otherwise is withheld at the
  create, and an item read may fill values a create's answer had not loaded.
- An unsent field respelled between null, [] and {} is unchanged.
- The skill asks for one create with every field and one with only what a
  person must give.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skill): show how to read signed-in pages before recording

Agents guessed capture keys (artifacts, dom, final) to see pages behind
sign-in, which inspect_web_page cannot reach.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): accept every empty spelling recorded creates sent

A create that fills only what a person must give sends `null` where the
full create sent a string, and `null` or `false` for the same flag. The
merged plan now accepts each recorded empty value, and a call that leaves
the member out is applied as the earliest call sent it.

Also adds the spec folder for this change.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skill): keep the vary-fields step neutral on UI or API writes

Agents read "the form" as a cue to write through the UI, where some
applications' request bodies are not exposed; a body-unusable UI write
now re-records once with request steps.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(environment): record verification for blank members and sorted writes

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): address review of blank members and sorted writes

- Bulk updates move their entities in turn against the others' earlier
  values and install the final order once, as replay does.
- A member's recorded empty values stay accepted once a later call teaches
  its default; a left-out member with a default keeps the default.
- Write answers spell empty lists and maps as their recorded answers did.
- A rename can teach a sort when the creates chose one by a field the
  update did not write, such as ids that ascend with titles.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(environment): serve the writes that later reads explain

- An item read whose object is marked deleted (deleted, isDeleted or
  is_deleted true) agrees with an absent entity, as EspoCRM answers a
  deleted record; such an object never seeds or adopts an entity.
- maxSize, max_results and maxResults are page sizes, for whole-collection
  list lines and the list query's limit role.
- An item view also shows the fields only reads after its type's first write
  show, when no initial entity holds them (Vikunja's task subscription),
  without requiring them; write answers leave such a field out for entities
  without it, and it no longer withholds the write as incomplete-entities.
- The skill asks updates to change the fields a person would change, and
  expects a deleted object to read back as 404 or marked deleted.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): answer sent empty members as sent, and keep a placed create's order

- A write answer spells an empty list or map the call did not send as its
  recorded answer did; one the call sent is answered as sent, so a create
  sending `[]` is not answered `null` because the first recorded create was.
- A read listing the whole collection that placed a create leaves the create
  only the sorted inserts whose order that read follows, so a later rename
  cannot teach a sort an earlier read ruled out.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): answer bulk updates without fields only later reads show

A bulk update's answer renders each entity with the same allowance a single
update's answer has: a field only reads after the type's first write showed,
which no initial entity holds, is left out for an entity without it, so a
bulk update of a seeded task is no longer refused for lacking it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(environment): correct how the EspoCRM contact update task passes

The agent trajectories show gpt-oss passes it by creating a replacement contact
and deleting the original, since the served update refuses emailAddress; the
update itself is refused in both builds.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): answer a read of a served-deleted entity marked deleted

An application that soft-deletes still answers a deleted record by id, marked
deleted. Serving now keeps an entity a served delete removed from a type whose
recorded item reads show that, until a reset or load and within the size
data/add loads, and answers a read of it with its last fields and the flag
true, instead of refusing the read.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): keep unloaded create answers, check bulk answers with sent members, narrow inserts only on collection-order reads

- A create whose item read loaded values its answer had not records them
  (`observation.read.unloaded`), and a served create's answer keeps them as
  the recorded answer showed them, a zero-time owner included, while the state
  holds the read's.
- Replay checks a bulk update's recorded answers with each target's sent
  members, as serving answers them, so echoed `null` and `[]` no longer make
  the plan body-unusable.
- A whole-collection read whose query sorts it shows the query's order, so it
  no longer narrows a create's sorted insert candidates.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(environment): record the census with main's unknown-clock rule merged

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(environment): serve query numbers, counted values and made objects

- A query parameter accepts a number or boolean, which serving sends as its
  JSON text, as agents send page sizes and flags.
- A sequence may start at the value its first create took above every value
  its type held (Vikunja's per-project task index), and a later create's
  composed, template or started-sequence rule is adopted when it also spells
  the first create's value.
- A made-object rule explains an object the application makes with each
  created entity: members the request stamped, members holding the entity's
  own id, and counts one past the held objects' values.
- A create's answer names the id of the one object a one-element array member
  holds, so bulk creates link to their type.
- The skill reads each listed object by id before writing.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(environment): let a linked create take path values it stores

A create linked to its type's own collection, whose path has no parameters,
accepts any path values when its plan stores each one in a field, as a Vikunja
task stores its project; replay and serving apply it alike.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): learn a sort from a rename only when moving its entities reaches it

Serving moves only the entities a write changes in a collection kept sorted,
so replay now tries a sort after a rename only when the entities no write the
read exercised changed already follow it. Seeded folders listed out of title
order no longer teach a title sort that serving would answer with another
order.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): keep unloaded create answers a sent value did not echo, and learn no sort from query-sorted reads

- A served create keeps a value its recorded answer had not loaded unless the
  call sent that member and the plan stores it as sent, so a create sending
  `active: false` is answered `false` as recorded while the state holds the
  read's `true`.
- A rename no longer teaches a sort from a whole-collection read its query
  sorts, whose ties need not follow the collection's own order.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(environment): verify query values, counted values and made objects

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): learn the soft-delete flag in the state view and serve by it

Extraction now records the member later item reads set true on a record the
application soft-deleted, and the item view carries it as `deleted_flag`.
Serving reads that fact instead of scanning recorded bodies itself, so one
owner decides what a soft delete looks like.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(environment): record the call-values checks at #7274 93821024

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): learn the soft-delete flag from item reads with a query too

A client may read a deleted record back only with a query, such as a
cache-busting parameter. Every successful read of the item tool now shows the
flag; only plain later reads still add fields.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(environment): record the call-values checks at #7274 45832406

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): learn sorts only from orders a list read confirmed

- A sort retry after a failed whole-collection read requires the entities no
  write changed since a list last showed them to follow it already, rather than
  excusing every entity the update's tool ever wrote: an earlier rename a list
  confirmed in place stays where it was.
- An update takes its creates' sort only from a create a list read showed
  placed; replay still tries a sort a create guessed from the initial order.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): drop entities read back as deleted and answer soft deletes as deleted

- A held entity a later read shows marked deleted is dropped, not adopted with
  the marked object's fields.
- Bulk-update replay renders answers with the optional and later-only fields
  serving uses, so a bulk update of created and seeded entities replays.
- A served delete of a soft-deleting type answers with the entity marked
  deleted, single and bulk.
- A deleted flag only query reads showed is a field of the item view, so a
  served tombstone keeps it where live records omit it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): check made objects as recorded and keep their ids and clocks honest

- A made object's clock counts as a write clock, so a capture with no clock
  anchor withholds the create instead of stamping 1970.
- Members holding the entity's identity follow the id written, generated or
  sent, through the type's id field.
- A later create's made object is checked as recorded, before its own members
  are rewritten, so one naming another entity withholds the plan.
- A later create's template is adopted only when it spells that create's value
  too, not just when the first create's text holds its source value.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): move updated entities only by placed sorts and keep query-sorted reads from confirming order

Replay moved an updated entity by a sort a create only guessed, while the
update plan exported only placed sorts, so a read could confirm an update
by a move serving would not make. Both now use the placed sort. A passing
list read clears its entities from the moved set only when the state
decides the order of every entity it lists, so a read whose query sorts
it no longer hides an earlier move from a later sort check.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(environment): count six withheld plans that explain more, not seven

The bulk task create counted among them is served on this branch.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): try a learned sort by moving each pending write in turn, as serving does

#retrySort sorted the whole collection, which a list could match while
serving, moving each renamed entity in turn against the others' values
before their own writes, gave another order. The state now records the
values each write replaced, and the retry replays the writes since a
list in collection order last showed their entities.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): answer a served delete with the deleted flag its recorded answer showed

An application may answer a delete with the record as it was and mark it
deleted only on later reads. Delete answers, single and bulk, now take
the flag's value from the recorded answer; reads of a served-deleted
entity still show it marked deleted.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* chore(environment): set aside two hunks next to lines main's saved-values change rewrites

Main's #7267 changes the lines beside the blank-member docs and the
empty-spelling check in explainUpdate. Setting them aside lets that merge
apply cleanly; the commit after it puts them back on top of main's text.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* chore(environment): restore the set-aside hunks on main and bump the skill to 51

Puts the blank-member docs and the empty-spelling check back beside main's
slug and composed text and heldSlugRule. Main is at skill 50 with
different text, so this branch's skill changes take 51.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* chore(environment): bump the skill to 52 above #7260's 51

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* chore(environment): put back the call-values hunks main's merge took its side for, and bump the skill to 53

The merge of main kept main's side where it overlapped this branch, so it
could commit without a conflict. This restores this branch's side next to
main's: made-object agreement beside the & slug spellings, sequence starts
in the earlier evidence (now built by earlierOf), the combined value
imports, and recording step 2 beside main's UI-first step 3. Main is at
skill 50 and #7274 at 52, so this branch takes 53.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): keep sorts a list showed in place ruled out, and created answers' nested members as recorded

A whole list in collection order that showed a renamed entity where a
sort by the renamed field would have moved it now rules that sort out, so
a later rename of the same entity cannot teach it. A create's answer
leaves out members its item read added inside an object the answer
showed, as the recorded answer did; the state still stores them.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(environment): serve writes whose calls send different members

Recordings now vary what a type's two creates and updates send, and REA
withheld writes where a later call left out what the first sent. An update
plan's required members narrow to those every call sent; a member one create
sent and another left for the application to number is a sequence that
takes the value a call sends; a PUT's left-out member the application
cleared becomes that member's default; and an update's clock is held as
the read after it spells it. The skill spaces creates (skill 54).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): keep a later create's made object only with clocks its request stamped

followsMadeObjects only checked that a made object's clocks parsed, so a
later create whose object showed an older clock still passed, while serving
stamps the write's clock. Its clocks must now be in the rule's format and
within the stamp window of that create's request.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): accept sent values either way, check defaults against every call, and store update clocks as reads spell them

A sequence that takes sent values now accepts one a later create sent
after the first left the member to be numbered. A default an update learned
must agree with every update that left its member out, else the plan is
withheld. An update plan whose read-back spells a stamped clock at a coarser
precision keeps that read as observation.read, and serving stores the
update's clocks in its spelling while answers keep the response's.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): check recorded member and sequence values

* fix(environment): compare soft-deleted read fields

* fix(environment): reject contradictory learned sequences

* fix(environment): preserve initially deleted item observations

* fix(environment): index stored deleted entities separately

* fix(environment): replay lists that include deleted records

* fix(environment): count a sequence number a create sends, as replay does

Serving now records a number a call sends for a `sequence` with `sent` in the
same count history it generates from, so after that entity is deleted the next
generated number still follows it, as replay's takeSequences does.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): make no count of a made-object member earlier creates held alike

A count only a later create shows starts at the exemplar's value, which is
right only while the exemplar is the one create accepted: creates accepted
after it showed the member unchanged, which a count would have moved. The
merge now refuses such a count, as the top-level sequence check does.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): check PUT defaults against reads, and respell update clocks by serving's rule

A default a PUT learns must also agree with what the read after an earlier or
the current call showed for the member its answer left out, so a call whose
read showed the member kept withholds a later clearing default.

Replay and serving now choose an update clock's stored spelling by one rule,
readSpellsCoarser: the read after it at a coarser precision in the same zone
and separator. A read in another zone or separator is no longer adopted by
replay, which serving could not store, and a create's plain clocks stay as
answered, as serving makes them.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): keep the largest sequence value in replay, and store update clocks by the kept read

Replay now keeps the largest value a sequence took, as serving's commitWrite
does, so a lower sent value no longer lowers the next number.

Serving stores every update's clocks in the spelling of the read the plan
keeps as observation.read; replay now does the same through
readSpelledClocks, instead of adopting each update's own read, so a later
update whose read spells its clock otherwise contradicts the plan. The plain
clock branch of restampedFields, which no caller used any more, is gone.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): keep a left-out default from numbering, a read's sub-millisecond digits, and references named by member

A member a call left out and the application stored a fixed value for is no
longer turned into a numbered sequence by a later call that left it out.
Serving shortens an update's clock to the read's precision from the answer's
own digits, keeping microseconds. A PUT that names a reference field's entity
by its member supplies that field, for the default check and in serving.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): count a reference a later PUT names toward the default an earlier one cleared

The default check now reads the reference rules the current call shows as
well as the plan's, so a PUT naming the owner by ownerId after one that
cleared it is served, as in the other order.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): number a left-out member past every sent value, and never fill it with a sent blank

A later create the application numbered at or below a value an earlier create
sent no longer turns the echo into a numbered sequence, since serving counts
past every sent value. A member a numbered sequence covers is not filled with
an empty value another call sent when a call leaves it out.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(environment): count sent values in the sequence's scope before numbering a left-out member

The largest value earlier creates sent is kept by type and by collection, and
a collection-scoped sequence adopted later checks only its own collection, so
a number sent in another collection no longer withholds a create numbered
within its own.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-10-29 16:14:05 +08:00

500 lines
19 KiB
TypeScript

import { createHash } from "node:crypto";
import type { JsonValue } from "../jsonValue.js";
import { readRecordedDateTime } from "../offsetDateTime.js";
import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js";
/** How a recorded clock value is spelled: ISO-8601 with its separator, fraction digits and zone, or an epoch number. */
export type ClockFormat =
| {
readonly kind: "iso";
readonly separator: "T" | " ";
readonly fraction: number;
readonly zone: string;
/** Fraction digits end at the last non-zero one, and the point goes with them, as Go spells nanoseconds; `fraction` is the most shown. */
readonly trimmed?: true;
}
| { readonly kind: "epoch"; readonly unit: "s" | "ms" };
export interface ClockReading {
readonly ms: number;
/** Nanoseconds after the floored millisecond, so distinct recorded instants stay distinct. */
readonly subMillisecond: number;
readonly format: ClockFormat;
/** The fraction ends in a zero, which a trimmed spelling never shows, so its width is fixed. */
readonly fixedWidth: boolean;
}
const MINUTE_MS = 60_000;
/** Minutes east of UTC that a zone suffix names; none and `Z` are UTC. */
const zoneMinutes = (zone: string): number | undefined => {
if (zone === "" || zone === "Z") return 0;
if (!/^[+-]\d{2}:?\d{2}$/u.test(zone)) return undefined;
const digits = zone.slice(1).replace(":", "");
const hours = Number(digits.slice(0, 2));
const remainder = Number(digits.slice(2, 4));
if (hours > 23 || remainder > 59) return undefined;
const minutes = hours * 60 + remainder;
return zone.startsWith("-") ? -minutes : minutes;
};
/** Whether a stored ISO clock format has a supported RFC 3339 offset suffix. */
export const isClockZone = (zone: string): boolean =>
zoneMinutes(zone) !== undefined;
/** Minutes east of UTC a stored format's zone names. */
export const zoneMinutesForFormat = (zone: string): number => {
const minutes = zoneMinutes(zone);
if (minutes === undefined)
throw new TypeError(`Invalid clock zone in a derived format: ${zone}`);
return minutes;
};
/** The instant of an ISO-8601 date-time and how it is spelled, or undefined when it is not one. */
const readIsoClock = (value: string): ClockReading | undefined => {
const reading = readRecordedDateTime(value);
if (
reading === undefined ||
!reading.spelling.secondsPresent ||
reading.spelling.fractionDigits > 9
)
return undefined;
const { instant, spelling } = reading;
return {
ms:
instant.epochSecond * 1000 + Number(`${instant.fraction}000`.slice(0, 3)),
subMillisecond: Number(instant.fraction.padEnd(9, "0").slice(3)),
format: {
kind: "iso",
separator: spelling.separator,
fraction: spelling.fractionDigits,
zone: spelling.zone,
},
// The instant's fraction drops trailing zeros, so a longer spelling ended in one.
fixedWidth: spelling.fractionDigits > instant.fraction.length,
};
};
/** The instant a clock value names and its format: an ISO-8601 date-time, or epoch seconds or milliseconds. */
export const readClock = (
value: JsonValue | undefined,
): ClockReading | undefined => {
if (typeof value === "string") return readIsoClock(value);
if (typeof value !== "number" || !Number.isSafeInteger(value))
return undefined;
if (value >= 1e9 && value < 1e10)
return {
ms: value * 1000,
subMillisecond: 0,
format: { kind: "epoch", unit: "s" },
fixedWidth: true,
};
if (value >= 1e12 && value < 1e13)
return {
ms: value,
subMillisecond: 0,
format: { kind: "epoch", unit: "ms" },
fixedWidth: true,
};
return undefined;
};
const pad = (value: number, width: number): string =>
String(value).padStart(width, "0");
/** An ISO-8601 date-time from the UTC fields of `at`, with fraction digits and a zone suffix. */
const isoText = (
at: Date,
separator: string,
digits: string,
zone: string,
): string | undefined => {
const year = at.getUTCFullYear();
if (!Number.isInteger(year) || year < 0 || year > 9999) return undefined;
return `${pad(year, 4)}-${pad(at.getUTCMonth() + 1, 2)}-${pad(at.getUTCDate(), 2)}${separator}${pad(at.getUTCHours(), 2)}:${pad(at.getUTCMinutes(), 2)}:${pad(at.getUTCSeconds(), 2)}${digits === "" ? "" : `.${digits}`}${zone}`;
};
/** An instant in a recorded format; undefined outside its four-digit ISO year range. Fraction digits past milliseconds are `subMillisecond`'s, else zeros. */
export const formatClock = (
ms: number,
format: ClockFormat,
subMillisecond = 0,
): JsonValue | undefined => {
if (format.kind === "epoch")
return format.unit === "s" ? Math.floor(ms / 1000) : ms;
const at = new Date(ms + zoneMinutesForFormat(format.zone) * MINUTE_MS);
const digits =
`${pad(at.getUTCMilliseconds(), 3)}${pad(subMillisecond, 6)}`.slice(
0,
format.fraction,
);
return isoText(
at,
format.separator,
format.trimmed === true ? digits.replace(/0+$/u, "") : digits,
format.zone,
);
};
/**
* A clock value spelled in another ISO format: its wall clock in the format's zone, under the format's separator, with
* the value's own fraction digits padded to the format's width, so no digit is lost; an epoch format is computed from
* the instant.
*/
export const respellClock = (
value: JsonValue,
format: ClockFormat,
): JsonValue => {
const clock = readClock(value);
if (clock === undefined) return value;
if (format.kind === "epoch") return formatClock(clock.ms, format) ?? value;
const millis = ((clock.ms % 1000) + 1000) % 1000;
// An epoch value's fraction is its milliseconds, without trailing zeros.
const digits =
clock.format.kind === "epoch"
? pad(millis, 3).replace(/0+$/u, "")
: `${pad(millis, 3)}${pad(clock.subMillisecond, 6)}`.slice(
0,
clock.format.fraction,
);
const at = new Date(
clock.ms - millis + zoneMinutesForFormat(format.zone) * MINUTE_MS,
);
return (
isoText(
at,
format.separator,
format.trimmed === true
? digits.replace(/0+$/u, "")
: digits.padEnd(format.fraction, "0"),
format.zone,
) ?? value
);
};
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/iu;
export const isUuid = (value: JsonValue | undefined): value is string =>
typeof value === "string" && UUID.test(value);
/** Digits a generated integer shows at least, as a time-based id does; counts and codes are shorter. */
const GENERATED_INTEGER_DIGITS = 10;
/** A large integer's digits, from a safe JSON number or a string of digits without a leading zero. */
export const generatedIntegerDigits = (
value: JsonValue | undefined,
): string | undefined => {
const digits =
typeof value === "number" && Number.isSafeInteger(value) && value > 0
? String(value)
: typeof value === "string" && /^[1-9]\d*$/u.test(value)
? value
: undefined;
return digits !== undefined &&
digits.length >= GENERATED_INTEGER_DIGITS &&
digits.length <= 20
? digits
: undefined;
};
/** A sent array of strings as an application stores it sorted: in code-point order, when that differs from the sent order and shows two distinct strings. */
export const sortedStrings = (
sent: JsonValue | undefined,
): string[] | undefined => {
if (
!Array.isArray(sent) ||
!sent.every((item) => typeof item === "string") ||
new Set(sent).size < 2
)
return undefined;
const sorted = (sent as string[]).toSorted(compareUnicodeCodePoints);
return sorted.some((item, index) => item !== sent[index])
? sorted
: undefined;
};
/** The value an echo rule stores for a sent value: as sent, or its strings sorted. */
export const echoed = (
rule: { readonly sorted?: true },
sent: JsonValue,
): JsonValue => (rule.sorted === true ? (sortedStrings(sent) ?? sent) : sent);
/** Whether a generated integer retains the exemplar's digit count and JSON kind. */
export const likeGeneratedInteger = (
value: JsonValue | undefined,
exemplar: JsonValue | undefined,
): boolean => {
const digits = generatedIntegerDigits(value);
return (
digits !== undefined &&
typeof value === typeof exemplar &&
digits.length === generatedIntegerDigits(exemplar)?.length
);
};
/** An integer value as a number, from a JSON number or a string of digits. */
export const integerOf = (value: JsonValue | undefined): number | undefined =>
typeof value === "number" && Number.isSafeInteger(value)
? value
: typeof value === "string" && /^\d{1,15}$/u.test(value)
? Number(value)
: undefined;
/**
* The next value of a sequence. With nothing taken yet, that is the next held integer, or 1 when
* the scope holds none. Otherwise it is one past the later of that and the last value taken.
* A missing side leaves a negative value negative.
*/
export const nextSequence = (
held: number | undefined,
taken: number | undefined,
): number =>
taken === undefined ? (held ?? 1) : Math.max(held ?? taken + 1, taken + 1);
/** The next held integer of a sequence that never goes below its `start`. */
export const startedSequence = (
held: number | undefined,
start: number | undefined,
): number | undefined =>
start === undefined ? held : Math.max(held ?? start, start);
/** An integer spelled as the recorded value spells integers: a number, or a string of digits. */
export const spelledLike = (exemplar: JsonValue, value: number): JsonValue =>
maskedCount(exemplar) !== undefined
? fillMask(exemplar as string, value)
: typeof exemplar === "string"
? String(value)
: value;
const MASKED_DIGITS = /^\d{2,}(?:-\d{2,})+$/u;
/** The count `-`-separated digit groups of one width spell, such as an asset number `000-005`; dates, versions and addresses are not counts. */
const maskedCount = (value: JsonValue | undefined): number | undefined => {
if (typeof value !== "string" || !MASKED_DIGITS.test(value)) return undefined;
const groups = value.split("-");
if (groups.some((group) => group.length !== groups[0]!.length))
return undefined;
const digits = groups.join("");
return digits.length <= 15 ? Number(digits) : undefined;
};
/** A count that outgrew the exemplar's mask, read only when it is spelled as serving spells it, so dates never match. */
const outgrownMask = (
value: JsonValue | undefined,
exemplar: JsonValue | undefined,
): number | undefined => {
if (typeof value !== "string" || maskedCount(exemplar) === undefined)
return undefined;
const digits = value.replaceAll("-", "");
if (!/^\d{1,15}$/u.test(digits)) return undefined;
const count = Number(digits);
return fillMask(exemplar as string, count) === value ? count : undefined;
};
/** A sequence's number: an integer, digit groups that count, or with the sequence's exemplar a count that outgrew its mask. */
export const sequenceNumber = (
value: JsonValue | undefined,
exemplar?: JsonValue,
): number | undefined =>
integerOf(value) ?? maskedCount(value) ?? outgrownMask(value, exemplar);
/** A number written into a mask's digit positions, zero-padded; digits that outgrow the mask lead it. */
const fillMask = (mask: string, value: number): string => {
const padded = String(value).padStart(mask.replace(/\D/gu, "").length, "0");
let next = padded.length - 1;
let out = "";
for (let at = mask.length - 1; at >= 0; at -= 1) {
const character = mask[at]!;
if (/\d/u.test(character)) {
out = padded[next]! + out;
next -= 1;
} else out = character + out;
}
return padded.slice(0, next + 1) + out;
};
/** Lower case, with every run of other characters joined by one `-` and none at either end. */
/** How a slug spells `&`: dropped when absent, as Django does, or as the word, as Go's gosimple/slug and JavaScript's slugify do. */
export type Ampersand = "and";
export const slugOf = (text: string, ampersand?: Ampersand): string =>
(ampersand === "and" ? text.replaceAll("&", " and ") : text)
.normalize("NFKD")
.replace(/[̀-ͯ]/gu, "")
.toLowerCase()
.replace(/[^a-z0-9]+/gu, "-")
.replace(/^-+|-+$/gu, "");
/** A text's slugs, each with the spelling that gives it: also with `&` as the word when the text holds one. */
export const slugSpellings = (
text: string,
): readonly { readonly slug: string; readonly ampersand?: Ampersand }[] =>
text.includes("&")
? [{ slug: slugOf(text) }, { slug: slugOf(text, "and"), ampersand: "and" }]
: [{ slug: slugOf(text) }];
const isAlphanumeric = (character: string | undefined): boolean =>
character !== undefined && /[\p{L}\p{N}]/u.test(character);
/** Source offsets stay UTF-16, while delimiter classification reads complete code points. */
const previousCharacter = (text: string, at: number): string | undefined => {
if (at === 0) return undefined;
const last = text.charCodeAt(at - 1);
if (last >= 0xdc00 && last <= 0xdfff && at > 1) {
const first = text.charCodeAt(at - 2);
if (first >= 0xd800 && first <= 0xdbff) return text.slice(at - 2, at);
}
return text[at - 1];
};
const nextCharacter = (text: string, at: number): string | undefined => {
const point = text.codePointAt(at);
return point === undefined ? undefined : String.fromCodePoint(point);
};
/** Whether the character before UTF-16 offset `at` is a letter or digit, joining text that starts there to its left. */
export const joinsBefore = (text: string, at: number): boolean =>
isAlphanumeric(previousCharacter(text, at));
/** Whether the character at UTF-16 offset `at` is a letter or digit, joining text that ends there to its right. */
export const joinsAfter = (text: string, at: number): boolean =>
isAlphanumeric(nextCharacter(text, at));
/** Start offsets of `token` in `text` where neither neighbour is a letter or digit. */
export const delimitedOffsets = (text: string, token: string): number[] => {
const offsets: number[] = [];
if (token.length === 0) return offsets;
let at = text.indexOf(token);
while (at >= 0) {
const delimited =
!joinsBefore(text, at) && !joinsAfter(text, at + token.length);
if (delimited) offsets.push(at);
// A rejected match may overlap a delimited one, so the search moves on by one character.
at = text.indexOf(token, at + (delimited ? token.length : 1));
}
return offsets;
};
/** `text` with every delimited occurrence of `token` replaced. */
export const replaceDelimited = (
text: string,
token: string,
replacement: string,
): string => {
let result = "";
let from = 0;
for (const at of delimitedOffsets(text, token)) {
result += text.slice(from, at) + replacement;
from = at + token.length;
}
return result + text.slice(from);
};
/** Whether any string inside a value holds `token` delimited. */
export const containsDelimited = (value: JsonValue, token: string): boolean => {
const pending: JsonValue[] = [value];
for (let next = pending.pop(); next !== undefined; next = pending.pop()) {
if (typeof next === "string") {
if (delimitedOffsets(next, token).length > 0) return true;
} else if (typeof next === "object" && next !== null)
// One at a time: spreading a wide array into push overflows the call stack.
for (const child of Array.isArray(next) ? next : Object.values(next))
pending.push(child);
}
return false;
};
/** Whether any number inside a value spells `digits`. */
export const containsNumber = (value: JsonValue, digits: string): boolean => {
const pending: JsonValue[] = [value];
for (let next = pending.pop(); next !== undefined; next = pending.pop()) {
if (typeof next === "number") {
if (String(next) === digits) return true;
} else if (typeof next === "object" && next !== null)
for (const child of Array.isArray(next) ? next : Object.values(next))
pending.push(child);
}
return false;
};
/** SHA-256 bytes of a seed, extended by counter blocks to the requested length. */
const seededBytes = (seed: string, length: number): Uint8Array => {
const bytes = new Uint8Array(length);
for (let block = 0; block * 32 < length; block += 1) {
const digest = createHash("sha256")
.update(`${seed}#${String(block)}`)
.digest();
bytes.set(
digest.subarray(0, Math.min(32, length - block * 32)),
block * 32,
);
}
return bytes;
};
/** A version 4 UUID derived from a seed, in the recorded value's letter case. */
export const seededUuid = (seed: string, exemplar: string): string => {
const bytes = seededBytes(seed, 16);
bytes[6] = (bytes[6]! & 0x0f) | 0x40;
bytes[8] = (bytes[8]! & 0x3f) | 0x80;
const hex = Buffer.from(bytes).toString("hex");
const uuid = `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20)}`;
return exemplar === exemplar.toUpperCase() && /[A-F]/u.test(exemplar)
? uuid.toUpperCase()
: uuid;
};
const DIGITS = "0123456789";
const LOWER = "abcdefghijklmnopqrstuvwxyz";
const UPPER = LOWER.toUpperCase();
/** The characters a token of this spelling draws from: hexadecimal, or the digit and letter cases it uses. */
const tokenAlphabet = (exemplar: string): string => {
if (/^[0-9a-f]+$/u.test(exemplar) && /[0-9]/u.test(exemplar))
return `${DIGITS}abcdef`;
if (/^[0-9A-F]+$/u.test(exemplar) && /[0-9]/u.test(exemplar))
return `${DIGITS}ABCDEF`;
const alphabet =
(/[0-9]/u.test(exemplar) ? DIGITS : "") +
(/[a-z]/u.test(exemplar) ? LOWER : "") +
(/[A-Z]/u.test(exemplar) ? UPPER : "");
return alphabet === "" ? LOWER : alphabet;
};
/** A seeded value spelled like the recorded one: a UUID for a UUID or when asked for one, else a token. */
export const seededLike = (
seed: string,
exemplar: JsonValue,
uuid = false,
): string =>
uuid || isUuid(exemplar)
? seededUuid(seed, String(exemplar))
: seededToken(seed, String(exemplar));
/** A seeded integer with as many digits as the recorded one and none leading zero, as a number when it is one and stays safe. */
export const seededInteger = (seed: string, exemplar: JsonValue): JsonValue => {
const length = String(exemplar).length;
const bytes = seededBytes(seed, length);
// A number keeps under the largest safe integer, which a sixteen-digit one starting with 9 can pass.
const leading = typeof exemplar === "number" && length >= 16 ? 8 : 9;
const digits = [...bytes]
.map((byte, index) =>
String(index === 0 ? 1 + (byte % leading) : byte % 10),
)
.join("");
return typeof exemplar === "number" ? Number(digits) : digits;
};
/** A token as long as the recorded one, from its alphabet, keeping its other characters in place. */
export const seededToken = (seed: string, exemplar: string): string => {
const alphabet = tokenAlphabet(exemplar);
const bytes = seededBytes(seed, exemplar.length);
return [...exemplar]
.map((character, index) =>
/[0-9A-Za-z]/u.test(character)
? alphabet[bytes[index]! % alphabet.length]!
: character,
)
.join("");
};