# Migration playbook: the server-centric document rewrite > **Superseded by `docs/single-author-playbook.md` (2026-08-28).** Kept as history. Implements `docs/collab-architecture.md` as pivoted on 2026-08-13 (see *The pivot* below). This file is the durable artifact for work that spans multiple sessions — resume by reading it, not by reconstructing context. The phase checklist at the end carries live status. --- ## The pivot: server-centric, not local-first (2026-08-13) The application moves from a local-first design (concurrent edits merged via causal metadata carried in the data) to a **server-centric** one: a central server owns the document, sequences every edit by arrival, and resolves concurrency by that order. Clients are edit surfaces — they render an optimistic prediction of server state, seal edits into commits, submit them, and reconcile against the server's replies. This is the "server-sequenced / hosted" branch of the two-orderings analysis in `docs/doc-ng-design-notes.md`, now chosen rather than parked. What this **deletes** (the causal machinery): - `Lamport`, the `Clock`, and the clock-resume invariant — the server's rev counter is the clock. - `ActorId` and `Stamp { lamport, actor }` — `WriteOrder`'s first component becomes the server-assigned `Rev`. - `CommitHash`, content addressing, parent sets, bytes-as-identity — a linear log needs no identity beyond `Rev`. - The DAG (heads, linearization, causal buffering) and the whole merge/repair/`Collision` layer — one apply point, validated at ingress. - The Loro/automerge build-vs-adopt spike — the three-way gate closed on the third option. - Offline editing. No server, no editing (a later "local mode" embeds the server in-process; the authority model is unchanged). What this **keeps** (decided with the pivot, 2026-08-13): - **LWW registers**, exactly as implemented — last write in server order wins. ("FWW" in the pivot discussion was a slip; first-write-wins with server-side rejection was considered and declined for v1.) - The **total `WriteOrder`** — now `{ rev: Rev, seq: Seq }` — and the self-contained register join it enables. - **Commits** as the sealed unit of edit, undo, and review, with total init structs and the compound-operation rule. A same-day follow-up decision slims the updates: **ops carry only the new value — `old` never crosses the wire**; `Swap` leaves the vocabularies and undo baselines are journal-captured in the client. - The typed per-kind model, the fold, `DocumentCache`, the derived draw-order policy, `FracVal` determinism, the versioned envelope. - The **server folds and validates**: it maintains the folded `Document` via the same shared fold clients run, and refuses invalid commits before sequencing them. Global invariants are ingress preconditions, not merge repairs. - Transport is **WebSocket** (axum), **CBOR frames** (decided 2026-08-16; JSON originally). Server storage is **SQLite** (rusqlite), one `(rev, wall_time, payload)` table — the payload column's format is still open, see the spec's §11. - **Crate split**: a Cargo workspace with a shared document crate (model + fold + commits + protocol), a server binary crate, and the egui client. The compiler enforces the ownership line (server can't see egui; client can't see rusqlite). - The **client owns undo** — inversion of its own commits, submitted as ordinary new commits. The earlier Layer A / Layer B framing (2026-08-11) is retired: Layer A proceeds as before (it was invariant across the choice, which is why it was safe to build first), and Layer B is no longer a deferred CRDT decision — it is the server, it is small, and it is **in scope now**. **Exit criterion for this playbook: two native clients connected to one server, live-editing the same document concurrently.** egui/wasm client support is explicitly deferred (the client networking choice, `ewebsock`, keeps that door open). --- ## The model collapse: `doc_ng` survives, `src/log` donates (2026-08-11, restated under the pivot) Two document models exist side by side — `src/log/` (phase 1: behavior and tests over a generic register-bag state, built for the causal design) and `src/doc_ng/` (the typed per-entity-kind model). Exactly one survives: **`doc_ng`'s shape survives; `src/log` is the organ donor.** The port is a bounded transplant executed **manually** (the user's explicit choice, to own every decision in the merged module) — the step-by-step guide is `docs/doc-ng-port-playbook.md`, rewritten for the pivot. The pivot *shrinks* the transplant. Still ported from `src/log`: - the **fold** skeleton and its tests (retyped per kind); - the **seal discipline** of the builder (consumed on seal; empty seals to `None`), now `CommitBuilder` — deferred until sealing has a caller (see port step 2 status); - the **encode/refusal test patterns** (retargeted: decode-forever, not byte identity); - the **proptest suite's structure and discipline** (mutation-checked properties, semantic oracles, shared fixtures) — reshaped into the reconciliation suite. No longer ported (deleted with the causal design): the `Clock` and its resume fix, `dag.rs` and all eight DAG tests, causal-subset delivery, the convergence-under-arbitrary-delivery properties, `Change`'s parents/hash machinery. **Exit criterion: the day the reconciliation suite is green, `src/log/` is deleted in the same commit, and `src/doc_ng/` becomes the shared document crate.** Time-box the transplant; if it starts growing new design, stop and re-read this section. **Met 2026-08-16.** The suite went green and `src/log/` was deleted in the same commit, as designed — there was never a moment with two models and no pressure to finish. `src/doc_ng/` becoming the shared crate is phase 3's first commit, deliberately not this one: one import-churning move, not two. --- ## Context Blockworx stores a diagram as a flat in-memory `Document`, persists it as hand-editable KDL, and implements undo as whole-state snapshots (`egui::util::undoer::Undoer`, `src/app.rs:367`). That design cannot reach the goal — a document with first-class, attributable, invertible edits, concurrently editable through a server. Three properties block it, and none is fixable in place: 1. **Identity is positional.** Ids are `usize` newtypes minted `max + 1` per map (`src/store.rs:50`); only `RectId` and `PinId` survive a reload — the rest are minted fresh on load in stored order (`src/document/change.rs:13-19`). Two clients inserting concurrently mint the *same* id for different things. 2. **Edits are not first-class.** The unit of history is a full document clone on a one-second quiescence timer. There is nothing to send to a server, nothing to sequence, nothing to invert. 3. **Document state and derived state are interleaved.** `AutoRoute` holds authored data *and* solver output in one struct (`src/document/auto_route.rs:36-54`); `PinPort.accents` and `TextBox.size` are caches living inside the undo state. Solver output cannot be sequenced and must not be logged. ### "Hand-editable" becomes "scriptable" KDL existed so a human could open the file and edit it. That role is retired: authoring moves to a small scripting language whose calls lower to the same primitive ops the editor emits. **That language is a later phase.** Its only claim on this plan is a constraint the design already satisfies: every op is expressible as an ordinary function call returning an element id. --- ## How this work is carried out - **Branch:** the migration mainline is `collab-document-log`. One branch. Nothing lands on `main` until the two-client demo works and `cargo xtask ci` is green. - **Per-session protocol:** read this playbook → pick the first unchecked phase → do it → run `cargo xtask ci` → tick the box and record any decision made, in the same commit → commit `todo.md` alongside, per `CLAUDE.md`. - The transplant (phase 2) is executed **manually by the user**, one port-playbook step per commit. Sessions assist; they do not bulk-execute it. - Deviations from this plan get written *into* this file as they are decided, with the reason. Deviations from the *spec* are additionally recorded in `docs/collab-architecture.md`. ## Decisions taken | Question | Decision | |---|---| | Topology | **Server-centric** (2026-08-13). Server owns log + folded state; clients own edit surface, undo, optimistic prediction. Local mode later = in-process server. "Offline editing dropped" is narrowed by the connection-loss row below (2026-08-25): a session that loses its socket keeps its queue and flushes it on reconnect. What stays dropped is offline *durability* — a queue that survives quitting. | | Ordering | **Server arrival order.** `Rev(u64)` per accepted commit, contiguous from 1; `WriteOrder { rev, seq }` keys every register. No clocks, no actors, no hashes. | | Register semantics | **LWW in server order** — last commit to reach the server wins; nothing rejected on conflict. | | Server role | **Folds and validates.** Shared `fold` + `validate` run at ingress; invalid commits are refused, never sequenced. | | Transport | **WebSocket**, JSON text frames. `Submit`/`Welcome`/`Committed`/`Rejected`/`Apply` per the spec §7. | | Server storage | **SQLite via rusqlite**, server-side only: `commits (rev INTEGER PRIMARY KEY, wall_time, payload TEXT)`. Missing file ⇒ new empty document. Clients persist nothing. | | Crate layout | Workspace members: `crates/doc` (package `blockworx-doc`: model, fold, commits, protocol — no egui, no I/O, no tokio), `crates/server` (package `blockworx-server`: axum + rusqlite), root package `blockworx` (egui client). Names adjustable at extraction time. | | Client networking | `ewebsock` (egui-ecosystem WS client; non-blocking, wasm-capable) on a background feed into the update loop. Fallback if it disappoints: tungstenite on a thread + channels. | | Document model | **`doc_ng`'s typed per-kind shape survives**; `src/log`'s fold/builder/test assets port onto it; `src/log` is then deleted. | | Nomenclature | **`Commit` stands** (2026-08-14, reversing the 2026-08-13 `ChangeSet` rename before any code carried it — step 2 was implemented with `Commit`, and the git-shaped connotation lost its force once parents and hashes were gone). Payload `Commit`, envelope `CommitEnvelope::CommitV1`, in `commit.rs`; the builder, when sealing lands, is `CommitBuilder`. | | Encoding | serde + JSON in the versioned enum envelope (`CommitEnvelope::CommitV1`). Goldens pin **decode compatibility** (old payloads decode forever), not byte identity — nothing is hashed anymore. Determinism (`FracVal`, ordered collections) stays mandatory: server fold and client fold must agree exactly. | | Element granularity | Every element is first-class with a Uuid id and owner/parent register (block icons excepted: an icon is an atomic value on its block). | | Identity | Per-kind id newtypes (`Id`, …); `Id::NULL` is the implicit document root. `ActorId` deleted. | | Ordering / draw order | Not document state. Overlap-capable kinds draw in chronological order of last modification — max `WriteOrder` over an entity's registers, ties longest-route-first then id — within the fixed cross-kind layer rank. The routing solver iterates in id order. | | Undo | Client-owned, **journal-based** (2026-08-13, user decision): update ops carry only the new value — `old` never crosses the wire; the session captures displaced values at seal time and undo submits the stored inverse as an ordinary commit. Figma round-trip invariant adopted as a property when undo lands. | | Legacy documents | **Hard break.** No importer. Fixtures, tutorial levels and goldens are regenerated. | | Boot without `--connect` | **An in-process host** (D5, ratified 2026-08-17). The editor always holds a session, so there is one session code path rather than a live one and a serverless one; nothing is persisted and the title says so. Not the deferred "local mode" — refusing to boot was the alternative. Its converse, 2026-08-25: booting *with* `--connect` opens no file at all, since the document comes from the server's `Welcome`. | | Artwork storage | **Assets ride the log** (D7, executed at the editor swap's step 5): content-addressed, create-only payload ops. Fat but simple — no second channel, and a replica holding every reference but not the bytes has not folded the same log. GC deferred; the size bound is the row below. | | Losing the connection | **Keep the queue and resume** (2026-08-25, user decision, superseding a read-only recommendation). A dropped socket is not a reason to stop editing: the client is already an optimistic replica with a queue, and the fold tolerates a stale one. Redial with backoff, ask the server to `Resume { from: Rev }`, fold the `Catchup` tail, re-ship the pending queue, and say in the title how many edits are waiting. Bounded to what fits in memory — a network drop, not a quit. | | Asset payload size | **4 MB** (2026-08-25, user decision; open since phase 6's step 10e). Assets ride the log, so an unbounded payload is replayed on every server start and shipped to every client forever. `ASSET_LIMIT` is enforced by the fold, so no replica can hold one and a client refuses before it sends. | | wasm client | Deferred to a later phase. Native clients only for the demo. | --- ## What exists today ### Two log modules (until phase 2 collapses them) | Module | Contents | |---|---| | ~~`src/log/`~~ | Phase-1 causal engine (donor). **Deleted 2026-08-16** by the port playbook's step 12; it lives in git history, and the donor citations throughout these playbooks resolve there. | | `src/doc_ng/` | The surviving model, types only (no fold yet): `Register` keyed by `WriteOrder { rev, seq }` (port step 1 done 2026-08-14: `stamp.rs`/`lamport.rs`/`ActorId` deleted, `rev.rs` added), typed entities in per-kind maps, `DocumentCache`, per-kind update/init vocabularies (plain new-value payloads; `Swap`/`Invert` deleted in port step 3, 2026-08-14), `Crud`/`OpCodes`, per-kind `Id`, `FracVal`. Modules: `register`, `document`, `block_model`, `geometry`, `values`, `opcode`, `operands`, `commit`, `rev`, `write_order`, `id`, `hash`. (`paths.rs` deleted 2026-08-14 — no in-commit coalescing, so the register catalog had no consumer.) | ### "KDL" is three separable things The single most important scoping fact for the editor-swap and demolition phases. Deleting the KDL *format* is not deleting the KDL *code*: 1. **The document text format** — `schema/{model,decode,encode,json}.rs`, `document/schema_convert.rs`, `docs/kdl-format.md`. **This goes.** 2. **A general hand-rolled KDL v2 parser** — `schema/kdl.rs` + `schema/kdl/imp.rs` (677 lines, span-carrying). It is the config parser for **tutorial level files** (`src/tutorial/level.rs:71`) and the **script step grammar** (`src/script/parse.rs:8,343`). **It survives**, re-homed to `src/kdl/`, on blast-radius grounds: deleting it means rewriting the script and tutorial layers (3,687 lines) in the same commit as the document swap. Its lifetime is bounded — the scripting language subsumes the step grammar eventually. 3. **The flat `schema::model` projection** — the read-only query surface for cues (`tutorial/cues.rs:26`), script targets (`script/step.rs:132`), `projected()` consumers, and the clipboard. **Needs a replacement surface**; the new document state serves it directly. ### Persistence (legacy path) `.bwx` is a directory container (`root.kdl`, content-addressed `assets/`, gzipped snapshot `history/`, `sessions.jsonl`, `lock` — see `todo.md`'s persistent-history section). It keeps serving the legacy document path until the editor swap; afterwards the server's SQLite log owns history, and the container machinery is retired with the demolition phase. Asset payloads (content-addressed by `AssetHash`) need a server-side story when images meet the new model — parked in *Open items*. ### Mutation surface Everything flows through `Drawing<'a> { document: &mut Document, path, index }` (`widget/drawing.rs:113`) — ~60 mutators — **but three methods punch straight through to raw fields** and are used from 17 of 27 tool files: `block_mut`, `shape_mut` (+ `.pins_mut`, `.title_mut`), `auto_route_mut`. These are why there is no interception point today, and they are the core of the mechanical work in the editor-swap phase. The complete edit inventory — every mutation, CRUD-classified, with gesture-level and on-disk parameter types and the systemic commit-time side effects — is `docs/document_mutations.md`. It is the checklist the editor swap must cover. ### Gesture model Tools mutate **live** during a drag, but only route geometry; the semantic commit points are the per-tool `DragStopped` arms (`move_block.rs:60`, `resize_block.rs:578`, `move_pin.rs:79`, `move_multi_pin.rs:121`, `multi_select.rs:165`, `edit_route.rs:174,213`, `route_tool.rs:277,280`, `new_block.rs:74,88`, `new_comment.rs:53`, `add_port.rs:59,68`, plus the `commit_*` functions in the rename/retype tools). These become the points where commits are sealed and submitted. The preview/commit distinction already exists as `RoutePass::{Preview, Commit}` (`routing.rs:30-35`). ### Sizes | Area | Lines | Fate | |---|---:|---| | `src/document/` | 3,240 | rewritten as the fold target | | `src/document/schema_convert.rs` | 1,461 | deleted | | `src/schema/` | 2,478 | parser re-homed (677); rest deleted | | `src/storage/` | 2,171 | container/history/compact half retired post-swap; `write_atomically` + `Storage` stay (exports, theme/font editor writes) | | `src/tools/` + `src/widget/` | 23,537 | call sites re-pointed at typed setters / command sink | | `src/script/` + `src/tutorial/` | 3,687 | projection surface re-pointed; goldens regenerated | ### Assets worth keeping - **`Document::generation()`** (`document/model.rs:20-43`) — the cache self-invalidation stamp; the fold bumps it once per applied commit and no cache-invalidation logic is rewritten. - **`widget/hit_target.rs`** — the canvas hit-test z-order, encoded once; the derived draw-order comparator plugs in here so drawing and hit-testing cannot disagree. - **`commands::apply_scripted`** (`tools/commands.rs:587`) — one verb implementation shared by the app dispatcher and the script driver; it becomes a commit emitter. - **`CommandId`** (`tools/commands.rs:52`) — ~40 stable typeable names; these become commit labels. --- ## Target architecture ### Crates and layering ``` crates/doc blockworx-doc pure: model, fold, validate, commits, protocol types. No egui, no I/O, no tokio. crates/server blockworx-server axum + tokio + rusqlite. Owns the log and the folded state. CLI: a SQLite path. (root) blockworx egui client. ClientSession (confirmed / pending / optimistic), undo, tools, render. CLI: a server address. ``` ``` Layer 1 server log (SQLite) Commit log ← the only authoritative representation Layer 2 doc crate Document = fold(log) — held by server and by every client Layer 3 client src/derived/ routes, crossings, accents, text extents — regenerated Layer 4 client src/widget/ spatial index, draw lists — ephemeral ``` ### The shared document crate (pure; no egui, no I/O) | Module | Contents | Provenance | |---|---|---| | `rev.rs`, `write_order.rs` | `Rev`; `Seq`, `WriteOrder { rev, seq }` | exists (port step 1, 2026-08-14) | | `register.rs` | `Register`, `Applied` | exists; final | | `document.rs` | `Document`, `DocumentCache` + index types | exists | | `block_model.rs` | `Live`, namespaces, typed entities | exists | | `entity.rs` | the `Entity` trait: `from_init` + `apply` per kind | exists (port step 4, 2026-08-14) | | `geometry.rs`, `values.rs` | fixed-point geometry; shared value enums | exists | | `opcode.rs` | `Crud`, `OpCodes`; the update/init vocabularies are generated by `entity!` beside their kinds | exists; plain new-value payloads (port step 3 done 2026-08-14); `operands.rs` deleted 2026-08-16 | | `commit.rs` | `Commit` payload + `CommitEnvelope` versioned envelope | step 2 done 2026-08-14; `CommitBuilder` deferred until sealing has a caller | | ~~`fold.rs`~~, ~~`validate.rs`~~ | — | never created (step 5, 2026-08-15): the fold is `Document::try_apply(&self, &Commit) -> Result`, homed in `document.rs` with the private interior it writes, and the ingress preconditions are a commit-end pass inside it | | `session.rs` | `Host` (document + log; `ingest` = one `try_apply`, which validates and mints the rev) and `ClientSession` (confirmed/pending/optimistic), `Nonce`, the undo journal | port step 8 | | `encode.rs` | CBOR round-trip, decode goldens, refusal, decode-boundary bounds | step 9 done 2026-08-16 | | `protocol.rs` | `ClientMsg`/`ServerMsg`, `Nonce` | new, thin | | `reconcile.rs` (tests) | the reconciliation proptest suite | port step 10 | ### Command vocabulary — unchanged by the pivot Per-kind and mirrored; the source of truth is `opcode.rs` with the `entity!` field lists in `block_model.rs`, and the invariants recorded in `docs/doc-ng-design-notes.md`: typing along the entity-kind axis, update enums mirroring register placement 1:1 with one plain new value per variant (no `old` on the wire — undo baselines are journal-captured client-side), total init structs, zero-value principle, delete-wins structural (edits cannot target presence), bundled concrete cascade deletes, the compound-operation rule. ### One writer per host, enforced by the type system Unchanged in shape, now true on both ends of the wire: ``` client: tool → Drawing { doc: &Document, cache, sink: &mut CommitBuilder } ↓ seals at gesture boundary ClientSession::commit(commit) // optimistic apply + Submit ↓ Document::try_apply // the ONLY write path server: ingress → Host::ingest → persist → broadcast └ try_apply: fold a clone, validate at commit end, mint the rev ``` Amended 2026-08-15/16: there is no `fold::apply` and no separate `validate` — the fold is `Document::try_apply(&self, &Commit) -> Result`, which validates at commit end and mints the rev, and an `Err` drops the clone. So the write path is not a `&mut` discipline at all: documents are immutable values, and a refused commit is unrepresentable as a mutation. `Drawing` never mutates. `Document`'s fields are private; `try_apply` is the only write path; the three raw escape hatches (`block_mut`, `shape_mut`, `auto_route_mut`) are deleted and replaced by typed setters that seal ops. Mid-gesture previews live in derived state, never the document. ### What still needs proving 1. **Fold determinism** — same log ⇒ same state, on every host. Guarded structurally (pure fold) and by the suite. 2. **Optimistic convergence** — random interleavings of edits, broadcasts and acks ⇒ every client's optimistic == confirmed == server state. 3. **Prediction reconciliation** — rebuild-on-Apply equals a from-scratch fold of (log + pending). 4. **Cache coherence** — incremental `DocumentCache` == `rebuild()` after every fold step. 5. **Undo round-trip** — `fold(fold(s, c), journal_inverse(c)) == s` on the visible projection; later, the Figma undo/redo round-trip invariant. --- ## Phases Each phase is a shippable commit (or series) with `cargo xtask ci` green. **2 — The transplant (manual, time-boxed).** Port `src/log`'s surviving behavior onto `doc_ng`'s types under the pivoted ordering; delete the donor. The step-by-step guide is `docs/doc-ng-port-playbook.md` (rewritten for the pivot: 12 steps — rev replaces stamp; envelope; wire slim-down + undo journal; entity trait; fold; cache; draw-order hook; Host/ClientSession; encoding; reconciliation suite; comment sweep; demolition of `src/log`). **Exit: suite green ⇒ `git rm -r src/log/` in the same commit.** **3 — The workspace split.** Extract `src/doc_ng` (post-transplant) into `crates/doc` (package `blockworx-doc`); add `protocol.rs` (`ClientMsg`/`ServerMsg`/`Nonce`); the app depends on the crate. Pure mechanical move — no behavior change, no test outcome change. Gate: the doc crate builds with no egui/eframe/tokio in its dependency tree. **4 — The server.** `crates/server` (package `blockworx-server`): axum + tokio + rusqlite. - CLI: `blockworx-server [--listen 127.0.0.1:4000]`. Missing file ⇒ created empty. On start: fold all rows in rev order (decode or fold failure = hard startup error), then accept connections. - One single-writer task owns `(Document, Connection, next Rev, fanout)`; per-connection tasks forward `Submit`s via mpsc and stream broadcasts back. `Welcome` is served from the same task, so no commit can fall between snapshot and subscription. - Ingress: decode → `validate` → fold-apply → `INSERT` → `Committed` to sender, `Apply` to the rest. Any failure ⇒ `Rejected`, nothing sequenced. - Tests: a headless client harness in the doc crate's terms (connect, submit, assert broadcasts); the integration test that is this phase's point — **two headless clients submit interleaved edits and both converge to the server's state**. Plus: restart-the-server-and-refold, reject-then-state-unchanged. **5 — The client session.** In the app: `ClientSession` wired to `ewebsock` (feed pumped in the update loop), `blockworx --connect ws://…` boots from `Welcome` instead of a file. Undo/redo re-pointed at the session's journal (inverse captured at seal → ordinary submit). The editor still *reads* the legacy document at this phase's start — the session rides alongside until phase 6 swaps the read path. (If that scaffolding proves awkward, fold phases 5 and 6 together; record the call here.) **The call, 2026-08-16: split, not folded.** The transport landed in phase 5; undo re-pointing moved into phase 6. Undo must change what the user sees, and until the read path swaps, what they see is the legacy document — so re-pointing it early breaks undo rather than scaffolding it. The rest of phase 5 stands alone and is proven by `examples/collab_smoke.rs` against a real server. **6 — The editor swap (the big one).** The editor's document becomes the doc crate's: ids unify to `Id`, membership derived via `DocumentCache`, derived state moves to `src/derived/`, `Drawing` drops to a read-only borrow plus a commit sink, the raw `&mut` escape hatches are deleted, tools are re-pointed at typed setters, mid-gesture previews move off the document, commits move from the quiescence timer to the `DragStopped` arms — each sealing a commit that is optimistically applied and submitted. The `schema::model` projection consumers (cues, script targets, clipboard) are re-pointed at the new document state. Coverage checklist: every row of `docs/document_mutations.md`. **7 — The demo, demolition, regeneration.** Two native clients against one server, editing concurrently — the exit criterion. Then: delete `schema/{model,decode,encode,json,enums,loc,tests}.rs`, `document/schema_convert.rs`, `document/change.rs`, `storage/history.rs`, `storage/compact.rs`, `src/store.rs`; re-home the KDL parser to `src/kdl/`; regenerate `xtask autogen`, router tests, benches, tutorial levels and goldens. Drop `flate2`/`zip` if nothing still needs them. **8 — Merge to `main`.** **Deferred beyond this playbook:** wasm client, presence/cursors, leases (now trivial server policy), auth, server snapshots + blame index + compaction, the scripting language, asset payload transport (images over the wire), local mode (in-process server). --- ## Intentional behavior changes Enumerated now rather than discovered later (items 1–7 carried from the pre-pivot plan, still true; 8–10 are the pivot's additions): 1. ~~**Undo no longer restores the navigation path** — the path is view state.~~ **Superseded 2026-08-23 (user decision).** The undo stack interleaves view steps with document steps: navigation and selection each get their own step, and a document step carries the view it was made from, so undoing an edit lands where the edit happened. See `src/history.rs`; the editor state cannot live in the session journal, which is headless by construction, so the app owns the ordering and delegates the document half. 2. **Undo granularity** moves from one-second quiescence to explicit gesture boundaries. 3. **Undo inverts this client's last commit**, not the global head — and under concurrency it can displace a collaborator's newer value (accepted; see the spec §10). 4. **Waypoints lose individual identity** (atomic list register). 5. **Z-order stops being document state** — chronological derived policy; undo brings the touched element to the top; "bring to front" as a persistent choice does not exist. 6. **Asset GC changes** — no snapshots to reparse; assets are never collected while the log is complete. 7. **"No accent" becomes `Role::Accent0`.** 8. **Editing requires a server.** The client cannot open or edit a document without a connection. Local single-file editing returns later as an in-process server, not as a parallel code path. 9. **The document lives in the server's SQLite file**, not in a `.bwx` container beside the client. Autosave, the timeline, and crash safety become server properties (the log is append-only and fsynced by SQLite). 10. **A concurrent edit can visibly correct an optimistic one**: when a competing commit reached the server first and wins LWW, the local value flips on ack. Rare (requires racing writes to one register within a round trip) and identical to watching a collaborator win. 11. **`--connect` opens no file.** The document comes from the server's `Welcome`, so a path argument is not read, not named in the title, and not locked. Found by the phase-6 dress rehearsal (`docs/editor-swap-playbook.md`, R1), which also made the arriving document a document swap: the view is re-framed on it rather than left on the boot document's scope and camera. 12. **Losing the connection is "disconnected", not "desynced"** (R2). A reset socket is the transport, not a disagreement about the document; `Status::Failed` now means only the latter, and the transport's reason goes to the console. 13. **"Rip up and autoroute" covers the level you are looking at**, and only that. A second rule reached every wire ending on the block anywhere in the document, including wires drawn a level down that nothing re-lays; it was unreachable from the UI and is deleted (2026-08-25, user decision). 14. **Artwork over 4 MB is refused**, at the pick and again by the fold. Previously unbounded. 15. **Accents and I/O directions are nameable commands** (`accent-3`, `io-output`), so a script can set them; previously reachable only by clicking a popup. They answer to their names without appearing as controls — the picker stays the mouse's route. --- ## Open items to resolve during phases 4–7 - **Tutorial level files embed documents** (`initial { … }` in KDL). With the format gone they need a new embedding — recommendation: a builder API or an opaque commit-log blob emitted by the authoring tool, replaced by a script once the scripting language lands. - **Connections crossing a copied subtree boundary** — keep today's behavior (drop routes with only one endpoint inside). - **Asset payloads over the wire.** `AssetHash` stays content-addressed; the prototype can defer images entirely (reject image ops) or inline payloads in a commit (fat but simple). Decide when phase 6 reaches the image tools. - **Boot UX without a server** — what `blockworx` with no `--connect` does during phases 5–6 (legacy path? refuse?). Decide at phase 5. - **Resuming a dropped connection** (R3, raised by the dress rehearsal; *what* to do is decided above, this is what building it needs). The editor already keeps authoring after the link dies — the gap is that the queue has nowhere to go. Three answers are still owed: - **Rejection must stop being silent.** `ClientSession::rejected` drops the commit and its undo-journal entry without a word. At one round trip that is an acceptable rarity; at a queue of forty it is losing someone's work quietly. The only refusal a stale queued commit can still draw is a block cycle (see below), but that is enough. - **A server whose log was reset** cannot honour `from`. It has to refuse the resume, and the client then faces a full `Welcome` it cannot reconcile a queue against — `deliver` rightly refuses one while `pending` is non-empty. What the user is told, and what happens to the queue, needs deciding rather than discovering. - **`Rejected` carries no way to say which of several is which** beyond the nonce, which is fine, but the editor has no surface for reporting one. Console or in-window is a call to make. Why this is smaller than it sounds, and worth recording so it is not re-derived: **deletion is a tombstone**, not a removal (`Live::delete` writes a `Liveness` register), and validation checks referential integrity against the tombstone-inclusive maps — so the obvious hazard, "you edited something deleted while you were away", is an ordinary LWW race, not a refusal. And a commit is stamped with the rev it is *sequenced* at, not the one it was made at (`apply` reads `doc.rev.minting()`), so flushed work outranks everything that landed while the client was gone instead of losing to it. The one refusal left is a block cycle, which takes two clients moving sibling subtrees into each other at once — see the swap playbook's "Only a race can cycle a block". - **`PinDir`'s meaningful zero** — `Input` today; the editor mints `InOut`. Confirm before the entity trait freezes init defaults (carried from the old parking lot). --- ## Verification Non-negotiable while any engine here is from scratch. The suite and rules live in the spec (§14) and the port playbook (step 10); the summary: - **The reconciliation suite** (ported structure of the convergence suite, new properties): server-order determinism; optimistic convergence under random interleavings; prediction-reconciliation oracle; idempotence; intra-commit seq resolution; undo (journal) round-trip with observability asserted; rejection safety. - **The mutation rule** (hard-won in phase 1): every property gets mutation-checked — break the rule it guards, watch it fail. The historical trap inverts under the pivot: one canonical linearization was a test *bug* in the causal world and is the *design* here, so the properties aim at the client's optimistic layer, where order-sensitivity now lives. - **Semantic oracles**: at least one assertion per merge decision about *which value won*. - **Structural, not tested:** one writer per host — `Document` fields private and its interior immutable, `try_apply` the only write path; one grep for `&mut Document` outside the doc crate. - **Encoding:** decode-goldens covering every op variant of every kind; unknown-variant and future-version refusal; decode validates at the trust boundary (range-checked geometry) — the server ingests payloads from arbitrary clients now, so this boundary is load-bearing, and `cargo-fuzz` targets decode-and-validate. - **Integrated path:** real gestures through the real tools (`script::Session`/`SimDriver`) against a real in-process server, asserting the sequenced log. Tutorial golden-replays stay green after regeneration. - **Regression:** router goldens are behavior-preserving refactor targets — no test outcome may change from re-pointing them at a builder API. - **Gate:** `cargo xtask ci` green at every phase boundary; `todo.md` committed alongside. --- ## Phase checklist Live status. Tick a box in the same commit as the work; record decisions and deviations underneath it. - [x] **0 — Setup.** Branch cut; playbook and spec committed. - [x] **1 — Engine core (`src/log/`, now the donor).** Built and reviewed under the causal design; 66 tests. Decisions that stand and port: liveness resolves by write order; unknown element id is a hard error; commit labels are free text. Decisions retired with the pivot: everything clock/DAG/hash-shaped (see *The pivot*). - [x] **1.5 — Wire-format spike (2026-08-08).** serde + JSON versioned enum envelope; refusal-not-repair decode. Carries forward with the goldens retargeted at decode compatibility. - [x] **1.6 — Mutation inventory + second-generation model (2026-08-10).** `docs/document_mutations.md`; `src/doc_ng/` types. - [x] **1.7 — Re-scope (2026-08-11).** Layer A/B split; model collapse decided (`doc_ng` survives). Superseded by 1.8 but the collapse decision stands. - [x] **1.75 — Total order + nomenclature (2026-08-12).** `WriteOrder { stamp, seq }`; `Commit`/`Crud`/`OpCodes`/`*Update` renames; ordering tests landed. The `stamp` half is unwound by port step 1; `seq` and the register keying survive. - [x] **1.8 — The pivot (2026-08-13).** Server-centric adopted; spec rewritten; this playbook rewritten; port playbook rewritten (14 → 12 steps); Loro/automerge spike closed; Layer A/B framing retired. Decisions: LWW in server order (FWW was a slip), server folds and validates, WebSocket transport, three-crate workspace, `Commit` → `ChangeSet` rename (reversed 2026-08-14 — `Commit` stands; see the Decisions table). Same-day follow-up (user): `old` never crosses the wire — update ops carry only the new value; `Swap`/`Invert` leave the vocabularies; undo baselines are captured into the client's session journal at seal time. - [x] **2 — The transplant. Complete 2026-08-16; `src/log/` deleted.** Manual, per `docs/doc-ng-port-playbook.md`, all twelve steps. **Three days, 2026-08-14 → 08-16.** The time-box was declared but never given a number, so there is nothing to measure against — record one next time. What made it short: the pivot cut 14 steps to 12 and deleted the clock and DAG steps outright, and the per-kind model ended up generated rather than transcribed. What the transplant produced beyond the plan: the `entity!` macro (one field list per kind generates struct, init, update vocabulary, and impls, so the four-shape mirror cannot drift); the head `Rev` moved *inside* `Document`, then its *kind* into the type (`Document` vs `Document`), so a prediction cannot be reported or stored as a log position; CBOR on the wire with decode goldens; and a reconciliation suite whose semantic oracle is independent of the fold. Four findings the mutation rituals bought, each a green test that proved nothing until it was fixed: `ops.reverse()` could be deleted with the whole suite still green (the donor's stated reason for it was wrong for this model — lifecycle *pairs* need it, repeated writes do not); ciborium accepts trailing bytes; inverting the LWW comparison left the convergence proptest passing, exactly the donor suite's original defect reappearing; and probing the encoded bytes found `Id`/`Hash` leaking their `PhantomData` kind tag as a trailing null on every id. - [x] **3 — The workspace split (2026-08-16).** `src/doc_ng/` → `crates/doc` (package `blockworx-doc`, 18 modules); `protocol.rs` added with `ClientMsg`/`ServerMsg`/`Nonce`, closing the parking-lot question of where `Nonce` lives. The app takes a path dependency on the crate and no longer declares the module. Shared dependencies hoisted to `[workspace.dependencies]` so two crates (soon three) cannot drift on a version; members add the features they need, which is how the doc crate takes `uuid` with `serde` but not `v4` — the fold never mints ids, and skipping `v4` keeps uuid's wasm randomness problem out of that tree entirely. The gate is **checked, not trusted**: `cargo xtask ci` grew a `headless` step that reads `blockworx-doc`'s dependency tree and fails on egui, eframe, tokio, wgpu, or winit. Verified by adding egui to the crate and watching it fail. No behavior change: 85 tests pass (the 84 that moved, plus a late-joiner test added because rewiring the reconciliation suite onto the real `ServerMsg` put `Welcome` in reach). - [x] **4 — The server (2026-08-16).** `crates/server` (`blockworx-server`): axum + tokio + rusqlite, ` [--listen ADDR]`. One writer task owns `(Host, Store, fanout)`; sockets reach it through one channel, so rev assignment needs no lock and `Welcome` is served from the task that appends. **Payload is CBOR** (user decision), the same bytes as the wire. The greppability goal is consequently unserved until someone writes a `blockworx log` dump command — recorded in the spec as a follow-up rather than quietly dropped. **Durability before visibility**, which needed a doc-crate change: `Host::ingest` split into `accept`/`publish`, so the server folds, persists, *then* publishes. A storage failure is therefore an ordinary `Rejected` — the client reverts an edit that did not happen — instead of a commit the clients hold and the log does not. `Welcome` carries `Vec`, not `Vec<(Rev, …)>`, deviating from §7: revs are a function of position, so shipping them alongside is redundant data that could disagree with the fold, with no rule for which to believe. The head rev is sent, and folding must reproduce it — that equality is the integrity check. Tests: two clients over real sockets submitting interleaved edits converge byte for byte; a stale client's cycle is refused, sequences nothing, and it re-converges; a late client is welcomed with the log; a server started on an existing log serves its history and keeps appending. Plus store-level restart-and-refold, and hard startup errors for a corrupt row and for a gap. - [x] **5 — The client session (2026-08-16), with undo deferred to 6.** `src/collab.rs`: an `ewebsock` socket and the `ClientSession` it feeds, dialled on the first frame (connecting needs a context to wake the UI with) and drained on every frame after. `blockworx --connect ws://.../ws` folds the server's log and tracks it live; the window title says where it stands. **The call the spec asked for.** The three parts do not separate equally. The transport is genuinely independent and lands here. But *undo re-pointed at the session's journal* cannot: undo has to change what the user sees, the user sees the legacy document, and re-pointing it now would make Ctrl+Z submit a commit against a document nobody is looking at while silently ceasing to undo the one they are. That is a regression wearing scaffolding's clothes, so it moves into phase 6's commit, beside the read-path swap that makes it observable. Likewise `--connect` "boots from `Welcome` instead of a file" is true of the *session* and not yet of the *editor*. Dispatch lives in `ClientSession::deliver`, not in the transport — the server crate's integration tests drive the same method over a different socket, so there is one implementation of the protocol rather than one per transport. `examples/collab_smoke.rs` exercises both halves from the console against a real server, which is the only thing that can until the editor has somewhere to send commits. Deferred with incremental resume: a second `Welcome` on a session with work in flight is refused (`UnexpectedWelcome`) rather than guessing whether to resubmit the queue, and a decode failure or rev gap parks the connection as desynced instead of resyncing. - [x] **6 — The editor swap. Complete 2026-08-25.** Fold target became the editor document; tools emit commits; previews off the document; coverage: every row of `docs/document_mutations.md`. Ran 2026-08-16 to 2026-08-25 against `docs/editor-swap-playbook.md` (13 steps in four stages: complete the model, make the `Drawing` waist real on the legacy side under golden protection, build pure op emitters, then a flag-day series). Planning decisions D1-D8 are recorded there; D5 and D7 have since moved into the Decisions table above, where the rest of the project's standing decisions live. **Step 13's three exit obligations, all met.** The coverage sweep: every one of the inventory's 45 rows has an emitter, a tool wiring, and a test through the path a user takes — the last eighteen closed 2026-08-25, including four that looked blocked on the OS clipboard and two that were only reachable from inside a popup until the write moved out of it. Ratification: Parts A and B decided in full (`docs/ratification-sheet.md`), the two that became work (B3's lock split, B9's text extent) done, and Part C's five engineering items closed. The dress rehearsal: two native clients against one `blockworx-server`, converging under genuine concurrency, which found three defects in the boot-and-link seam — two fixed, one (resume on reconnect) decided and recorded above. - [x] **7 — Demo + round trip + demolition. Complete 2026-08-26.** Two clients live (done 2026-08-25); KDL becomes the durable interchange format — readable and writable forever — while the legacy runtime model, the `.bwx` container machinery, and the JSON codec are deleted. Thirteen steps in five stages: `docs/demolition-playbook.md`. **Revised 2026-08-26 (author's call): the one-way door is repealed.** The KDL reader (`schema::{kdl, decode}` + `lower`) survives for import, the courtesy load, and the levels; a new `raise` projection (folded document → schema structs) feeds the *existing* encoder, so "Export → KDL" closes the round trip with one format authority. The original stage-2 convert-or-lose-it sweep evaporates — nothing becomes unreadable; what remains is a round-trip proof gate and a one-time `.json` → `.kdl` conversion before the JSON codec dies. Rides along (`docs/type-level-invariants.md`, P4): `BlockId::NULL` becomes a `Scope { Root, Block }` in the editor's vocabulary, converted once at the document boundary. **Scheduled last rather than interleaved** (D5 there): the rationale for doing it *inside* the demolition was that `schema_convert` and the legacy readers hold several of the `NULL` conversions — but those are deleted by step 8, not migrated by P4, so converting them first is work on sites about to vanish. P3 (one `Drag` lifecycle for the fourteen tools that hand-roll it) is scheduled by appetite, and wants P1's `Preview` to exist first. **D1 answered (2026-08-26):** `blockworx` with no `--connect` opens as normal — the courtesy load stays, "Import → KDL" lowers a document into the current log, "Export → KDL" writes it back out. Serverless persistence *is* the round trip; local sqlite mode stays deferred. - [x] **8 — Merge to `main`. Complete via pull request, 2026-08-27.** --- ## Phase 1 review findings — disposition after the pivot The three-agent review of `src/log/` produced ~25 findings, re-triaged 2026-08-11 (serde spike / model collapse) and again 2026-08-13 (the pivot). Full text in git history; what remains actionable: **Resolved by the serde spike (landed, still in force):** derive-based encoding (no forgotten fields, no length-prefix amplification, no varint padding); `BTreeSet` parents → now moot entirely (no parents); versioned enum envelope refusal. **Mooted structurally by the model collapse (unchanged):** init structs are total (no smuggled writes); per-kind maps (no kind disagreement); no `PropTag`; no random-uuid `Default` on ids. **Mooted by the pivot (new):** everything clock- and DAG-shaped — `Clock::resuming_at` and the resume invariant, `Dag::key`/`Key`/`linearize` findings, causal-subset delivery, the Lamport-saturation comment. None of it ports; none of it needs fixing. **Carried into the transplant (port-playbook steps in parentheses):** - Decode is a trust boundary — validate, never normalize; range-check `GridRect` extents (step 9; also the server's fuzz target). - Envelope fields private — ops order is semantic; a `pub` field invites mutation of a sealed commit (step 2). - Golden coverage: one instance per op variant per kind — retargeted at decode compatibility (step 9). - Vacuous-test fixes: the undo round-trip property asserts observability and covers Create/Delete/Restore and batches; a semantic oracle asserting which value won (step 10). - One `#[cfg(test)] mod fixtures`, not six copies (step 10). - The session types belong in the module, not the test file — now `Host`/`ClientSession` (step 8). - Comment density: code keeps only what surprises; essays live in docs (step 11). - `Applied` stays returned even though nothing consumes losses yet — the loss path stays testable; a future review UI is its consumer (steps 5, 10). **Retired with their subsystems:** compaction-frontier reconciliation (snapshots are a future server optimization; re-open then); `AssetId::of` duplication (dies with `schema_convert.rs` in phase 7); merge scenario tests (edit-vs-delete is covered structurally + by the suite; cycle repair is now ingress validation — test it in `validate`).