rzen/lanework
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Memoize the reload parse and short-circuit value-equal snapshots

Browse Source 
The loader gains a ParseMemo — the previous walk's parsed documents keyed
by root-relative path, trusted on the git-index heuristic (mtime + size,
no hashing) and passed as an input so the loader stays stateless. A hit
skips exactly one file read; schema, order, coercions, dedupe, and every
directory listing run fresh, so memoized and cold walks are output-
identical (golden-corpus equivalence suite). Entries record only past the
schema gate, so a defect can never be answered from the memo.

The store skips the snapshot assignment wholesale when the fresh model is
value-equal — no @Observable churn, no render pass, no snapshotGeneration
bump — and a new landedReloads counter carries walk-completion for the
three consumers whose subject is the walk, not the snapshot: the card
window's comment thread, the comment search index, and the auto-committer's
covering gate (which now counts a completed walk as covering even when
nothing changed). Warnings and defects move on their own equality; failed
reloads bump neither counter. An injectable ParseCounter makes the
single-file-echo claim a test.

Claude-Session: https://claude.ai/code/session_01CqjXB7ASoWtbyoGod68k97
This commit is contained in:
rzen
2026-08-01 11:38:25 -04:00
parent 5e6417e749
commit 988a7245a3
11 changed files with 1085 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Kanban/Storage/BoardLoader.swift
+229 -12
View File
@@ -1,4 +1,5 @@
import Foundation
import Synchronization
import os
/// Walks a board's folder tree and produces an immutable `BoardModel` snapshot — a pure
@@ -193,6 +194,150 @@ public enum BoardLoader: Sendable {
return GitignoreRules(parsing: text)
}
// MARK: - The parse memo
/// **One `index.md`'s git-index heuristic record** (02-architecture.md § Live-reload resilience,
/// blessed 2026-07-31: "The walk memoizes its parse, never its result").
///
/// Modification date and byte count, and deliberately nothing else: "The mtime+size trust is the
/// git-index heuristic; a writer that defeats it — content changed, mtime and size both
/// preserved — is outside the app's care." No content hashing, because a hash is a read of the
/// whole file and reading the whole file is the cost the memo exists to avoid.
///
/// Stat'd through `FileManager.attributesOfItem`, **never** `URL.resourceValues`, which caches
/// its answers on the `URL` instance: a cached mtime would let the memo answer from a stamp taken
/// a reload ago, which is exactly the staleness the heuristic exists to detect.
public struct FileStamp: Sendable, Equatable {
public let modified: Date
public let size: Int
/// `nil` where the file cannot be stat'd at all — read as "cannot tell", and therefore as a
/// memo miss: the walk parses, exactly as it did before the memo existed.
init?(of url: URL) {
guard let attributes = try? FileManager.default.attributesOfItem(atPath: url.path),
let modified = attributes[.modificationDate] as? Date,
let size = attributes[.size] as? NSNumber
else {
return nil
}
self.modified = modified
self.size = size.intValue
}
}
/// **The previous walk's parsed documents, indexed by root-relative path** — the memo
/// (02-architecture.md § Live-reload resilience, blessed 2026-07-31: "the loader may reuse the
/// previous snapshot's parsed item for any `index.md` whose path, mtime, and size are unchanged
/// — the previous snapshot *is* the memo").
///
/// **An input to `load`, never hidden state.** The loader is stateless statics and stays that
/// way: a caller that holds no memo gets a cold walk, and one that holds the last walk's memo
/// gets the same answer faster. That is the contract stated exactly — "result-purity with cost
/// unspecified: same tree in, same snapshot out, and the memo can only change how fast".
///
/// ### Scope: the parse, and nothing else
///
/// A hit skips one thing — opening and parsing one file. Everything *derived* from the document
/// (`schema`, `order`, the coercion trace, the identity dedupe, the trash's `kind`) is recomputed
/// from it on every walk, unchanged, which is what makes memoized and cold walks indistinguishable
/// in output rather than merely intended to be.
///
/// **Directory enumeration is never memoized**: folder discovery, attachment listings, loose-file
/// detection, the trash's entries and the noise gate are read fresh every walk, "because
/// attachment changes never touch `index.md`" — a memo that covered them would go blind to
/// precisely the changes the snapshot is supposed to show.
///
/// ### A defect can never be answered from it
///
/// An entry is recorded only where the file parsed **and** its `schema` reading succeeded — the
/// two steps that can produce a `BoardLoadError` at all. So a defective `index.md` is never in the
/// memo, which settles both halves of the collect-all walk's question: a file that is broken and
/// stays broken has nothing to hit and is re-read and re-collected every walk, and a file whose
/// defect was repaired moved its mtime and size and would miss anyway. The skip channel inherits
/// this by construction — a skipped path *is* a defect path — so a skip is recomputed from a fresh
/// parse every walk and can never be decided from a memo.
///
/// Withheld duplicate occurrences *are* recorded, and correctly so: their files parsed cleanly and
/// only the board-wide dedupe kept them out of the model, and that dedupe runs over the fresh walk
/// either way.
public struct ParseMemo: Sendable {
fileprivate struct Entry: Sendable {
let stamp: FileStamp
let document: FrontmatterDocument
}
fileprivate var entries: [String: Entry] = [:]
/// The empty memo — a cold walk. The only one a caller ever constructs; every other comes
/// out of a `LoadResult`.
public init() {}
/// How many documents this memo can answer for. The walk never asks; the suites do.
public var count: Int { entries.count }
fileprivate func document(at path: String, stamp: FileStamp) -> FrontmatterDocument? {
guard let entry = entries[path], entry.stamp == stamp else { return nil }
return entry.document
}
fileprivate mutating func record(_ document: FrontmatterDocument, at path: String, stamp: FileStamp?) {
guard let stamp else { return }
entries[path] = Entry(stamp: stamp, document: document)
}
}
/// **What one walk actually read** — the memo's whole claim, made assertable.
///
/// The loader's contract is result-purity with *cost unspecified*, and a cost nothing can observe
/// is a cost nothing can regress: this is the observation handle, so "a single-file echo re-parses
/// one file, not the tree" is a test rather than a hope.
///
/// Injected rather than a static tally, for `IdentityHistoryRanker`'s reason: `load` is stateless
/// statics called from several tasks at once, and a shared counter would be one mutable answer to
/// a per-walk question. `nil` — every production call — costs nothing at all.
public final class ParseCounter: Sendable {
/// One walk's tally: files opened and parsed, and documents answered from the memo.
public struct Counts: Sendable, Equatable {
public var parsed = 0
public var reused = 0
}
private let state = Mutex(Counts())
public init() {}
public var counts: Counts { state.withLock { $0 } }
fileprivate func noteParse() { state.withLock { $0.parsed += 1 } }
fileprivate func noteReuse() { state.withLock { $0.reused += 1 } }
}
/// One `index.md`, read through the memo — the memo's only point of contact with the walk.
///
/// A hit is a document the previous walk parsed out of a file whose path, mtime and size have not
/// moved since; a miss is the ordinary `readDocument(at:path:)`, byte for byte the same call the
/// loader has always made. The stamp travels back out so the caller can record the document into
/// *this* walk's memo once its `schema` reading has succeeded — `ParseMemo` states why that, and
/// not the read, is the recording point.
///
/// A file whose stamp cannot be read (`nil`) is always parsed and never recorded: "cannot tell"
/// reads as "not memoizable", which is the direction that costs a parse rather than correctness.
private static func memoizedDocument(
at url: URL,
path: String,
memo: ParseMemo?,
counter: ParseCounter?
) throws(BoardLoadError) -> (document: FrontmatterDocument, stamp: FileStamp?) {
let stamp = FileStamp(of: url)
if let stamp, let hit = memo?.document(at: path, stamp: stamp) {
counter?.noteReuse()
return (hit, stamp)
}
counter?.noteParse()
return (try readDocument(at: url, path: path), stamp)
}
// MARK: - Entry point
/// Walks the board and answers a snapshot — or **every fail-fast defect the walk found**, as one
@@ -231,10 +376,20 @@ public enum BoardLoader: Sendable {
///
/// **Root paths are unskippable** (`unskippablePaths`) — an entry naming the root's own
/// `index.md` is ignored and the defect collected anyway.
///
/// - Parameter memo: **the previous walk's parsed documents** (`ParseMemo`, blessed 2026-07-31).
/// `nil` — a first load, a template read, a HEAD snapshot — is a cold walk. Passing the last
/// walk's memo cannot change a single thing about the result, only how many files this one
/// opens; see `ParseMemo` for the scope and for why a defect can never be answered from it.
///
/// - Parameter counter: where this walk tallies what it read (`ParseCounter`). `nil` everywhere
/// but the suites.
public static func load(
boardRoot: URL,
skipping: Set<String> = [],
historyRanker: IdentityHistoryRanker? = nil
historyRanker: IdentityHistoryRanker? = nil,
memo: ParseMemo? = nil,
counter: ParseCounter? = nil
) throws(BoardLoadFailure) -> LoadResult {
// Environmental, so immediate: a root that cannot be listed has no walk to collect from.
do throws(BoardLoadError) {
@@ -253,6 +408,11 @@ public enum BoardLoader: Sendable {
// what `BoardLoadFailure` carries when it does not.
var failures: [BoardLoadError] = []
// **This walk's own memo, for the next one** (`ParseMemo`). Built as the walk goes and
// handed out on the `LoadResult`, so the loader keeps no state between calls: what the store
// passes back in is what came out of the walk before it.
var freshMemo = ParseMemo()
/// Records one fail-fast defect — unless this open's user already consented to skipping that
/// exact path.
///
@@ -277,12 +437,16 @@ public enum BoardLoader: Sendable {
let boardIndexURL = boardRoot.appendingPathComponent(indexFileName)
if FileManager.default.fileExists(atPath: boardIndexURL.path) {
do throws(BoardLoadError) {
let document = try readDocument(at: boardIndexURL, path: indexFileName)
let read = try memoizedDocument(
at: boardIndexURL, path: indexFileName, memo: memo, counter: counter)
// **The root's own `schema` stays required** (01-storage-format.md § Malformed input,
// re-ruled 2026-07-31): it is the this-really-is-a-board gate, and the one `schema` on
// the board that does not read as 1 when absent.
boardSchema = try validatedRootSchema(in: document, path: indexFileName)
boardDocument = document
boardSchema = try validatedRootSchema(in: read.document, path: indexFileName)
boardDocument = read.document
// Recorded past the schema gate, never before it — `ParseMemo`'s "a defect can never
// be answered from it".
freshMemo.record(read.document, at: indexFileName, stamp: read.stamp)
} catch {
record(error)
}
@@ -386,12 +550,19 @@ public enum BoardLoader: Sendable {
let lanePath = laneName + "/" + indexFileName
let laneDocument: FrontmatterDocument
let laneSchema: (schema: Int, coerced: CoercedField?)
let laneStamp: FileStamp?
// **A broken lane takes its subtree with it** (the collect-and-skip rule above): the
// defect is recorded, the lane's cards are not enumerated, and the repair's re-check is
// what surfaces whatever they were hiding.
do throws(BoardLoadError) {
laneDocument = try readDocument(
at: laneURL.appendingPathComponent(indexFileName), path: lanePath)
let read = try memoizedDocument(
at: laneURL.appendingPathComponent(indexFileName),
path: lanePath,
memo: memo,
counter: counter
)
laneDocument = read.document
laneStamp = read.stamp
// Below the root both keys are optional (re-ruled 2026-07-31): a missing `schema`
// reads as 1, a missing or unusable `order` as append-at-end. Both readings are
// coerce-tier — recorded here, logged, and acted on by nothing until the file's next
@@ -401,6 +572,7 @@ public enum BoardLoader: Sendable {
record(error)
continue
}
freshMemo.record(laneDocument, at: lanePath, stamp: laneStamp)
let laneOrder = IntegrityRules.resolvedOrder(in: laneDocument)
noteCoercions(
in: laneDocument,
@@ -423,11 +595,14 @@ public enum BoardLoader: Sendable {
let card: WalkedCard
do throws(BoardLoadError) {
card = try parseCard(at: cardURL, path: cardRelPath)
card = try parseCard(at: cardURL, path: cardRelPath, memo: memo, counter: counter)
} catch {
record(error)
continue
}
// `parseCard` returning at all means the parse and the `schema` reading both
// succeeded, which is the recording point one level up spells out longhand.
freshMemo.record(card.document, at: cardRelPath + "/" + indexFileName, stamp: card.stamp)
noteCoercions(in: card.document, at: cardRelPath + "/" + indexFileName, plus: card.coercions)
// **The card-level claimed name** (01-storage-format.md § Fractal layout ▸ Rules,
@@ -539,17 +714,25 @@ public enum BoardLoader: Sendable {
let entryPath = entryRelPath + "/" + indexFileName
let document: FrontmatterDocument
let schema: (schema: Int, coerced: CoercedField?)
let stamp: FileStamp?
// Collected and skipped, the lane arm's rule one container over: a trash entry that will
// not parse leaves the trash rather than refusing the board, and its own subtree was
// never walked to begin with (the entry is opaque by design).
do throws(BoardLoadError) {
document = try readDocument(
at: entryURL.appendingPathComponent(indexFileName), path: entryPath)
let read = try memoizedDocument(
at: entryURL.appendingPathComponent(indexFileName),
path: entryPath,
memo: memo,
counter: counter
)
document = read.document
stamp = read.stamp
schema = try resolvedSchema(in: document, path: entryPath)
} catch {
record(error)
continue
}
freshMemo.record(document, at: entryPath, stamp: stamp)
let order = IntegrityRules.resolvedOrder(in: document)
noteCoercions(
in: document,
@@ -727,6 +910,7 @@ public enum BoardLoader: Sendable {
return LoadResult(
model: model,
warnings: warnings,
memo: freshMemo,
defects: defects,
// Keyed by identity, so a withheld entry's reading has to go with it: two folders sharing
// an id would otherwise leave a `kind` answer standing for the *other* one — the exact
@@ -929,6 +1113,9 @@ public enum BoardLoader: Sendable {
/// This card's coerce-tier records for the strict fields, which only the rulebook can make
/// (a missing key leaves no trace in `document.coercedFields`).
let coercions: [CoercedField]
/// What this card's `index.md` looked like to `stat(2)` as the walk read it — the key the
/// next walk's memo hit is decided by, `nil` where the file could not be stat'd at all.
let stamp: FileStamp?
var title: FieldValue<String> { document.title }
var isDeleted: Bool { !document.deleted.isMissing }
@@ -961,9 +1148,25 @@ public enum BoardLoader: Sendable {
///
/// `path` is root-relative and names the *folder*; the errors this throws name its `index.md`.
/// Callers guard `isUUIDShaped` and `hasIndex` first, exactly as the lane walk always has.
private static func parseCard(at cardURL: URL, path: String) throws(BoardLoadError) -> WalkedCard {
///
/// The **attachment listing stays fresh** here, memo or no memo (`ParseMemo` ▸ Scope): a hit
/// spares this card's `index.md` read and nothing else, because an attachment arriving in
/// `attachments/` never touches `index.md` and a card whose paperclip went stale would be the
/// memo lying about the tree.
private static func parseCard(
at cardURL: URL,
path: String,
memo: ParseMemo?,
counter: ParseCounter?
) throws(BoardLoadError) -> WalkedCard {
let cardPath = path + "/" + indexFileName
let document = try readDocument(at: cardURL.appendingPathComponent(indexFileName), path: cardPath)
let read = try memoizedDocument(
at: cardURL.appendingPathComponent(indexFileName),
path: cardPath,
memo: memo,
counter: counter
)
let document = read.document
let schema = try resolvedSchema(in: document, path: cardPath)
let order = IntegrityRules.resolvedOrder(in: document)
@@ -973,7 +1176,8 @@ public enum BoardLoader: Sendable {
storedOrder: order.order,
attachments: attachmentNames(in: cardURL),
document: document,
coercions: [schema.coerced, order.coerced].compactMap { $0 }
coercions: [schema.coerced, order.coerced].compactMap { $0 },
stamp: read.stamp
)
}
@@ -1331,6 +1535,19 @@ public struct LoadResult: Sendable {
public var model: BoardModel
public var warnings: [LoadWarning]
/// **What this walk parsed, ready to be the next walk's memo** (`BoardLoader.ParseMemo`, blessed
/// 2026-07-31).
///
/// It rides out here rather than being derived from `model` for two reasons. The stamps are not
/// in the snapshot and never will be — mtime and size are facts about files, not about a board —
/// and the documents that *are* in the snapshot would have to be re-indexed by path to be usable,
/// which is the walk's own knowledge being thrown away and re-derived. Carrying both together
/// keeps the loader a pure function whose caller holds the whole of what the next call may reuse.
///
/// A caller that ignores it gets a cold walk every time, which is exactly what
/// `TemplateEngine`, `GitHeadSnapshot` and every first load do.
public var memo: BoardLoader.ParseMemo = BoardLoader.ParseMemo()
/// **The typed defect stream** — everything this walk found that is pending *work*
/// (02-architecture.md ▸ Components ▸ IntegrityRules, settled 2026-07-29). One channel, not
/// three: loose card files, legacy `deleted:` keys, and a claimed board-root name held by the