rzen/lanework
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Dedupe duplicate ids and heal them silently

Browse Source 
The crash-class gap the integrity design pass found (DESIGN/01 -
Fractal layout rules; 02 - Live-reload resilience): the loader had no
board-wide dedupe at all, so two hand-copied folders sharing a UUID put
two equal ItemIDs into one snapshot - which SwiftUI's ForEach does not
tolerate. Built to the day's re-rulings, both landing mid-flight: the
user-gated Repair banner retired (176c852 - the heal runs silently) and
the container boundary became the first tie-break (f153e79 - the
visible card never loses to its own trash ghost).

IntegrityRules.dedupe (pure, occurrence list in, verdict out): group by
canonical identity, collapse case-spelled twins first - spellings with
a live occurrence outrank trash-only spellings, then canonical
lowercase, then lexicographically first; losers are silent strays
(LoadWarning.caseTwinIgnored - spelling artifacts, never reminted) -
then earlier-occurrence-wins across the surviving spelling's folders on
a four-rung ladder: live-before-trashed, git path history rank, FS
birth date (nil is no comparison, never .distantPast), traversal
order. Occurrences are exactly the identity-bearing folders: lanes,
cards, .trash entries - a UUID-shaped folder under a card is content.

BoardLoader walks lanes as WalkedLane and builds Lane values only on
the far side of the verdict, so a withheld card can never reach a
snapshot; a name-only gate keeps the healthy-board cost at one
dictionary pass, no disk reads. Withheld subtrees are still walked - a
hand-copied lane's nested cards are their own withheld occurrences,
reminted at the finest grain like the import boundary would have. A
withheld trash entry's trashKinds reading leaves with it. The git rung
is a seam (BoardLoader.IdentityHistoryRanker, one closure keyed by
root-relative path) because base links no git machinery - base injects
nothing and falls through; pro-m1 owns the ranker (card annotated).

The heal: Defect.duplicateIdentity (signature duplicate:<path>:<id>)
rides HealScheduler as the fourth scheduled heal, ordered last among
the content heals because a remint renames folders and would stale the
paths the same load handed the relocation and migration.
BoardWriter.remintDuplicateIdentity re-verifies twice at write time -
the folder still carries the losing identity AND something else still
does (the vanished-duplicate race no-ops from either side) - then
renames to a fresh v4 minted against the whole board's identity bag.
A rename and nothing else: no index.md opened, no modified stamp, no
modified-by clear; the receipt is heal-marked (pro-m1's committer
splits it out, named by 06's kept Repair verb); no undo step - heals
are not gestures. The notice is the design's own sentence ("Repaired
duplicate id - 'Fix login'"; several fold to a count), a loss row on
the relocation's reasoning; WriteOperation.repairDuplicateID carries
the failure mirror.

Fixture repair rode along: duplicate-order-tie-break.kanban had a lane
and its own card sharing a UUID - a genuine duplicate the new pass
correctly withholds; the folder rename landed in feae6d0, the matching
test constant lands here.

66 tests added (DuplicateIdentityTests: the ladder rung by rung, the
straddles, withheld-lane subtrees, remint idempotence and races, the
one-heal-cycle window, all phrasing). 1804 green on both schemes.

Claude-Session: https://claude.ai/code/session_01SR4XGjmBE16ZUYWpfFHXwY
This commit is contained in:
rzen
2026-07-29 18:19:52 -04:00
parent 68fa503250
commit 0463540aea
8 changed files with 1979 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Kanban/Storage/BoardLoader.swift
+238 -18
View File
@@ -142,7 +142,10 @@ public enum BoardLoader: Sendable {
// MARK: - Entry point
public static func load(boardRoot: URL) throws(BoardLoadError) -> LoadResult {
public static func load(
boardRoot: URL,
historyRanker: IdentityHistoryRanker? = nil
) throws(BoardLoadError) -> LoadResult {
try checkIsReadableDirectory(boardRoot)
let boardIndexURL = boardRoot.appendingPathComponent(indexFileName)
@@ -184,7 +187,11 @@ public enum BoardLoader: Sendable {
warn(.boardLevelDeletedIgnored)
}
var lanes: [Lane] = []
// Every lane the walk read, in folder order — **not** `Lane` values yet. The board-wide
// identity dedupe below decides which folders render at all, and a `Lane` is built only on
// the far side of that decision, because a `Lane` carrying a withheld card would be exactly
// the snapshot the invariant forbids.
var walkedLanes: [WalkedLane] = []
for laneURL in try directoryCandidates(in: boardRoot) {
let laneName = laneURL.lastPathComponent
// The app-claimed board-root names are not strays and must not warn as such. Hidden
@@ -257,21 +264,12 @@ public enum BoardLoader: Sendable {
logger.info("\(laneName, privacy: .public): legacy 'deleted' key — lane to be returned live with the key removed")
}
lanes.append(Lane(
id: ItemID(rawValue: laneName),
walkedLanes.append(WalkedLane(
name: laneName,
schema: laneSchema,
title: laneDocument.title,
created: laneDocument.created,
modified: laneDocument.modified,
modifiedBy: laneDocument.modifiedBy,
deleted: laneDocument.deleted,
background: laneDocument.background,
icon: laneDocument.icon,
iconColor: laneDocument.iconColor,
order: laneOrder,
width: laneDocument.width,
cards: Ranks.sortedForDisplay(cards, order: \.order, name: { $0.id.rawValue }),
document: laneDocument
document: laneDocument,
cards: Ranks.sortedForDisplay(cards, order: \.order, name: { $0.id.rawValue })
))
}
@@ -301,6 +299,40 @@ public enum BoardLoader: Sendable {
trash.append(entry)
}
// **The board-wide identity dedupe** (01-storage-format.md § Fractal layout ▸ Rules:
// "Duplicate ids within a board are never tolerated … the loader keeps exactly one occurrence
// per id, board-wide"). Live lanes *and* `.trash/`, because board-wide means both containers —
// and a snapshot carrying two items with equal ids is the one thing SwiftUI's `ForEach` does
// not tolerate, which is why this is subtractive rather than advisory.
//
// Display order is settled here rather than in the `BoardModel` call below, because the
// dedupe's last tie-break *is* traversal order and the rule needs the occurrences in it.
let orderedLanes = Ranks.sortedForDisplay(walkedLanes, order: \.order, name: \.name)
let orderedTrash = Ranks.sortedForDisplay(trash, order: \.order, name: { $0.id.rawValue })
let verdict = dedupeIdentities(
inBoardAt: boardRoot,
lanes: orderedLanes,
trash: orderedTrash,
historyRanker: historyRanker
)
// Case twins are the **tolerate** tier: a spelling artifact of the item that rendered, with
// nothing to do about it (§ Fractal layout ▸ Rules). They warn, exactly like every other
// skipped stray, and are deliberately absent from `defects`.
for twin in verdict.caseTwins {
warn(.caseTwinIgnored(path: twin.path, winner: twin.winner))
}
// Content duplicates are **work**: withheld here, reminted by the store's scheduled heal
// (re-ruled 2026-07-29 — the loader itself still never writes).
for duplicate in verdict.duplicates {
defects.append(.duplicateIdentity(duplicate))
logger.warning(
"\(duplicate.path, privacy: .public): duplicate id, withheld in favour of \(duplicate.winner, privacy: .public) — to be reminted"
)
}
let withheld = Set(verdict.duplicates.map(\.path) + verdict.caseTwins.map(\.path))
let model = BoardModel(
rootURL: boardRoot,
schema: boardSchema,
@@ -313,8 +345,8 @@ public enum BoardLoader: Sendable {
icon: boardDocument.icon,
iconColor: boardDocument.iconColor,
template: boardDocument.value(for: templateKey),
lanes: Ranks.sortedForDisplay(lanes, order: \.order, name: { $0.id.rawValue }),
trash: Ranks.sortedForDisplay(trash, order: \.order, name: { $0.id.rawValue }),
lanes: orderedLanes.compactMap { $0.rendered(withholding: withheld) },
trash: orderedTrash.filter { !withheld.contains(trashFolderName + "/" + $0.id.rawValue) },
document: boardDocument
)
@@ -322,10 +354,172 @@ public enum BoardLoader: Sendable {
model: model,
warnings: warnings,
defects: defects,
trashKinds: trashKinds
// Keyed by identity, so a withheld entry's reading has to go with it: two folders sharing
// an id would otherwise leave a `kind` answer standing for the *other* one — the exact
// ambiguity the dedupe exists to remove.
trashKinds: trashKinds.filter { !withheld.contains(trashFolderName + "/" + $0.key.rawValue) }
)
}
// MARK: - The board-wide identity dedupe
/// Every identity-bearing folder in the board, deduped — the loader's half of the rule whose
/// *deciding* is `IntegrityRules.dedupe(_:)`.
///
/// **The gate comes first, and it is why this costs nothing on a healthy board**: a pass over the
/// names alone answers "does any identity appear twice at all", and on the overwhelmingly common
/// answer — no — nothing further happens: no `creationDateKey` reads, no history probes, no
/// grouping. The disk reads below are paid for only by a board that actually has a collision.
///
/// **Withheld subtrees are still walked** (settled — the import boundary's finest-grain rule read
/// for the heal): a withheld *lane*'s cards are ordinary depth-2 occurrences and enter the
/// inventory like any other, so a collision nested inside a losing lane is its own withheld
/// occurrence with its own remint. Nothing deeper is an occurrence at all — "level is position",
/// and a UUID-shaped folder under a card is content.
///
/// **The container side is told to the rule, not derived from the path**: this walk knows which
/// container it is in, and the container boundary is the rule's *first* tie-break (01-storage-format.md
/// § Fractal layout ▸ Rules, stated 2026-07-29 — a live occurrence keeps the identity regardless of
/// age), so re-deriving it from a `".trash/"` prefix downstream would be parsing back a fact that
/// was in hand here.
///
/// Occurrence order is the traversal order the rule's *last* tie-break is stated in: lanes in
/// display order, each lane immediately followed by its cards in display order, then `.trash/`'s
/// flat entries in display order. The trash sits last only because the walk meets it last —
/// nothing rides on that any more, because the straddle case is decided by rung 0 long before
/// traversal order is consulted.
private static func dedupeIdentities(
inBoardAt root: URL,
lanes: [WalkedLane],
trash: [Card],
historyRanker: IdentityHistoryRanker?
) -> IntegrityRules.DedupeVerdict {
typealias Container = IntegrityRules.IdentityOccurrence.Container
// (path, name, container, title) in traversal order — the inventory, before anything is read
// from disk.
var inventory: [(path: String, name: String, container: Container, title: String?)] = []
for lane in lanes {
inventory.append((
path: lane.name,
name: lane.name,
container: .live,
title: lane.document.title.value
))
for card in lane.cards {
inventory.append((
path: lane.name + "/" + card.id.rawValue,
name: card.id.rawValue,
container: .live,
title: card.title.value
))
}
}
for entry in trash {
inventory.append((
path: trashFolderName + "/" + entry.id.rawValue,
name: entry.id.rawValue,
container: .trashed,
title: entry.title.value
))
}
// The gate.
var seen: Set<String> = []
var collides = false
for entry in inventory where !seen.insert(IntegrityRules.canonicalIdentity(entry.name)).inserted {
collides = true
break
}
guard collides else { return IntegrityRules.DedupeVerdict(duplicates: [], caseTwins: []) }
return IntegrityRules.dedupe(inventory.map { entry in
IntegrityRules.IdentityOccurrence(
path: entry.path,
name: entry.name,
container: entry.container,
title: entry.title,
birth: birthDate(of: root.appendingPathComponent(entry.path, isDirectory: true)),
historyRank: historyRanker?.rank(entry.path)
)
})
}
/// A folder's filesystem birth date, `nil` when the volume does not keep one or the read fails —
/// the second rung of the earlier-occurrence-wins ladder (§ Fractal layout ▸ Rules: "without
/// history, the older folder (filesystem birth date) wins").
///
/// `nil` is a first-class answer rather than a fallback date: an unreadable birth date must drop
/// the comparison to traversal order, and substituting `.distantPast` here would silently make an
/// unreadable folder *win* every comparison it entered.
private static func birthDate(of folder: URL) -> Date? {
(try? folder.resourceValues(forKeys: [.creationDateKey]))?.creationDate
}
/// One lane as the walk read it, before the dedupe decided what renders — a `Lane` minus the
/// decision, which is the only reason it exists rather than the walk building `Lane` values
/// directly.
///
/// It carries the folder *name* rather than an `ItemID` because the dedupe's whole subject is
/// spelling and identity being different questions, and the verbatim name is what answers both.
private struct WalkedLane {
let name: String
let schema: Int
let order: Double
let document: FrontmatterDocument
/// Already in display order — the traversal the dedupe's last tie-break is stated in.
let cards: [Card]
/// The `Lane` this becomes, or `nil` when the lane folder itself is withheld — a withheld
/// lane takes its subtree out of the snapshot with it, and its cards' own collisions were
/// already decided (they are occurrences in their own right).
func rendered(withholding withheld: Set<String>) -> Lane? {
guard !withheld.contains(name) else { return nil }
return Lane(
id: ItemID(rawValue: name),
schema: schema,
title: document.title,
created: document.created,
modified: document.modified,
modifiedBy: document.modifiedBy,
deleted: document.deleted,
background: document.background,
icon: document.icon,
iconColor: document.iconColor,
order: order,
width: document.width,
cards: cards.filter { !withheld.contains(name + "/" + $0.id.rawValue) },
document: document
)
}
}
// MARK: - The earlier-occurrence-wins history seam
/// **Where git path history plugs into the duplicate-id winner rule** (01-storage-format.md
/// § Fractal layout ▸ Rules: "on git boards, the path history already tracks outranks the
/// newcomer (both tracked: the path that entered history first)").
///
/// A seam rather than an implementation because the first rung of that ladder is unbuildable in
/// base: base Lanework links no git machinery at all (12-editions.md; `scripts/verify-editions.sh`
/// scans the base binary to prove it), so the loader consults an injected ranker and falls through
/// to birth date and traversal order when there is none — which is every base board, and every Pro
/// board without a repo.
///
/// Deliberately one closure and no protocol: the loader asks one question — "how early did this
/// path enter history" — and pro-m1's implementation answers it from `git log --diff-filter=A
/// --follow`-shaped plumbing behind the edition seam. `nil` means "untracked, or no history
/// here", which the rule reads as *outranked by anything tracked*.
///
/// - Parameter rank: keyed by the occurrence's **board-root-relative path**, which is what a
/// repo's path history knows; lower is earlier.
public struct IdentityHistoryRanker: Sendable {
public let rank: @Sendable (String) -> Int?
public init(rank: @escaping @Sendable (String) -> Int?) {
self.rank = rank
}
}
/// Whether a folder in `.trash/` has the *shape* of a lane — at least one identity-shaped child
/// holding its own `index.md` (01-storage-format.md § Deletion: "UUID-shaped children with
/// their own `index.md` → lane … else card").
@@ -739,6 +933,17 @@ public struct LoadResult: Sendable {
public var claimedNameSquatters: [ClaimedNameSquatter] {
defects.compactMap { if case let .claimedNameSquatted(work) = $0 { work } else { nil } }
}
/// The later occurrences this walk withheld from `model` — a **view over `defects`**
/// (01-storage-format.md § Fractal layout ▸ Rules; re-ruled 2026-07-29 — the silent remint).
///
/// **The one defect the snapshot is already missing.** Every other defect names something that is
/// both on disk and in the model; a withheld duplicate is on disk and deliberately *not* in the
/// model, which is what makes the one-item-per-id invariant hold by construction rather than by
/// hope. In traversal order — which is the order the remint writes them and the notice names them.
public var duplicateIdentities: [DuplicateIdentity] {
defects.compactMap { if case let .duplicateIdentity(work) = $0 { work } else { nil } }
}
}
/// A tolerated anomaly the loader kept going past. Never blocks a load — see `BoardLoadError`
@@ -763,6 +968,19 @@ public enum LoadWarning: Sendable, Equatable, CustomStringConvertible {
/// (01-storage-format.md § Deletion) — ignored, never tombstones the board.
case boardLevelDeletedIgnored
/// A folder whose name is a **case-spelled twin** of another occurrence of the same identity —
/// one item typed two ways (01-storage-format.md § Fractal layout ▸ Rules): a spelling some *live*
/// occurrence carries wins over one only trash ghosts carry (the container preference, stated
/// 2026-07-29), and among the candidates the canonical all-lowercase spelling wins where present,
/// else the lexicographically first. This is the loser. Skipped, preserved verbatim, never
/// rendered — **and never reminted**: it is a spelling artifact of the item that rendered, not a
/// copy, so there is nothing to heal.
///
/// Its home is here rather than in the defect stream precisely because of that: a warning says
/// "this was ignored, it is staying exactly where it is, there is nothing to do", which is the
/// whole of the tolerate tier's verdict on it. Both paths are root-relative.
case caseTwinIgnored(path: String, winner: String)
public var description: String {
switch self {
case let .missingIndex(path):
@@ -771,6 +989,8 @@ public enum LoadWarning: Sendable, Equatable, CustomStringConvertible {
"\(path): folder name is not UUID-shaped, ignored as a stray"
case .boardLevelDeletedIgnored:
"index.md: board-level 'deleted' key is meaningless, ignored"
case let .caseTwinIgnored(path, winner):
"\(path): case-spelled twin of \(winner), ignored as a spelling artifact"
}
}
}