diff --git a/DESIGN/01-storage-format.md b/DESIGN/01-storage-format.md index 932d623..cfefa31 100644 --- a/DESIGN/01-storage-format.md +++ b/DESIGN/01-storage-format.md @@ -81,7 +81,7 @@ Body: lane description / WIP policy / notes. Body: the card's content — the whole point. -Schema-owned display fields are Lanework's to interpret — **coerce where a sensible reading exists, fall back to the field's default where none does** (settled). A scalar of the wrong YAML type reads as its source text (`title: 2048` displays as "2048", `width: "2"` reads as 2); where no sensible reading exists — a sequence or mapping where a scalar belongs, a non-integer width, an unparseable timestamp — the field falls back to its default: untitled placeholder, width 1, no color, no icon. Coercion is read-side only; the bytes on disk are **preserved verbatim, never rewritten**. One tombstone nuance: any *present* `deleted:` key tombstones the item — an unusable timestamp still deletes, its date merely unknown (the user's intent to delete outranks the broken date). **Duplicate keys: last one wins** (settled — the coercing read; strict YAML would reject the file, so this is a deliberate divergence in the editor's favor): a key appearing twice reads as its last occurrence, earlier occurrences preserved verbatim on disk and invisible. An app write of a duplicated key rewrites the winning (last) occurrence and removes the earlier ones — the app owns the keys it writes, and leaving a stale twin would resurrect it if the winner were later removed; removing a key removes all its occurrences. **App rewrites preserve comments** (settled): comments on their own lines always survive a rewrite; an inline comment on a rewritten value line is re-spliced after the new value — best-effort, guaranteed for plain scalar lines (the realistic case), dropped only in pathological shapes. Fail-fast remains reserved for structure (`schema`, `order`, YAML validity) — and it covers *malformed*, not just missing: an `order` that is present but non-numeric is the same loud malformed-input rejection as a missing one. **Readable-but-uneditable shapes load; writes to them refuse** (settled): frontmatter the surgical editor can't key by spans — a whole-frontmatter flow mapping, non-scalar keys — reads and renders normally, and any app write to that file fails loudly through the per-file write-failure banner (02-architecture.md ▸ Write-failure surfacing) naming the shape, never a silent corruption and never a load rejection. +Schema-owned display fields are Lanework's to interpret — **coerce where a sensible reading exists, fall back to the field's default where none does** (settled). A scalar of the wrong YAML type reads as its source text (`title: 2048` displays as "2048", `width: "2"` reads as 2); where no sensible reading exists — a sequence or mapping where a scalar belongs, a non-integer width, an unparseable timestamp — the field falls back to its default: untitled placeholder, width 1, no color, no icon. Coercion is read-side only; the bytes on disk are **preserved verbatim, never rewritten**. One tombstone nuance: any *present* `deleted:` key tombstones the item — an unusable timestamp still deletes, its date merely unknown (the user's intent to delete outranks the broken date). **Duplicate keys: last one wins** (settled — the coercing read; strict YAML would reject the file, so this is a deliberate divergence in the editor's favor): a *top-level* key appearing twice reads as its last occurrence, earlier occurrences preserved verbatim on disk and invisible. A duplicate inside a nested mapping value remains a YAML error — the rescue applies where the editor's slip actually happens, the top level. An app write of a duplicated key rewrites the winning (last) occurrence and removes the earlier ones — the app owns the keys it writes, and leaving a stale twin would resurrect it if the winner were later removed; removing a key removes all its occurrences. **App rewrites preserve comments** (settled): comments on their own lines always survive a rewrite; an inline comment on a rewritten value line is re-spliced after the new value — best-effort, guaranteed for plain scalar lines (the realistic case), dropped only in pathological shapes. Fail-fast remains reserved for structure (`schema`, `order`, YAML validity) — and it covers *malformed*, not just missing: an `order` that is present but non-numeric is the same loud malformed-input rejection as a missing one. **Readable-but-uneditable shapes load; writes to them refuse** (settled): frontmatter the surgical editor can't key by spans — a whole-frontmatter flow mapping, non-scalar keys — reads and renders normally, and any app write to that file fails loudly through the per-file write-failure banner (02-architecture.md ▸ Write-failure surfacing) naming the shape, never a silent corruption and never a load rejection. ## Enhanced schema (reserved, out of scope) diff --git a/Kanban/Storage/FrontmatterDocument.swift b/Kanban/Storage/FrontmatterDocument.swift index 9936530..e580484 100644 --- a/Kanban/Storage/FrontmatterDocument.swift +++ b/Kanban/Storage/FrontmatterDocument.swift @@ -50,6 +50,17 @@ public struct FrontmatterDocument: Sendable, Equatable { // MARK: - Parsing + /// Reads a file. YAML is composed to validate and interpret it; the text itself is retained + /// verbatim, so the parse only decides how the document *reads*, never how it is written. + /// + /// **Duplicate top-level keys: last one wins** (01-storage-format.md § Frontmatter) — a + /// deliberate divergence from strict YAML, which rejects the file outright. The key's last + /// occurrence supplies the value, earlier twins are invisible to every read, and each + /// occurrence still keeps its own span, so the file round-trips byte-identically. `keys` and + /// `fields` present that effective view: one entry per key, at the position of the occurrence + /// that won (`a`, `b`, `a` reads as `b`, `a`) — which is also the order the file itself ends + /// up in once `set` collapses the twins. A duplicate *nested* inside a value's own mapping is + /// out of scope: it stays `unparseableYAML`, as does everything else libYAML rejects. public static func parse(_ text: String) throws(FrontmatterError) -> FrontmatterDocument { let lines = Self.lines(of: text) guard let first = lines.first, Self.isDelimiter(first) else { throw .missingOpeningDelimiter } @@ -58,14 +69,7 @@ public struct FrontmatterDocument: Sendable, Equatable { } let yamlLines = Array(lines[1 ..< closingIndex]) - let node: Node? - do { - node = try Yams.compose(yaml: yamlLines.joined()) - } catch let error as YamlError { - throw .unparseableYAML(message: "\(error)", line: Self.fileLine(of: error)) - } catch { - throw .unparseableYAML(message: "\(error)", line: nil) - } + let (node, placeholders) = try Self.composeToleratingDuplicateKeys(yamlLines) let mapping: Node.Mapping switch node { @@ -81,27 +85,86 @@ public struct FrontmatterDocument: Sendable, Equatable { } } - let keys = mapping.map { $0.key.string } - let values = zip(keys, mapping).compactMap { key, pair -> KeyedValue? in - key.map { KeyedValue(key: $0, value: YAMLValue(pair.value)) } - } + let keys = mapping.map { pair in pair.key.string.map { placeholders[$0] ?? $0 } } return FrontmatterDocument( openingDelimiter: first, closingDelimiter: lines[closingIndex], spans: Self.makeSpans(lines: yamlLines, keys: keys), - values: values, + values: Self.lastWinsValues(keys: keys, mapping: mapping), body: lines[(closingIndex + 1)...].joined() ) } + /// Composes the YAML block, tolerating repeated top-level keys. + /// + /// libYAML refuses a mapping with a duplicated key, but its error is informative enough to + /// disarm: it names the duplicated keys and marks the *first* occurrence of one of them. That + /// occurrence's key is renamed to a placeholder and the block re-composed, until nothing is + /// duplicated. Renaming leaves the line count — and therefore every subsequent mark, and the + /// whole span layout — identical to the original, so the composed mapping still lines up with + /// the file's lines one for one; the returned table maps each placeholder back to the key it + /// stands for. Anything that is not a duplicate at column 0 (a nested duplicate included) is + /// reported as `unparseableYAML`, unchanged. + private static func composeToleratingDuplicateKeys( + _ lines: [String] + ) throws(FrontmatterError) -> (node: Node?, placeholders: [String: String]) { + var work = lines + var placeholders: [String: String] = [:] + let prefix = Self.placeholderPrefix(avoiding: lines.joined()) + + while true { + do { + return (try Yams.compose(yaml: work.joined()), placeholders) + } catch let error as YamlError { + guard case let .duplicatedKeysInMapping(duplicates, context) = error else { + throw FrontmatterError.unparseableYAML(message: "\(error)", line: Self.fileLine(of: error)) + } + // Marks are 1-based and count only the YAML block, which the rename never resizes. + let index = context.mark.line - 1 + guard placeholders.count < lines.count, work.indices.contains(index), + let key = duplicates.first(where: { Self.lineStartsKey(work[index], key: $0) }) + else { + throw FrontmatterError.unparseableYAML(message: "\(error)", line: Self.fileLine(of: error)) + } + let placeholder = prefix + String(placeholders.count) + let line = work[index] + placeholders[placeholder] = key + work[index] = placeholder + String(line.dropFirst(Self.keyText(of: line, key: key).count)) + } catch { + throw FrontmatterError.unparseableYAML(message: "\(error)", line: nil) + } + } + } + + /// A key stem that appears nowhere in the file, so a renamed duplicate can never collide with + /// a key the author actually wrote. + private static func placeholderPrefix(avoiding text: String) -> String { + var prefix = "__lanework_duplicate_" + while text.contains(prefix) { prefix += "_" } + return prefix + } + + /// One entry per distinct key, carrying the *last* occurrence's value and sitting where that + /// occurrence sits — the last-wins reading of a file with repeated keys. + private static func lastWinsValues(keys: [String?], mapping: Node.Mapping) -> [KeyedValue] { + var values: [KeyedValue] = [] + for (key, pair) in zip(keys, mapping) { + guard let key else { continue } + values.removeAll { $0.key == key } + values.append(KeyedValue(key: key, value: YAMLValue(pair.value))) + } + return values + } + public func serialized() -> String { openingDelimiter + spans.map(\.text).joined() + closingDelimiter + body } // MARK: - Generic access - /// Top-level keys in document order. + /// Top-level keys in document order — the effective, last-wins view: a key written twice + /// appears once, where its winning (last) occurrence sits (see `parse`). public var keys: [String] { values.map(\.key) } public var fields: [FrontmatterField] { @@ -122,18 +185,29 @@ public struct FrontmatterDocument: Sendable, Equatable { values.first { $0.key == key }?.value } - /// The key's value text exactly as written, minus the `key:` header and surrounding whitespace. + /// The key's value text exactly as written, minus the `key:` header and surrounding + /// whitespace. Read from the winning (last) occurrence, like every other read. public func rawValue(for key: String) -> String? { - spans.first { $0.key == key }.map { Self.valueText(of: $0.value, key: key) } + spans.last { $0.key == key }.map { Self.valueText(of: $0.value, key: key) } } // MARK: - Surgical edits /// Rewrites only this key's value lines. A new key is appended before the closing delimiter. + /// + /// Two rules from 01-storage-format.md § Frontmatter land here: + /// + /// - **Duplicates collapse.** The winning (last) occurrence is rewritten and every earlier + /// twin is deleted in the same call — the app owns the keys it writes, and a stale twin left + /// behind would resurrect itself the moment the winner were removed. + /// - **Comments survive.** Comments on their own lines are never touched, and an inline + /// comment trailing the rewritten value line is re-spliced after the new value at its + /// original distance (`order: 3072 # keep at top` → `order: 4096 # keep at top`). That + /// re-splice is best-effort — see `inlineComment(of:keyText:)`. public mutating func set(_ key: String, to value: FrontmatterValue) { - if let index = spans.firstIndex(where: { $0.key == key }) { - let keyText = Self.keyText(of: spans[index].value, key: key) - spans[index].value = "\(keyText): \(value.yamlText)\n" + if let index = spans.lastIndex(where: { $0.key == key }) { + spans[index].value = Self.rewritten(spans[index].value, key: key, to: value) + for twin in (0 ..< index).reversed() where spans[twin].key == key { clearSpan(at: twin) } } else { spans.append(Span(key: key, value: "\(FrontmatterValue.emitScalar(key)): \(value.yamlText)\n", tail: "")) } @@ -146,19 +220,23 @@ public struct FrontmatterDocument: Sendable, Equatable { } } - /// Deletes the key's value lines. Trailing comments and blank lines survive — they read as + /// Deletes the key's value lines — *every* occurrence of it, so a removed key cannot be + /// resurrected by an earlier twin. Trailing comments and blank lines survive: they read as /// belonging to whatever follows, and losing user text is worse than an orphaned comment. public mutating func remove(_ key: String) { - if let index = spans.firstIndex(where: { $0.key == key }) { - if spans[index].tail.isEmpty { - spans.remove(at: index) - } else { - spans[index] = Span(key: nil, value: "", tail: spans[index].tail) - } - } + for index in spans.indices.reversed() where spans[index].key == key { clearSpan(at: index) } values.removeAll { $0.key == key } } + /// Drops one span's value lines, keeping its trailing comments and blank lines. + private mutating func clearSpan(at index: Int) { + if spans[index].tail.isEmpty { + spans.remove(at: index) + } else { + spans[index] = Span(key: nil, value: "", tail: spans[index].tail) + } + } + // MARK: - Line splitting private static func lines(of text: String) -> [String] { @@ -176,7 +254,8 @@ public struct FrontmatterDocument: Sendable, Equatable { /// Splits the YAML block into per-key spans. A line only starts a span when it opens the key /// the parser expects next, so an unindented value continuation (multi-line flow collections) - /// can never be mistaken for a key. + /// can never be mistaken for a key. `keys` lists every top-level occurrence, duplicates + /// included, so each twin gets its own span and the block still serializes back verbatim. private static func makeSpans(lines: [String], keys: [String?]) -> [Span] { var spans: [Span] = [] var current: Span? @@ -235,6 +314,118 @@ public struct FrontmatterDocument: Sendable, Equatable { keyCandidates(key).first { spanValue.hasPrefix($0 + ":") } ?? FrontmatterValue.emitScalar(key) } + // MARK: - Rewriting a value line + + /// The replacement text for a span's value lines: `key: newValue`, plus whatever inline + /// comment the old lines carried, ending the way they ended (a CRLF file stays CRLF). + private static func rewritten(_ spanValue: String, key: String, to value: FrontmatterValue) -> String { + let keyText = Self.keyText(of: spanValue, key: key) + let (comment, terminator) = Self.inlineComment(of: spanValue, keyText: keyText) + return "\(keyText): \(value.yamlText)\(comment)\(terminator)" + } + + /// The trailing inline comment on a span, its leading whitespace included so it re-splices at + /// the distance the author left, along with the line's own terminator. + /// + /// **Best-effort by design** (01-storage-format.md § Frontmatter). Guaranteed for the + /// realistic case — a value written on one line as a plain, single-quoted, or double-quoted + /// scalar — where YAML's own rule decides: `#` opens a comment only when preceded by + /// whitespace and not inside a quoted scalar, so the `#` in `title: "a # b"` is value text and + /// survives as part of the old value rather than being re-spliced. Pathological shapes lose + /// the comment instead of making the editor guess: a value spanning several lines (block + /// scalar, multi-line flow collection) is not scanned at all, and neither is a single-line + /// flow collection whose quotes do not balance. + private static func inlineComment(of spanValue: String, keyText: String) -> (comment: String, terminator: String) { + // A CRLF pair is one `Character`, so both endings have to be named explicitly. + var characters = Array(spanValue) + var terminator = "\n" + if characters.last == "\r\n" { + characters.removeLast() + terminator = "\r\n" + } else if characters.last == "\n" { + characters.removeLast() + } + // Only a value written on a single line is scanned; anything taller keeps no comment. + guard !characters.contains(where: { $0 == "\n" || $0 == "\r\n" }) else { return ("", terminator) } + let header = Array(keyText + ":") + guard characters.starts(with: header) else { return ("", terminator) } + + var valueStart = header.count + while valueStart < characters.count, isSpaceOrTab(characters[valueStart]) { valueStart += 1 } + guard let hash = commentIndex(in: characters, valueStart: valueStart) else { return ("", terminator) } + + var start = hash + while start > 0, isSpaceOrTab(characters[start - 1]) { start -= 1 } + return (String(characters[start...]), terminator) + } + + /// Where this line's comment begins, or nil if it has none. The value's opening character + /// says how much of the line is off limits: a quoted scalar is skipped whole, a flow + /// collection is scanned with its quotes tracked, and a plain scalar cannot contain ` #` at + /// all — that sequence is exactly what ends it. + private static func commentIndex(in characters: [Character], valueStart: Int) -> Int? { + guard valueStart < characters.count else { return nil } + var index = valueStart + switch characters[valueStart] { + case "\"", "'": + guard let end = endOfQuoted(characters, from: valueStart) else { return nil } + index = end + case "[", "{": + return flowCommentIndex(characters, from: valueStart) + default: + break + } + while index < characters.count { + if characters[index] == "#", index > 0, isSpaceOrTab(characters[index - 1]) { return index } + index += 1 + } + return nil + } + + /// One past the closing quote of the quoted scalar starting at `start`, or nil if it does not + /// close on this line. `\"` escapes inside double quotes, `''` inside single ones. + private static func endOfQuoted(_ characters: [Character], from start: Int) -> Int? { + let quote = characters[start] + var index = start + 1 + while index < characters.count { + if quote == "\"", characters[index] == "\\" { + index += 2 + continue + } + if characters[index] == quote { + if quote == "'", index + 1 < characters.count, characters[index + 1] == "'" { + index += 2 + continue + } + return index + 1 + } + index += 1 + } + return nil + } + + /// A flow collection on one line: a quote here really does open a quoted scalar, so a `#` + /// inside one is value text. + private static func flowCommentIndex(_ characters: [Character], from start: Int) -> Int? { + var index = start + while index < characters.count { + switch characters[index] { + case "\"", "'": + guard let end = endOfQuoted(characters, from: index) else { return nil } + index = end + case "#" where index > 0 && isSpaceOrTab(characters[index - 1]): + return index + default: + index += 1 + } + } + return nil + } + + private static func isSpaceOrTab(_ character: Character) -> Bool { + character == " " || character == "\t" + } + private static func valueText(of spanValue: String, key: String) -> String { let header = keyText(of: spanValue, key: key) + ":" guard spanValue.hasPrefix(header) else { diff --git a/KanbanTests/FrontmatterTests.swift b/KanbanTests/FrontmatterTests.swift index 395a39c..a985b96 100644 --- a/KanbanTests/FrontmatterTests.swift +++ b/KanbanTests/FrontmatterTests.swift @@ -69,6 +69,19 @@ private enum Fixture { --- body """ + + /// A hand-edited file that says `title` twice — strict YAML would reject it; the editor reads + /// the last one and keeps both on disk (01-storage-format.md § Frontmatter). + static let duplicated = """ + --- + schema: 1 + title: First + order: 1024 + title: Second + --- + body + + """ } private func parseError(_ text: String) -> FrontmatterError? { @@ -122,6 +135,11 @@ struct FrontmatterRoundTripTests { "---\nschema: 1\n---\nbody without newline", // CRLF throughout "---\r\nschema: 1\r\n---\r\nbody\r\n", + // duplicate top-level keys — every occurrence keeps its own span + Fixture.duplicated, + "---\ntitle: a\ntitle: b\ntitle: c\n---\nbody\n", + // a duplicate whose earlier twin owns a comment and a block scalar + "---\nnotes: |\n one\n# about notes\nnotes: two\n---\nbody\n", ]) func serializingAnUntouchedDocumentIsByteIdentical(text: String) throws { let document = try FrontmatterDocument.parse(text) @@ -165,10 +183,13 @@ struct FrontmatterMalformedTests { "---\nfoo: [1, 2\n---\nbody\n", "---\nschema: 1\n\tindented-with-tab: 2\n---\nbody\n", "---\nschema: 1\n bad: indent\n---\nbody\n", - "---\ndupe: 1\ndupe: 2\n---\nbody\n", "---\n\"unterminated: 1\n---\nbody\n", "---\nfoo: {a: 1\n---\nbody\n", "---\n*undefined-alias\n---\nbody\n", + // last-wins rescues duplicates at the top level only — nested ones stay a hard failure + "---\nouter:\n a: 1\n a: 2\n---\nbody\n", + "---\nflow: {a: 1, a: 2}\n---\nbody\n", + "---\ndupe: 1\ndupe: 2\nouter:\n a: 1\n a: 2\n---\nbody\n", ]) func unparseableYAML(text: String) { guard case .unparseableYAML = parseError(text) else { @@ -215,6 +236,155 @@ struct FrontmatterMalformedTests { #expect(parseError(text) == nil) } } + + /// Strict YAML rejects a repeated mapping key; the editor deliberately does not. + @Test func duplicateTopLevelKeysAreNotAParseError() throws { + for text in [ + Fixture.duplicated, + "---\ndupe: 1\ndupe: 2\n---\nbody\n", + "---\ntitle: a\ntitle: b\ntitle: c\n---\nbody\n", + ] { + #expect(parseError(text) == nil) + } + } +} + +// MARK: - Duplicate keys (last one wins) + +/// 01-storage-format.md § Frontmatter: a key written twice reads as its last occurrence, the +/// earlier ones preserved verbatim on disk and invisible; an app write of that key rewrites the +/// winner and removes the twins; a removal removes them all. +struct FrontmatterDuplicateKeyTests { + @Test func theLastOccurrenceIsTheOneThatReads() throws { + let document = try FrontmatterDocument.parse(Fixture.duplicated) + #expect(document.title == .valid("Second")) + #expect(document.value(for: "title") == .string("Second")) + #expect(document.rawValue(for: "title") == "Second") + #expect(document.contains("title")) + } + + @Test func threeOccurrencesStillReadAsTheLast() throws { + let document = try FrontmatterDocument.parse("---\ntitle: a\ntitle: b\ntitle: c\n---\nbody\n") + #expect(document.title == .valid("c")) + #expect(document.keys == ["title"]) + } + + /// The effective view has one entry per key, sitting where the winner sits — the order the + /// file itself takes once `set` collapses the twins. + @Test func iterationPresentsTheEffectiveView() throws { + let document = try FrontmatterDocument.parse(Fixture.duplicated) + #expect(document.keys == ["schema", "order", "title"]) + #expect(document.fields.map(\.key) == ["schema", "order", "title"]) + #expect(document.fields.last?.rawValue == "Second") + } + + @Test func bothOccurrencesSurviveAnUntouchedRoundTrip() throws { + let document = try FrontmatterDocument.parse(Fixture.duplicated) + let output = document.serialized() + #expect(output == Fixture.duplicated) + #expect(output.contains("title: First")) + #expect(output.contains("title: Second")) + } + + @Test func settingRewritesTheWinnerAndDeletesTheTwins() throws { + var document = try FrontmatterDocument.parse(Fixture.duplicated) + document.set("title", to: .string("Third")) + #expect(document.serialized() == "---\nschema: 1\norder: 1024\ntitle: Third\n---\nbody\n") + #expect(document.title == .valid("Third")) + } + + @Test func settingCollapsesThreeOccurrencesToOne() throws { + var document = try FrontmatterDocument.parse("---\ntitle: a\ntitle: b\ntitle: c\n---\nbody\n") + document.set("title", to: .string("d")) + #expect(document.serialized() == "---\ntitle: d\n---\nbody\n") + } + + /// A stale twin left behind would resurrect itself the moment the winner were removed. + @Test func removingTakesEveryOccurrence() throws { + var document = try FrontmatterDocument.parse(Fixture.duplicated) + document.remove("title") + #expect(document.serialized() == "---\nschema: 1\norder: 1024\n---\nbody\n") + #expect(document.title == .missing) + #expect(!document.contains("title")) + #expect(document.keys == ["schema", "order"]) + } + + @Test func removingTakesAllThreeOccurrences() throws { + var document = try FrontmatterDocument.parse("---\nschema: 1\ntitle: a\ntitle: b\ntitle: c\n---\nbody\n") + document.remove("title") + #expect(document.serialized() == "---\nschema: 1\n---\nbody\n") + } + + /// Own-line comments outlive the twin they were written above (01-storage-format.md). + @Test func aCommentAboveTheFirstOccurrenceSurvivesTheCollapse() throws { + let text = "---\n# the original title\ntitle: First\ntitle: Second\n---\nbody\n" + var document = try FrontmatterDocument.parse(text) + #expect(document.serialized() == text) + + document.set("title", to: .string("Third")) + #expect(document.serialized() == "---\n# the original title\ntitle: Third\n---\nbody\n") + } + + /// Same rule when the comment trails the earlier twin rather than heading it. + @Test func aCommentBelowTheFirstOccurrenceSurvivesTheCollapse() throws { + var document = try FrontmatterDocument.parse("---\ntitle: First\n# still relevant\ntitle: Second\n---\nbody\n") + document.remove("title") + #expect(document.serialized() == "---\n# still relevant\n---\nbody\n") + } + + @Test func twoDifferentKeysMayEachBeDuplicated() throws { + let text = "---\na: 1\nb: 1\na: 2\nb: 2\n---\nbody\n" + var document = try FrontmatterDocument.parse(text) + #expect(document.serialized() == text) + #expect(document.value(for: "a") == .int(2)) + #expect(document.value(for: "b") == .int(2)) + #expect(document.keys == ["a", "b"]) + + document.set("a", to: .int(3)) + #expect(document.serialized() == "---\nb: 1\na: 3\nb: 2\n---\nbody\n") + } + + /// A duplicated twin whose value is a block scalar still spans its own lines. + @Test func aDuplicateWithAMultiLineTwinCollapsesCleanly() throws { + let text = "---\nnotes: |\n one\n# about notes\nnotes: two\n---\nbody\n" + var document = try FrontmatterDocument.parse(text) + #expect(document.serialized() == text) + #expect(document.value(for: "notes") == .string("two")) + + document.set("notes", to: .string("three")) + #expect(document.serialized() == "---\n# about notes\nnotes: three\n---\nbody\n") + } + + /// The twins need not be spelled alike — YAML reads `"title"` and `title` as the same key, + /// and each occurrence is rewritten (or dropped) in its own spelling. + @Test func twinsSpelledDifferentlyAreStillTheSameKey() throws { + let text = "---\n\"title\": First\ntitle: Second\n---\nbody\n" + var document = try FrontmatterDocument.parse(text) + #expect(document.serialized() == text) + #expect(document.title == .valid("Second")) + + document.set("title", to: .string("Third")) + #expect(document.serialized() == "---\ntitle: Third\n---\nbody\n") + } + + @Test func duplicatesInACRLFFileRoundTripAndCollapse() throws { + let text = "---\r\ntitle: First\r\norder: 1\r\ntitle: Second\r\n---\r\nbody\r\n" + var document = try FrontmatterDocument.parse(text) + #expect(document.serialized() == text) + #expect(document.title == .valid("Second")) + + document.set("title", to: .string("Third")) + #expect(document.serialized() == "---\r\norder: 1\r\ntitle: Third\r\n---\r\nbody\r\n") + } + + @Test func duplicatesSurviveReparsing() throws { + var document = try FrontmatterDocument.parse(Fixture.duplicated) + document.set("order", to: .double(2048)) + let reparsed = try FrontmatterDocument.parse(document.serialized()) + #expect(reparsed.serialized() == document.serialized()) + #expect(reparsed.title == .valid("Second")) + #expect(reparsed.order == .valid(2048)) + } } // MARK: - Generic access @@ -436,7 +606,7 @@ struct FrontmatterEditTests { var document = try FrontmatterDocument.parse(text) document.set("project", to: .string("kanban")) #expect(document.serialized() - == "---\nschema: 1\nproject: kanban\n\n# a note\norder: 5\n---\nbody\n") + == "---\nschema: 1\nproject: kanban # agent overlay\n\n# a note\norder: 5\n---\nbody\n") } @Test func appendingANewKeyGoesBeforeTheClosingDelimiter() throws { @@ -538,6 +708,112 @@ struct FrontmatterEditTests { } } +// MARK: - Inline comments on rewritten lines + +/// 01-storage-format.md § Frontmatter, "App rewrites preserve comments": an inline comment on a +/// rewritten value line is re-spliced after the new value — best-effort, guaranteed for plain +/// single-line scalars, allowed to drop in pathological shapes. +struct FrontmatterInlineCommentTests { + private func edited(_ frontmatter: String, _ key: String, _ value: FrontmatterValue) throws -> String { + var document = try FrontmatterDocument.parse("---\n\(frontmatter)\n---\nbody\n") + document.set(key, to: value) + return String(document.serialized().dropFirst("---\n".count).dropLast("\n---\nbody\n".count)) + } + + @Test func aPlainScalarKeepsItsCommentAndItsSpacing() throws { + #expect(try edited("order: 3072 # keep at top", "order", .double(4096)) + == "order: 4096 # keep at top") + #expect(try edited("order: 3072 # tight", "order", .double(4096)) == "order: 4096 # tight") + #expect(try edited("order: 3072\t# tabbed", "order", .double(4096)) == "order: 4096\t# tabbed") + } + + @Test func aValueLineWithoutACommentIsRewrittenAsBefore() throws { + #expect(try edited("order: 3072", "order", .double(4096)) == "order: 4096") + #expect(try edited("title: My Board", "title", .string("Renamed")) == "title: Renamed") + } + + /// A `#` inside a quoted scalar is value text, not a comment — it must not be re-spliced. + @Test func aHashInsideAQuotedScalarIsNotAComment() throws { + #expect(try edited("title: \"hash # inside\"", "title", .string("Renamed")) == "title: Renamed") + #expect(try edited("title: 'hash # inside'", "title", .string("Renamed")) == "title: Renamed") + #expect(try edited("background: \"#ff8800\"", "background", .string("slate")) == "background: slate") + } + + /// …but a real comment *after* such a value still is one. + @Test func aCommentAfterAQuotedScalarIsStillSpliced() throws { + #expect(try edited("title: \"hash # inside\" # real note", "title", .string("Renamed")) + == "title: Renamed # real note") + #expect(try edited("title: 'it''s # fine' # real note", "title", .string("Renamed")) + == "title: Renamed # real note") + } + + /// An apostrophe mid-scalar does not open a quoted scalar, so the comment after it is found. + @Test func aPlainScalarWithAnApostropheKeepsItsComment() throws { + #expect(try edited("title: it's fine # note", "title", .string("Renamed")) + == "title: Renamed # note") + #expect(try edited("title: say \"hi\" # note", "title", .string("Renamed")) + == "title: Renamed # note") + } + + @Test func aSingleLineFlowCollectionKeepsItsComment() throws { + #expect(try edited("labels: [a, \"b # c\"] # note", "labels", .raw("[x]")) + == "labels: [x] # note") + #expect(try edited("template: {order: 3} # picker slot", "template", .raw("{order: 7}")) + == "template: {order: 7} # picker slot") + } + + @Test func aCommentOnAnEmptyValueIsKept() throws { + #expect(try edited("title: # to be filled in", "title", .string("Named")) + == "title: Named # to be filled in") + } + + @Test func theCommentSurvivesSuccessiveSets() throws { + var document = try FrontmatterDocument.parse("---\norder: 3072 # keep at top\n---\nbody\n") + document.set("order", to: .double(4096)) + document.set("order", to: .double(8192)) + #expect(document.serialized() == "---\norder: 8192 # keep at top\n---\nbody\n") + } + + @Test func theCommentSurvivesAReparse() throws { + var document = try FrontmatterDocument.parse("---\norder: 3072 # keep at top\n---\nbody\n") + document.set("order", to: .double(4096)) + let reparsed = try FrontmatterDocument.parse(document.serialized()) + #expect(reparsed.order == .valid(4096)) + #expect(reparsed.serialized() == document.serialized()) + } + + /// The winning occurrence's own comment is what carries over; a collapsed twin's inline + /// comment goes with the twin. + @Test func theWinningOccurrencesCommentIsTheOneKept() throws { + var document = try FrontmatterDocument.parse("---\norder: 1 # old\norder: 2 # current\n---\nbody\n") + document.set("order", to: .double(3)) + #expect(document.serialized() == "---\norder: 3 # current\n---\nbody\n") + } + + /// Pathological shapes are allowed to lose the comment rather than have the editor guess. + /// Pinning the chosen behavior: a value spanning several lines is not scanned at all. + @Test func aMultiLineValueDropsItsHeaderComment() throws { + var document = try FrontmatterDocument.parse("---\nnotes: | # about the notes\n line one\n---\nbody\n") + document.set("notes", to: .string("flattened")) + #expect(document.serialized() == "---\nnotes: flattened\n---\nbody\n") + + var flow = try FrontmatterDocument.parse("---\nflow: {a: 1,\nb: 2} # spread out\nlast: x\n---\nbody\n") + flow.set("flow", to: .raw("{a: 9}")) + #expect(flow.serialized() == "---\nflow: {a: 9}\nlast: x\n---\nbody\n") + } + + /// Comments on their own lines were always preserved and still are — including the one that + /// heads the frontmatter and the one that trails it. + @Test func ownLineCommentsAreUntouchedByARewrite() throws { + var document = try FrontmatterDocument.parse(Fixture.rich) + document.set("title", to: .string("Renamed")) + let output = document.serialized() + #expect(output.hasPrefix("---\n# board settings, hand-written\nschema: 1\n")) + #expect(output.contains("\n# trailing note\n---\n")) + #expect(output.contains("project: lanework # agent overlay\n")) + } +} + // MARK: - Scalar emission struct FrontmatterEmissionTests {