lanework/KanbanUITests/EndToEndVerification.md

End-to-end verification

The golden-path UI pass: the flows a user actually performs, driven through the shipping app. It sits beside AccessibilityVerification.md — that document is the accessibility gate, this one is the wiring gate — and shares its prerequisites, because both drive the real app through the real menu bar.

Three suites, one fixture mechanism, and a short list of flows that stayed manual and say so.

Contents

1. Before you start
2. The fixture boards
3. Running the suites
4. The flow inventory
5. Manual-only flows
6. The performance budgets
7. When something fails

Before you start

Same prerequisites as the accessibility pass, and for the same reason — these tests move a real pointer and open real menus:

• A real, unlocked display and a real keyboard. Nothing here survives a headless session, a locked screen, or screen sharing.
• Accessibility automation permission for the test runner: System Settings ▸ Privacy & Security ▸ Accessibility. Without it every test fails on its first click(), for a reason that has nothing to do with the app. Xcode itself usually needs to be listed; so does the terminal, if you are running xcodebuild from one.
• Nothing else driving the pointer while a run is in progress.

A run that is interrupted can leave a stray Lanework process behind — killall Lanework between runs.

Everything a run writes goes into the app's sandbox container (~/Library/Containers/dev.rzen.indie.Kanban/Data/tmp/LaneworkUITestFixture/), which is wiped at the start of every launch. A run never touches your real boards or your recents list: the fixture flag redirects the board registry into the same scratch directory (Kanban/App/UITestLaunch.swift).

The fixture boards

The app builds its own fixture, inside its own container, through its own BoardWriter. It cannot be handed a board on the command line: the sandbox grants files.user-selected.read-write and nothing else, so a path arriving as an argument is a path the app may not read. UITestLaunch carries the full argument.

Three variants, selected by a pair of launch arguments — the base flag, which means "this launch is synthetic", plus the variant's own:

Variant	Launch arguments	Board	Window title
standard	--ui-test-fixture-board --ui-test-fixture-standard	3 lanes (To Do 3 · Doing 1 · Done 1), one rich card with an attachment, one card in .trash/	Audit Board
large	--ui-test-fixture-board --ui-test-fixture-large	8 lanes × 40 cards, titles cycling through four lengths so masonry has heights to balance	Large Board
malformed	--ui-test-fixture-board --ui-test-fixture-malformed	2 lanes, 3 cards — one card's index.md overwritten with unparseable frontmatter	Malformed Board (never appears)

--ui-test-fixture-board on its own still means standard, which is what the accessibility suite has always passed. The flags are exact-matched and double-dashed for the base flag's reasons: a launch switch with a fuzzy edge can be tripped by accident, and a -key value pair would be swallowed by UserDefaults' argument domain and become a preference.

Every variant is built through BoardWriter — createBoard, createLane, createCard, and for the standard board writeBody, importAttachments and deleteCardToTrash. The one exception is the malformed variant's final act, a raw Data.write over one card's index.md: no Writer call produces a file its own loader would reject, and none should.

The builders themselves are unit-tested (KanbanTests/UITestLaunchTests.swift): the flag and variant parsing, the large board's exact counts and ordering, and the malformed board's fail-fast plus its byte fidelity after a refused load. Those run anywhere — no display, no permission — and they are what stops a fixture that quietly came out wrong from turning a UI run into a pass over the wrong board.

Running the suites

The whole UI bundle, accessibility audits included:

xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
  -destination 'platform=macOS,arch=arm64' \
  -only-testing:KanbanUITests

The golden flows on their own — the fastest useful run, and the one to reach for after a change to a command, a focused value, or the store's write path:

xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
  -destination 'platform=macOS,arch=arm64' \
  -only-testing:KanbanUITests/EndToEndFlowTests

Fail-fast on a malformed board:

xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
  -destination 'platform=macOS,arch=arm64' \
  -only-testing:KanbanUITests/FailFastLaunchTests

The performance pass, which is slow — each measured iteration builds the 320-card board from scratch — and is therefore worth invoking on its own rather than leaving in a routine run:

xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
  -destination 'platform=macOS,arch=arm64' \
  -only-testing:KanbanUITests/LargeBoardPerformanceTests

To check only that the suites still compile — which is all that can be done on a machine without a display or without automation permission:

xcodebuild build-for-testing -project Kanban.xcodeproj -scheme Kanban \
  -destination 'platform=macOS,arch=arm64'

The UI bundle is base-only by design (project.yml): these tests launch the real app and drive its menu bar, and a second copy would double the slowest part of the suite to re-assert something edition-blind. The Pro app's own launch is already exercised on every LaneworkPro unit run, which uses it as the test host.

The flow inventory

EndToEndFlowTests.swift, on the standard fixture. Every flow asserts on lane labels — a lane's accessibility label is "⟨title⟩, lane, N cards" and the count is the rendered one, so one assertion covers the visible badge, the spoken count and the layout at once.

Test	Flow	Driven by	Asserted by
testCreateCard	Create a card	File ▸ New Card, type, Return	The new card's element, and To Do at 4 cards
testCreateLane	Create a lane	File ▸ New Lane	An "Untitled, lane, 0 cards" container; the other three lanes unchanged
testRenameCardInline	Inline rename	Board ▸ Rename, ⌘A, type, Return	The new title's element; the lane count unchanged
testDragCardBetweenLanes	Drag a card across lanes	XCUICoordinate press + drag + hold	Both lanes' counts
testCutAndPasteCardAcrossLanes	Cut/paste across lanes	Edit ▸ Cut, select, Edit ▸ Paste	Both lanes' counts
testUndoAndRedoOfAMove	Undo and redo a move	⌘Z / ⇧⌘Z after a cut/paste	Counts back, then forward again
testDeleteToTrashAndRestore	Delete, show trash, restore	File ▸ Delete (no confirmation), View ▸ Show Trash, Edit ▸ Cut in the trash, Edit ▸ Paste into a lane	Lane counts and the trash column's value, both directions
testEmptyTrash	Empty Trash	View ▸ Show Trash, File ▸ Empty Trash…, confirm	The alert's sentence and both buttons; the trash at 0 cards
testTemplateChooserUpToTheSavePanel	Template chooser	File ▸ New Board…, select Basic, read Choose, Cancel	The chooser window, the Basic tile, Choose enabled — and no further (see below)

FailFastLaunchTests.swift, on the malformed fixture:

Test	Claim
testMalformedBoardFailsLoudlyAndOpensNoWindow	Welcome appears; no board window; a welcome row carries the loader's own sentence, naming index.md and saying "unparseable YAML"
testMalformedBoardIsNeverRepaired	The malformed bytes are unchanged on disk (where the runner can read the container), and a relaunch is refused identically

The trash grammar these flows encode is the current one: delete moves a card into <root>/.trash/; ⌘⌫ stages by place (board → trash, no confirmation; trash → permanent, with one); ⌥⌘⌫ skips the trash from anywhere, with a confirmation; restore is an ordinary move out — a drag, or ⌘X in the trash and ⌘V into a lane. There is no Put Back.

Two things this pass deliberately does not cover, because they are not base's: any git flow (commit, pull, push — pro-m1's, and base has no git operations at all; that a .git folder is inert in base is pinned by KanbanTests/InertGitTests.swift), and the accessibility audits, which are AccessibilityVerification.md's.

Manual-only flows

Three, each with a reason it is not automated and a note on where its logic is covered.

Instantiating a template (File ▸ New Board… ▸ Choose). Choose runs an NSSavePanel, which in a sandboxed app is Powerbox — a separate, system-owned process (com.apple.appkit.xpc.openAndSavePanelService). Driving it means driving another application's UI, on a surface with no accessibility contract of its own that Apple changes between releases. A test that did it would fail for reasons that are not this app's, which is worse than no test: a flaky gate teaches people to re-run rather than to read. The automated flow stops at the panel and asserts everything before it. Check by hand, per release: File ▸ New Board…, pick a template, Choose, accept the suggested name, and confirm the new board opens in its own window with the template's lanes and starter cards, and that its row is on welcome afterwards. Everything behind the panel — the copy, the template: key, the collision ladder, cancellation — is KanbanTests/TemplateEngineTests.swift.

File ▸ Duplicate's save-panel fallback. The silent Finder-style sibling is attempted first and only a permission refusal raises the panel, so the automated path cannot reach the panel without first arranging an unwritable parent folder — and once it does, the panel is Powerbox again. Check by hand when the duplicate path changes: open a board inside a folder the app has no write grant for, File ▸ Duplicate, and confirm the panel opens pre-filled with the parent folder and the "copy" name, that Cancel says nothing at all, and that choosing a location completes the duplicate.

File ▸ Open… (⌘O). The open panel is Powerbox for the same reason. The whole fixture mechanism exists precisely because a sandboxed app cannot be handed a path, so there is nothing here to automate. Check by hand by opening a board from the Finder and from ⌘O, and confirming both land on the same window when the board is already open.

The performance budgets

LargeBoardPerformanceTests.swift, on the large fixture (8 × 40 = 320 cards).

Each test does two things, and they do different jobs. The measure block records the metric, so the test report carries a distribution and Xcode can offer a local baseline. The explicit wall-clock XCTAssertLessThan is the gate: XCTest baselines are stored per machine in the project's xcshareddata and do not travel, so on a fresh clone measure alone would record a number and pass unconditionally — a log line, not a gate.

Budget	Value	Covers
launchBudget	30 s	Process start, materializing 328 folders through BoardWriter, the load, and the first frame with a lane on it
interactionBudget	5 s	View ▸ Show Trash on a 320-card board: the re-divide, every lane's reflow, and the column arriving

They are regression bounds, not targets — roughly 5× the measured cost on the development machine. What they have to catch is a tenfold change (an O(n²) reflow, a synchronous tree walk on the main thread, a per-card watcher); what they must never do is fail because a laptop was busy. The launch budget's dominant term is the fixture build rather than the app: writing an index.md per card takes about two seconds on the development machine, which KanbanTests ▸ UITestLargeFixtureBoardTests measures without a window. That is deliberate — the number stays honest about what a launch on this fixture actually costs — but it is why the budget is loose, and why a launch regression shows up here as a big move rather than a small one.

Tighten a budget only with a measurement in hand, and record which machine it was measured on.

When something fails

Read which line failed before reading the test:

• A clickMenuItem failure is a validation failure. The row was missing or disabled, which means a focused value did not reach it — a focusedSceneValue that stopped being published, a command scoped to the wrong window, a predicate that grew a clause.
• An awaitLane / awaitTrash failure is the flow itself. The command ran and the board did not end up where the design says. This is the interesting kind.
• A waitForExistence on a card or an editor is navigation, in the test file: the fixture's shape and the test's idea of it have drifted.
• testDragCardBetweenLanes failing alone is worth one re-run before it is believed. It is the suite's only real AppKit dragging session and the only gesture whose success depends on timing thresholds rather than on state.
• Everything failing on the first click is automation permission, not the app.

File anything that fails on the Redesign board, with the test name it came from.