lanework/KanbanUITests/EndToEndVerification.md

# End-to-end verification

The golden-path UI pass: the flows a user actually performs, driven through the shipping app. It sits beside `AccessibilityVerification.md` — that document is the accessibility gate, this one is the wiring gate — and shares its prerequisites, because both drive the real app through the real menu bar.

Three suites, one fixture mechanism, and a short list of flows that stayed manual and say so.

## Contents

1. [Before you start](#before-you-start)
2. [The fixture boards](#the-fixture-boards)
3. [Running the suites](#running-the-suites)
4. [The flow inventory](#the-flow-inventory)
5. [Manual-only flows](#manual-only-flows)
6. [The performance budgets](#the-performance-budgets)
7. [When something fails](#when-something-fails)

## Before you start

Same prerequisites as the accessibility pass, and for the same reason — these tests move a real pointer and open real menus:

- A **real, unlocked display** and a real keyboard. Nothing here survives a headless session, a locked screen, or screen sharing.
- **Accessibility automation permission** for the test runner: System Settings ▸ Privacy & Security ▸ Accessibility. Without it every test fails on its first `click()`, for a reason that has nothing to do with the app. Xcode itself usually needs to be listed; so does the terminal, if you are running `xcodebuild` from one.
- Nothing else driving the pointer while a run is in progress.

A run that is interrupted can leave a stray `Lanework` process behind — `killall Lanework` between runs.

Everything a run writes goes into the app's sandbox container (`~/Library/Containers/dev.rzen.indie.Kanban/Data/tmp/LaneworkUITestFixture/`), which is wiped at the start of every launch. **A run never touches your real boards or your recents list**: the fixture flag redirects the board registry into the same scratch directory (`Kanban/App/UITestLaunch.swift`).

## The fixture boards

The app builds its own fixture, inside its own container, through its own `BoardWriter`. It cannot be handed a board on the command line: the sandbox grants `files.user-selected.read-write` and nothing else, so a path arriving as an argument is a path the app may not read. `UITestLaunch` carries the full argument.

Three variants, selected by a pair of launch arguments — the base flag, which means "this launch is synthetic", plus the variant's own:

| Variant | Launch arguments | Board | Window title |
| --- | --- | --- | --- |
| `standard` | `--ui-test-fixture-board --ui-test-fixture-standard` | 3 lanes (To Do 3 · Doing 1 · Done 1), one rich card with an attachment, one card in `.trash/` | `Audit Board` |
| `large` | `--ui-test-fixture-board --ui-test-fixture-large` | 8 lanes × 40 cards, titles cycling through four lengths so masonry has heights to balance | `Large Board` |
| `malformed` | `--ui-test-fixture-board --ui-test-fixture-malformed` | 2 lanes, 3 cards — one card's `index.md` overwritten with unparseable frontmatter | `Malformed Board` (never appears) |

`--ui-test-fixture-board` on its own still means `standard`, which is what the accessibility suite has always passed. The flags are exact-matched and double-dashed for the base flag's reasons: a launch switch with a fuzzy edge can be tripped by accident, and a `-key value` pair would be swallowed by `UserDefaults`' argument domain and become a preference.

Every variant is built through `BoardWriter` — `createBoard`, `createLane`, `createCard`, and for the standard board `writeBody`, `importAttachments` and `deleteCardToTrash`. The **one** exception is the malformed variant's final act, a raw `Data.write` over one card's `index.md`: no Writer call produces a file its own loader would reject, and none should.

The builders themselves are unit-tested (`KanbanTests/UITestLaunchTests.swift`): the flag and variant parsing, the large board's exact counts and ordering, and the malformed board's fail-fast plus its byte fidelity after a refused load. **Those run anywhere** — no display, no permission — and they are what stops a fixture that quietly came out wrong from turning a UI run into a pass over the wrong board.

## Running the suites

The whole UI bundle, accessibility audits included:

```
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
  -destination 'platform=macOS,arch=arm64' \
  -only-testing:KanbanUITests
```

The golden flows on their own — the fastest useful run, and the one to reach for after a change to a command, a focused value, or the store's write path:

```
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
  -destination 'platform=macOS,arch=arm64' \
  -only-testing:KanbanUITests/EndToEndFlowTests
```

Fail-fast on a malformed board:

```
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
  -destination 'platform=macOS,arch=arm64' \
  -only-testing:KanbanUITests/FailFastLaunchTests
```

The performance pass, which is slow — each measured iteration builds the 320-card board from scratch — and is therefore worth invoking on its own rather than leaving in a routine run:

```
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
  -destination 'platform=macOS,arch=arm64' \
  -only-testing:KanbanUITests/LargeBoardPerformanceTests
```

To check only that the suites still *compile* — which is all that can be done on a machine without a display or without automation permission:

```
xcodebuild build-for-testing -project Kanban.xcodeproj -scheme Kanban \
  -destination 'platform=macOS,arch=arm64'
```

The UI bundle is base-only by design (`project.yml`): these tests launch the real app and drive its menu bar, and a second copy would double the slowest part of the suite to re-assert something edition-blind. The Pro app's own launch is already exercised on every `LaneworkPro` unit run, which uses it as the test host.

## The flow inventory

`EndToEndFlowTests.swift`, on the `standard` fixture. Every flow asserts on **lane labels** — a lane's accessibility label is "⟨title⟩, lane, N cards" and the count is the rendered one, so one assertion covers the visible badge, the spoken count and the layout at once.

| Test | Flow | Driven by | Asserted by |
| --- | --- | --- | --- |
| `testCreateCard` | Create a card | File ▸ New Card, type, Return | The new card's element, and To Do at 4 cards |
| `testCreateLane` | Create a lane | File ▸ New Lane | An "Untitled, lane, 0 cards" container; the other three lanes unchanged |
| `testRenameCardInline` | Inline rename | Board ▸ Rename, ⌘A, type, Return | The new title's element; the lane count unchanged |
| `testDragCardBetweenLanes` | Drag a card across lanes | `XCUICoordinate` press + drag + hold | Both lanes' counts |
| `testCutAndPasteCardAcrossLanes` | Cut/paste across lanes | Edit ▸ Cut, select, Edit ▸ Paste | Both lanes' counts |
| `testUndoAndRedoOfAMove` | Undo and redo a move | ⌘Z / ⇧⌘Z after a cut/paste | Counts back, then forward again |
| `testDeleteToTrashAndRestore` | Delete, show trash, restore | File ▸ Delete (no confirmation), View ▸ Show Trash, Edit ▸ Cut in the trash, Edit ▸ Paste into a lane | Lane counts and the trash column's value, both directions |
| `testEmptyTrash` | Empty Trash | View ▸ Show Trash, File ▸ Empty Trash…, confirm | The alert's sentence and both buttons; the trash at 0 cards |
| `testTemplateChooserUpToTheSavePanel` | Template chooser | File ▸ New Board…, select Basic, read Choose, Cancel | The chooser window, the Basic tile, Choose enabled — **and no further** (see below) |

`FailFastLaunchTests.swift`, on the `malformed` fixture:

| Test | Claim |
| --- | --- |
| `testMalformedBoardFailsLoudlyAndOpensNoWindow` | Welcome appears; **no** board window; a welcome row carries the loader's own sentence, naming `index.md` and saying "unparseable YAML" |
| `testMalformedBoardIsNeverRepaired` | The malformed bytes are unchanged on disk (where the runner can read the container), and a relaunch is refused identically |

The trash grammar these flows encode is the current one: delete moves a card into `<root>/.trash/`; ⌘⌫ stages by place (board → trash, no confirmation; trash → permanent, with one); ⌥⌘⌫ skips the trash from anywhere, with a confirmation; restore is an ordinary move out — a drag, or ⌘X in the trash and ⌘V into a lane. **There is no Put Back.**

Two things this pass deliberately does **not** cover, because they are not base's: any git flow (commit, pull, push — pro-m1's, and base has no git operations at all; that a `.git` folder is inert in base is pinned by `KanbanTests/InertGitTests.swift`), and the accessibility audits, which are `AccessibilityVerification.md`'s.

## Manual-only flows

Three, each with a reason it is not automated and a note on where its logic *is* covered.

**Instantiating a template (File ▸ New Board… ▸ Choose).** Choose runs an `NSSavePanel`, which in a sandboxed app is Powerbox — a separate, system-owned process (`com.apple.appkit.xpc.openAndSavePanelService`). Driving it means driving another application's UI, on a surface with no accessibility contract of its own that Apple changes between releases. A test that did it would fail for reasons that are not this app's, which is worse than no test: a flaky gate teaches people to re-run rather than to read. The automated flow stops at the panel and asserts everything before it. **Check by hand, per release:** File ▸ New Board…, pick a template, Choose, accept the suggested name, and confirm the new board opens in its own window with the template's lanes and starter cards, and that its row is on welcome afterwards. Everything behind the panel — the copy, the `template:` key, the collision ladder, cancellation — is `KanbanTests/TemplateEngineTests.swift`.

**File ▸ Duplicate's save-panel fallback.** The silent Finder-style sibling is attempted first and only a *permission refusal* raises the panel, so the automated path cannot reach the panel without first arranging an unwritable parent folder — and once it does, the panel is Powerbox again. **Check by hand** when the duplicate path changes: open a board inside a folder the app has no write grant for, File ▸ Duplicate, and confirm the panel opens pre-filled with the parent folder and the "copy" name, that Cancel says nothing at all, and that choosing a location completes the duplicate.

**File ▸ Open… (⌘O).** The open panel is Powerbox for the same reason. The whole fixture mechanism exists precisely because a sandboxed app cannot be handed a path, so there is nothing here to automate. **Check by hand** by opening a board from the Finder and from ⌘O, and confirming both land on the same window when the board is already open.

## The performance budgets

`LargeBoardPerformanceTests.swift`, on the `large` fixture (8 × 40 = 320 cards).

Each test does two things, and they do different jobs. The `measure` block **records** the metric, so the test report carries a distribution and Xcode can offer a local baseline. The explicit wall-clock `XCTAssertLessThan` is the **gate**: XCTest baselines are stored per machine in the project's `xcshareddata` and do not travel, so on a fresh clone `measure` alone would record a number and pass unconditionally — a log line, not a gate.

| Budget | Value | Covers |
| --- | --- | --- |
| `launchBudget` | 30 s | Process start, materializing 328 folders through `BoardWriter`, the load, and the first frame with a lane on it |
| `interactionBudget` | 5 s | View ▸ Show Trash on a 320-card board: the re-divide, every lane's reflow, and the column arriving |

They are **regression bounds, not targets** — roughly 5× the measured cost on the development machine. What they have to catch is a tenfold change (an O(n²) reflow, a synchronous tree walk on the main thread, a per-card watcher); what they must never do is fail because a laptop was busy. The launch budget's dominant term is the fixture *build* rather than the app: writing an `index.md` per card takes about two seconds on the development machine, which `KanbanTests ▸ UITestLargeFixtureBoardTests` measures without a window. That is deliberate — the number stays honest about what a launch on this fixture actually costs — but it is why the budget is loose, and why a launch regression shows up here as a big move rather than a small one.

Tighten a budget only with a measurement in hand, and record which machine it was measured on.

## When something fails

Read *which line* failed before reading the test:

- **A `clickMenuItem` failure is a validation failure.** The row was missing or disabled, which means a focused value did not reach it — a `focusedSceneValue` that stopped being published, a command scoped to the wrong window, a predicate that grew a clause.
- **An `awaitLane` / `awaitTrash` failure is the flow itself.** The command ran and the board did not end up where the design says. This is the interesting kind.
- **A `waitForExistence` on a card or an editor is navigation**, in the test file: the fixture's shape and the test's idea of it have drifted.
- **`testDragCardBetweenLanes` failing alone is worth one re-run before it is believed.** It is the suite's only real AppKit dragging session and the only gesture whose success depends on timing thresholds rather than on state.
- **Everything failing on the first click** is automation permission, not the app.

File anything that fails on the Redesign board, with the test name it came from.