rzen/lanework
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Build the end-to-end UI test pass

Browse Source 
The golden-path XCUITest suite, adapted to current rulings where the
card body had gone stale: trash flows follow the materialized-trash
grammar (no Put Back, restore is drag or cut/paste out), git flows are
pro-m1 scope and skipped, and fixtures extend m11's in-container
--ui-test-fixture-board mechanism (the sandbox forbids the card's
--open-board path handoff) with exact-match variant flags: standard
(the three-lane audit board), large (8 lanes x 40 cards for
masonry/reflow), malformed (BoardWriter-built board with one card's
index.md overwritten to unterminated YAML, opened through the ORDINARY
path so the failure is the loader's own).

EndToEndFlowTests: create card/lane, inline rename, coordinate drag
across lanes, cut/paste, undo/redo of a move, delete-to-trash /
show-trash / restore-by-cut-paste / Empty Trash confirm - all asserting
on lane accessibility labels. FailFastLaunchTests: welcome appears, no
board window ever, a welcome row carries the loader's sentence naming
the file; byte-fidelity of the malformed board pinned unconditionally
in KanbanTests plus an identically-refused relaunch. Performance:
launch metric plus explicit wall-clock gates (30s launch / 5s Show
Trash on 320 cards) since XCTest baselines don't travel. Powerbox
panels (template save panel, Duplicate fallback, Open) are documented
as manual in EndToEndVerification.md, not faked.

The smoke test now launches on the standard fixture (it launched bare
before, opening the developer's real boards); README's everyday test
command scopes to -only-testing:KanbanTests.

Suite compiles on both schemes (build-for-testing verified); flows
await a real display + automation permission to execute - run
instructions in KanbanUITests/EndToEndVerification.md. +8 unit tests;
1669 green both schemes.

Claude-Session: https://claude.ai/code/session_01SR4XGjmBE16ZUYWpfFHXwY
This commit is contained in:
rzen
2026-07-29 13:40:18 -04:00
parent 71b112d04c
commit f34707e17e
13 changed files with 1627 additions and 133 deletions
Download Patch File Download Diff File Expand all files Collapse all files
KanbanUITests/AccessibilityAuditTests.swift
+3 -98
View File
@@ -35,7 +35,9 @@ import XCTest
/// Every test launches with `UITestLaunch.fixtureFlag`, which makes the app build a known board
/// inside its own container and open it (see `UITestLaunch` for why the board cannot simply be
/// handed over on the command line — the sandbox). The board is three lanes, six cards, one card
/// with a rich Markdown body and an attachment, and one card already in `.trash/`.
/// with a rich Markdown body and an attachment, and one card already in `.trash/`. That is the
/// `standard` fixture variant; the launch helpers and the other two shapes live in
/// `UITestSupport.swift`.
///
/// ### Running these
///
@@ -165,100 +167,3 @@ final class AccessibilityAuditTests: XCTestCase {
try app.performAccessibilityAudit()
}
}
// MARK: - Driving the app
extension XCUIApplication {
/// One timeout for the whole suite. Generous, because a cold launch here also builds a board on
/// disk, and because a UI test that is slow is not a UI test that is wrong.
static let uiTimeout: TimeInterval = 30
/// The first element anywhere in the app carrying `label`, whatever type it turned out to be.
///
/// SwiftUI decides for itself which `XCUIElement.ElementType` a flattened accessibility element
/// lands as — a card face and a lane container are both "one element" by design and neither is
/// promised to be a `staticText` — so the waits in this file match on the label, which *is*
/// specified (`AccessibilityPhrases`), rather than on a type that is not.
@MainActor
func element(labeled label: String) -> XCUIElement {
descendants(matching: .any)
.matching(NSPredicate(format: "label == %@", label))
.firstMatch
}
/// Launches the app on the audit fixture board and waits for its window.
///
/// The flag's spelling is `UITestLaunch.fixtureFlag` in the app target; it is written out here
/// because a UI test bundle does not link the app it drives — the two ends of a launch argument
/// are always a pair of literals, and this is the one place this end of the pair is spelled.
@MainActor
static func launchedWithFixtureBoard() -> XCUIApplication {
let app = XCUIApplication()
app.launchArguments += ["--ui-test-fixture-board"]
app.launch()
XCTAssertTrue(
app.wait(for: .runningForeground, timeout: uiTimeout),
"the app did not reach the foreground"
)
// `UITestLaunch.boardTitle`, which is also the window title (`BoardWindowHost.windowTitle`).
// Waiting on *this* window rather than on `windows.firstMatch` is what makes a failed fixture
// build a failure here instead of an audit that quietly passed over the welcome window.
XCTAssertTrue(
app.windows["Audit Board"].waitForExistence(timeout: uiTimeout),
"the fixture board window did not open — check the app's launch log for a fixture build failure"
)
return app
}
/// Clicks a menu-bar row by its title.
///
/// **By title, because the titles are API** (04-interactions.md ▸ Configurable bindings: menu
/// items are remapped by title through `NSUserKeyEquivalents`, so the strings in
/// `KanbanApp.menuCommands` are already a published contract). Driving the menus rather than
/// typing the chords also means these tests exercise the same path a keyboard user takes, and a
/// row that silently disabled itself fails here as an un-hittable element rather than as a
/// keystroke that went nowhere.
@MainActor
func clickMenuItem(_ title: String, in menu: String) {
let bar = menuBars.firstMatch
let menuBarItem = bar.menuBarItems[menu]
XCTAssertTrue(
menuBarItem.waitForExistence(timeout: Self.uiTimeout),
"the \(menu) menu is missing from the menu bar"
)
menuBarItem.click()
let item = bar.menuItems[title]
XCTAssertTrue(
item.waitForExistence(timeout: Self.uiTimeout),
"\(menu) ▸ \(title) is missing"
)
XCTAssertTrue(item.isEnabled, "\(menu) ▸ \(title) is disabled")
item.click()
}
/// Selects the fixture's rich card and opens its window.
///
/// **The selection is seeded with one arrow press**, which is the app's own documented rule:
/// "an empty selection seeds at the first lane's first card" (04-interactions.md ▸ Grammar,
/// `BoardView.seed`). From there the walk to the rich card is ordinary arrow navigation — the
/// same keys a user without a pointer would press — so this helper needs to know no identifiers
/// and no geometry, only the fixture's shape.
///
/// The rich card is the sole card of lane 2 ("Doing"), so: one press to seed into lane 1's first
/// card, then one `→` to step into lane 2. `→` steps to the nearest card in that direction, and
/// with one card in the lane there is only one it can be.
@MainActor
func openRichCardWindow() throws {
typeKey(.downArrow, modifierFlags: [])
typeKey(.rightArrow, modifierFlags: [])
clickMenuItem("Open Card", in: "Board")
// `UITestLaunch.cardTitles[1][0]`, which is the card window's title
// (`CardWindowHost.windowTitle`).
XCTAssertTrue(
windows["Write the smoke script"].waitForExistence(timeout: Self.uiTimeout),
"the card window did not open"
)
}
}
KanbanUITests/AccessibilityVerification.md
+3 -1
View File
@@ -4,6 +4,8 @@ The whole of DESIGN/10-accessibility.md § Verification, in one place: the autom
It lives beside `AccessibilityAuditTests.swift` because the design says it should — "a manual VoiceOver smoke script lives with the test plan" — and because the automated and manual halves are one pass, not two.
This is the **accessibility** gate. The golden-path pass — the flows a user performs, the fail-fast launch, and the performance budgets — is `EndToEndVerification.md`, beside it, sharing this document's prerequisites and the same fixture mechanism.
## Contents
1. [Before you start](#before-you-start)
@@ -49,7 +51,7 @@ The eight surfaces, and how each test gets there:
| `testTemplateChooser` | Template chooser | File ▸ New Board… |
| `testBoardInfoPopover` | Board popover | File ▸ Board Info |
Every test launches the app with `--ui-test-fixture-board`, which makes the app build a known board inside its own container and open it — three lanes ("To Do", "Doing", "Done"), six cards, one card with a rich Markdown body and an attachment, one card already in the trash. The board and the registry both live in a scratch directory that is wiped on every launch, so an audit run never touches your real boards or your recents list. See `Kanban/App/UITestLaunch.swift` for why the board cannot simply be handed to the app on the command line (the sandbox).
Every test launches the app with `--ui-test-fixture-board`, which makes the app build a known board inside its own container and open it — three lanes ("To Do", "Doing", "Done"), six cards, one card with a rich Markdown body and an attachment, one card already in the trash. That is the `standard` fixture variant; the bare flag means it, and the other two shapes (`large`, `malformed`) belong to the end-to-end pass. The board and the registry both live in a scratch directory that is wiped on every launch, so an audit run never touches your real boards or your recents list. See `Kanban/App/UITestLaunch.swift` for why the board cannot simply be handed to the app on the command line (the sandbox).
If a test fails, read the issue's `compactDescription` and fix the app. Adding a waiver is a design change and needs an entry on the Redesign board first.
KanbanUITests/EndToEndFlowTests.swift
+312
View File
@@ -0,0 +1,312 @@
import XCTest
/// **The golden paths, driven through the shipping app** — create, rename, move, cut and paste,
/// delete, restore, empty the trash, undo and redo.
///
/// ### What this suite is for, and what it is not
///
/// Every rule these flows exercise is already pinned somewhere in `KanbanTests`: the write paths by
/// `*WriteTests`, the grammar by `SelectionGrammarTests` and `KeyboardGrammarTests`, the trash by
/// `TrashModelTests` and `TrashWriteTests`, the stack by `UndoWriteTests`. Not one assertion below is
/// a fact those suites cannot state faster and more precisely.
///
/// What they cannot state is that the pieces are *wired together* — that File ▸ New Card reaches
/// `beginNewCard`, that the editor it opens is focused, that Return commits it, that the write lands,
/// that the watcher rounds it back, and that the lane the user is looking at now says "4 cards". This
/// suite is that sentence, once per flow, and nothing else. When one of these fails and the unit
/// suites stay green, the defect is in the wiring — a menu item scoped to the wrong focus value, a
/// command that lost its shortcut, a reload that never arrived.
///
/// ### How a flow asserts
///
/// **On lane labels**, almost always (`XCUIApplication.awaitLane`). A lane's accessibility label is
/// "⟨title⟩, lane, N cards" and the count is the *rendered* one, so one label assertion covers the
/// visible badge, the spoken count and the layout at once — 10-accessibility.md ▸ Trash lane makes
/// them one number, and this is where that pays. A card's own element carries only its title, which
/// does not change when the card changes lanes, so asserting on the card would assert nothing about
/// where it went.
///
/// The fixture is the `standard` board: three lanes — **To Do** (3 cards), **Doing** (1), **Done**
/// (1) — plus one card already in `.trash/`. Every flow below starts from exactly that and states
/// its own arithmetic against it.
///
/// ### Running these
///
/// They drive the real app through the real menu bar, so the machine running them must have granted
/// the test runner Accessibility control (System Settings ▸ Privacy & Security ▸ Accessibility) and
/// must not be locked or headless — the whole of `KanbanUITests/EndToEndVerification.md`'s
/// prerequisites section, which is where the run command lives too.
///
/// ### Reading a failure
///
/// A failure on a `clickMenuItem` is a *validation* failure: the row was missing or disabled, which
/// means a focused value did not reach it. A failure on an `awaitLane` is the flow itself: the
/// command ran and the board did not end up where the design says it should. A failure on a
/// `waitForExistence` for a card or an editor is navigation, in this file.
final class EndToEndFlowTests: XCTestCase {
override func setUp() {
super.setUp()
// A flow is a sequence: once a step has failed, every later assertion is about a board in a
// state nobody wrote down.
continueAfterFailure = false
}
// MARK: - Creating
/// **File ▸ New Card (⌘N)** — the placeholder opens focused in the target lane, Return commits,
/// and the card is on the board (04-interactions.md ▸ Grammar: "Return commits and re-selects the
/// lane").
///
/// The target is made explicit by selecting a card in "To Do" first. ⌘N with nothing selected
/// would also land there (`NewCardTarget.resolve` falls back to the first lane), but a flow that
/// relies on a fallback is a flow that would pass for the wrong reason.
@MainActor
func testCreateCard() throws {
let app = XCUIApplication.launchedWithFixtureBoard()
app.selectCard(StandardBoard.firstLaneCards[0])
app.clickMenuItem("New Card", in: "File")
app.typeIntoInlineEditor("A card the suite made", prompt: Phrase.cardTitlePrompt)
XCTAssertTrue(
app.element(labeled: "A card the suite made").waitForExistence(timeout: XCUIApplication.uiTimeout),
"the new card never appeared on the board"
)
app.awaitLane(StandardBoard.toDo, cards: 4, "the new card should have landed in To Do")
}
/// **File ▸ New Lane (⇧⌘N)** — a lane with no title and no editor (`BoardStore.createLane`), so
/// it arrives reading the untitled placeholder with an empty count.
///
/// The count is the assertion that matters: a new lane that arrived holding cards would mean the
/// create had landed somewhere it should not have.
@MainActor
func testCreateLane() throws {
let app = XCUIApplication.launchedWithFixtureBoard()
app.clickMenuItem("New Lane", in: "File")
app.awaitLane(Phrase.untitled, cards: 0, "File ▸ New Lane should add one empty, untitled lane")
// The three lanes that were already there are untouched — a create is not a re-divide of the
// board's contents.
app.awaitLane(StandardBoard.toDo, cards: 3)
app.awaitLane(StandardBoard.doing, cards: 1)
app.awaitLane(StandardBoard.done, cards: 1)
}
// MARK: - Renaming
/// **Board ▸ Rename** on a card — the inline editor, seeded with the current title, committed
/// with Return (04-interactions.md ▸ Selection; the menu row exists "for completeness and
/// remapping", the pointer path being a slow double click).
///
/// Select All then type is how the seeded title is replaced: inside a focused field that chord
/// belongs to the field editor, because the board's own Select All disables while an inline
/// editor is open.
@MainActor
func testRenameCardInline() throws {
let app = XCUIApplication.launchedWithFixtureBoard()
let original = StandardBoard.firstLaneCards[3]
let renamed = "Ship the audit, renamed"
app.selectCard(original)
app.clickMenuItem("Rename", in: "Board")
app.typeIntoInlineEditor(renamed, prompt: Phrase.cardTitlePrompt, replacingExisting: true)
XCTAssertTrue(
app.element(labeled: renamed).waitForExistence(timeout: XCUIApplication.uiTimeout),
"the renamed card never appeared"
)
// A rename is not a move: the lane still holds the same three cards.
app.awaitLane(StandardBoard.toDo, cards: 3)
}
// MARK: - Moving
/// **A pointer drag across lanes** (04-interactions.md ▸ Drag and drop) — the card leaves "To Do"
/// and lands in "Doing", which the two lanes' counts say and nothing else has to.
///
/// This is the suite's one gesture that goes through a real AppKit dragging session rather than
/// through a menu, and it is correspondingly the most environment-sensitive
/// (`XCUIApplication.dragCard` carries the details). It earns its place anyway: the drag is the
/// board's headline interaction, its reflow is the largest piece of geometry in the app, and no
/// unit test can press a mouse button.
@MainActor
func testDragCardBetweenLanes() throws {
let app = XCUIApplication.launchedWithFixtureBoard()
app.dragCard(StandardBoard.firstLaneCards[0], onto: StandardBoard.richCard)
app.awaitLane(StandardBoard.doing, cards: 2, "the dragged card should have landed in Doing")
app.awaitLane(StandardBoard.toDo, cards: 2, "the dragged card should have left To Do")
}
/// **Cut and paste across lanes (⌘X / ⌘V)** — the keyboard's own move (04-interactions.md
/// ▸ Clipboard), driven through the Edit menu because the board answers `cut:`/`paste:` as a
/// responder and the menu is where that wiring is visible.
///
/// The paste lands the card **after** the anchor card, which is why the destination is selected
/// by clicking a card in it rather than by clicking the lane.
@MainActor
func testCutAndPasteCardAcrossLanes() throws {
let app = XCUIApplication.launchedWithFixtureBoard()
app.selectCard(StandardBoard.firstLaneCards[0])
app.clickMenuItem("Cut", in: "Edit")
app.selectCard(StandardBoard.richCard)
app.clickMenuItem("Paste", in: "Edit")
app.awaitLane(StandardBoard.doing, cards: 2, "the cut card should have landed in Doing")
app.awaitLane(StandardBoard.toDo, cards: 2, "the cut card should have left To Do")
}
/// **⌘Z and ⇧⌘Z over a move** (13-native-undo.md ▸ Rules) — the chords rather than the menu rows,
/// because those rows are titled dynamically ("Undo Move Card") by `NSUndoManager` and a test
/// that spelled the title would be asserting the platform's composition rather than the app's
/// stack.
///
/// The move is the cut/paste above, so what undo has to put back is a folder that physically
/// changed lanes — the interesting case, and the one the native provider registers an inverse
/// for at the Writer boundary.
@MainActor
func testUndoAndRedoOfAMove() throws {
let app = XCUIApplication.launchedWithFixtureBoard()
app.selectCard(StandardBoard.firstLaneCards[0])
app.clickMenuItem("Cut", in: "Edit")
app.selectCard(StandardBoard.richCard)
app.clickMenuItem("Paste", in: "Edit")
app.awaitLane(StandardBoard.doing, cards: 2)
app.typeKey("z", modifierFlags: .command)
app.awaitLane(StandardBoard.toDo, cards: 3, "⌘Z should have returned the card to To Do")
app.awaitLane(StandardBoard.doing, cards: 1)
app.typeKey("z", modifierFlags: [.shift, .command])
app.awaitLane(StandardBoard.doing, cards: 2, "⇧⌘Z should have moved the card back into Doing")
app.awaitLane(StandardBoard.toDo, cards: 2)
}
// MARK: - The trash
/// **Delete, then restore** — the whole of the current trash grammar's board side
/// (03-board-ui.md § Trash, resettled 2026-07-28 for the materialized trash).
///
/// Four claims in one flow, because they are one story:
///
/// 1. **File ▸ Delete on a board card does not confirm.** The card moves into `<root>/.trash/`
/// and that is recoverable, so there is no alert — this flow would hang on one if there were.
/// 2. **View ▸ Show Trash** puts the column on the board as the last container, holding what was
/// already in it plus what just arrived.
/// 3. **Restore is an ordinary move out**: ⌘X in the trash, ⌘V into a lane. There is no Put Back
/// and there is nothing else to press.
/// 4. The counts on both sides move, both times.
@MainActor
func testDeleteToTrashAndRestore() throws {
let app = XCUIApplication.launchedWithFixtureBoard()
let doomed = StandardBoard.firstLaneCards[0]
app.selectCard(doomed)
app.clickMenuItem("Delete", in: "File")
app.awaitLane(StandardBoard.toDo, cards: 2, "the deleted card should have left the lane")
app.clickMenuItem("Show Trash", in: "View")
app.awaitTrash(cards: 2)
// The restore: cut in the trash, paste into a live lane.
app.selectCard(doomed)
app.clickMenuItem("Cut", in: "Edit")
app.selectCard(StandardBoard.doneCard)
app.clickMenuItem("Paste", in: "Edit")
app.awaitLane(StandardBoard.done, cards: 2, "the restored card should have landed in Done")
app.awaitTrash(cards: 1)
}
/// **File ▸ Empty Trash… (⇧⌘⌫)** — "the alert always appears" (03-board-ui.md § Trash), naming
/// the true count, and confirming it purges the container.
///
/// The trash has to be *shown* for the row to be enabled at all ("hidden, it is invisible to
/// every gesture"), which is the first click here and is itself part of the claim.
@MainActor
func testEmptyTrash() throws {
let app = XCUIApplication.launchedWithFixtureBoard()
app.clickMenuItem("Show Trash", in: "View")
app.awaitTrash(cards: 1)
app.clickMenuItem("Empty Trash…", in: "File")
// The prompt is `TrashModel.emptyTrashPrompt`: the count in the title, Delete and Cancel as
// the two answers. It is asserted rather than merely dismissed, because "the alert always
// appears" is the design rule under test.
//
// Queried app-wide rather than through `dialogs`/`sheets`, deliberately: which of those a
// SwiftUI `.alert` surfaces as on macOS is the framework's choice and not a contract, while
// the button titles and the sentence are `TrashConfirmations`' and `TrashModel`'s.
let confirm = app.buttons["Delete"]
XCTAssertTrue(
confirm.waitForExistence(timeout: XCUIApplication.uiTimeout),
"Empty Trash… did not raise its confirmation"
)
XCTAssertTrue(
app.element(labelContaining: "Permanently delete \(Phrase.cards(1))").exists,
"the confirmation did not name the trash's count"
)
XCTAssertTrue(app.buttons["Cancel"].exists, "the confirmation had no Cancel")
confirm.click()
app.awaitTrash(cards: 0)
}
// MARK: - Templates
/// **File ▸ New Board… (⌥⌘N) as far as this suite can drive it** — the chooser opens, the bundled
/// templates are listed and selectable, and Choose is live.
///
/// ### Why it stops there
///
/// Choose runs an `NSSavePanel`, and in a sandboxed app that panel is **Powerbox** — a separate,
/// system-owned process (`com.apple.appkit.xpc.openAndSavePanelService`). Driving it means
/// driving another application's UI from inside this one's test, on a surface Apple changes
/// between releases and which has no accessibility contract of its own. A test that did it would
/// fail for reasons that have nothing to do with this app, which is worse than not having the
/// test: a flaky gate teaches people to re-run rather than to read.
///
/// So the automated half ends at the panel, and **instantiating a template is a manual step** —
/// written down in `KanbanUITests/EndToEndVerification.md` ▸ Manual-only flows rather than faked
/// here. Everything behind the panel is unit-tested (`TemplateEngineTests` covers the copy, the
/// `template:` key, the collision ladder and the cancellation).
@MainActor
func testTemplateChooserUpToTheSavePanel() throws {
let app = XCUIApplication.launchedWithFixtureBoard()
app.clickMenuItem("New Board…", in: "File")
let chooser = app.windows["New Board"]
XCTAssertTrue(
chooser.waitForExistence(timeout: XCUIApplication.uiTimeout),
"the template chooser did not appear"
)
// "Basic" is the bundled tier's lowest `template.order`, so it is the row the chooser opens
// on — and a tile is a button to the accessibility tree, labeled by its name
// (10-accessibility.md ▸ Template chooser).
let basic = app.element(labeled: "Basic")
XCTAssertTrue(basic.waitForExistence(timeout: XCUIApplication.uiTimeout), "the Basic template is not listed")
basic.click()
let choose = chooser.buttons["Choose"]
XCTAssertTrue(choose.exists, "the chooser has no Choose button")
XCTAssertTrue(choose.isEnabled, "Choose is disabled on a loadable template")
// Cancel rather than Choose: pressing Choose hands the flow to Powerbox (see above). The
// chooser closing is the last thing this test can honestly assert.
chooser.buttons["Cancel"].click()
XCTAssertTrue(
chooser.waitForNonExistence(timeout: XCUIApplication.uiTimeout),
"Cancel did not dismiss the template chooser"
)
}
}
KanbanUITests/EndToEndVerification.md
+152
View File
@@ -0,0 +1,152 @@
# End-to-end verification
The golden-path UI pass: the flows a user actually performs, driven through the shipping app. It sits beside `AccessibilityVerification.md` — that document is the accessibility gate, this one is the wiring gate — and shares its prerequisites, because both drive the real app through the real menu bar.
Three suites, one fixture mechanism, and a short list of flows that stayed manual and say so.
## Contents
1. [Before you start](#before-you-start)
2. [The fixture boards](#the-fixture-boards)
3. [Running the suites](#running-the-suites)
4. [The flow inventory](#the-flow-inventory)
5. [Manual-only flows](#manual-only-flows)
6. [The performance budgets](#the-performance-budgets)
7. [When something fails](#when-something-fails)
## Before you start
Same prerequisites as the accessibility pass, and for the same reason — these tests move a real pointer and open real menus:
- A **real, unlocked display** and a real keyboard. Nothing here survives a headless session, a locked screen, or screen sharing.
- **Accessibility automation permission** for the test runner: System Settings ▸ Privacy & Security ▸ Accessibility. Without it every test fails on its first `click()`, for a reason that has nothing to do with the app. Xcode itself usually needs to be listed; so does the terminal, if you are running `xcodebuild` from one.
- Nothing else driving the pointer while a run is in progress.
A run that is interrupted can leave a stray `Lanework` process behind — `killall Lanework` between runs.
Everything a run writes goes into the app's sandbox container (`~/Library/Containers/dev.rzen.indie.Kanban/Data/tmp/LaneworkUITestFixture/`), which is wiped at the start of every launch. **A run never touches your real boards or your recents list**: the fixture flag redirects the board registry into the same scratch directory (`Kanban/App/UITestLaunch.swift`).
## The fixture boards
The app builds its own fixture, inside its own container, through its own `BoardWriter`. It cannot be handed a board on the command line: the sandbox grants `files.user-selected.read-write` and nothing else, so a path arriving as an argument is a path the app may not read. `UITestLaunch` carries the full argument.
Three variants, selected by a pair of launch arguments — the base flag, which means "this launch is synthetic", plus the variant's own:
| Variant | Launch arguments | Board | Window title |
| --- | --- | --- | --- |
| `standard` | `--ui-test-fixture-board --ui-test-fixture-standard` | 3 lanes (To Do 3 · Doing 1 · Done 1), one rich card with an attachment, one card in `.trash/` | `Audit Board` |
| `large` | `--ui-test-fixture-board --ui-test-fixture-large` | 8 lanes × 40 cards, titles cycling through four lengths so masonry has heights to balance | `Large Board` |
| `malformed` | `--ui-test-fixture-board --ui-test-fixture-malformed` | 2 lanes, 3 cards — one card's `index.md` overwritten with unparseable frontmatter | `Malformed Board` (never appears) |
`--ui-test-fixture-board` on its own still means `standard`, which is what the accessibility suite has always passed. The flags are exact-matched and double-dashed for the base flag's reasons: a launch switch with a fuzzy edge can be tripped by accident, and a `-key value` pair would be swallowed by `UserDefaults`' argument domain and become a preference.
Every variant is built through `BoardWriter` — `createBoard`, `createLane`, `createCard`, and for the standard board `writeBody`, `importAttachments` and `deleteCardToTrash`. The **one** exception is the malformed variant's final act, a raw `Data.write` over one card's `index.md`: no Writer call produces a file its own loader would reject, and none should.
The builders themselves are unit-tested (`KanbanTests/UITestLaunchTests.swift`): the flag and variant parsing, the large board's exact counts and ordering, and the malformed board's fail-fast plus its byte fidelity after a refused load. **Those run anywhere** — no display, no permission — and they are what stops a fixture that quietly came out wrong from turning a UI run into a pass over the wrong board.
## Running the suites
The whole UI bundle, accessibility audits included:
```
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
-destination 'platform=macOS,arch=arm64' \
-only-testing:KanbanUITests
```
The golden flows on their own — the fastest useful run, and the one to reach for after a change to a command, a focused value, or the store's write path:
```
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
-destination 'platform=macOS,arch=arm64' \
-only-testing:KanbanUITests/EndToEndFlowTests
```
Fail-fast on a malformed board:
```
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
-destination 'platform=macOS,arch=arm64' \
-only-testing:KanbanUITests/FailFastLaunchTests
```
The performance pass, which is slow — each measured iteration builds the 320-card board from scratch — and is therefore worth invoking on its own rather than leaving in a routine run:
```
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
-destination 'platform=macOS,arch=arm64' \
-only-testing:KanbanUITests/LargeBoardPerformanceTests
```
To check only that the suites still *compile* — which is all that can be done on a machine without a display or without automation permission:
```
xcodebuild build-for-testing -project Kanban.xcodeproj -scheme Kanban \
-destination 'platform=macOS,arch=arm64'
```
The UI bundle is base-only by design (`project.yml`): these tests launch the real app and drive its menu bar, and a second copy would double the slowest part of the suite to re-assert something edition-blind. The Pro app's own launch is already exercised on every `LaneworkPro` unit run, which uses it as the test host.
## The flow inventory
`EndToEndFlowTests.swift`, on the `standard` fixture. Every flow asserts on **lane labels** — a lane's accessibility label is "⟨title⟩, lane, N cards" and the count is the rendered one, so one assertion covers the visible badge, the spoken count and the layout at once.
| Test | Flow | Driven by | Asserted by |
| --- | --- | --- | --- |
| `testCreateCard` | Create a card | File ▸ New Card, type, Return | The new card's element, and To Do at 4 cards |
| `testCreateLane` | Create a lane | File ▸ New Lane | An "Untitled, lane, 0 cards" container; the other three lanes unchanged |
| `testRenameCardInline` | Inline rename | Board ▸ Rename, ⌘A, type, Return | The new title's element; the lane count unchanged |
| `testDragCardBetweenLanes` | Drag a card across lanes | `XCUICoordinate` press + drag + hold | Both lanes' counts |
| `testCutAndPasteCardAcrossLanes` | Cut/paste across lanes | Edit ▸ Cut, select, Edit ▸ Paste | Both lanes' counts |
| `testUndoAndRedoOfAMove` | Undo and redo a move | ⌘Z / ⇧⌘Z after a cut/paste | Counts back, then forward again |
| `testDeleteToTrashAndRestore` | Delete, show trash, restore | File ▸ Delete (no confirmation), View ▸ Show Trash, Edit ▸ Cut in the trash, Edit ▸ Paste into a lane | Lane counts and the trash column's value, both directions |
| `testEmptyTrash` | Empty Trash | View ▸ Show Trash, File ▸ Empty Trash…, confirm | The alert's sentence and both buttons; the trash at 0 cards |
| `testTemplateChooserUpToTheSavePanel` | Template chooser | File ▸ New Board…, select Basic, read Choose, Cancel | The chooser window, the Basic tile, Choose enabled — **and no further** (see below) |
`FailFastLaunchTests.swift`, on the `malformed` fixture:
| Test | Claim |
| --- | --- |
| `testMalformedBoardFailsLoudlyAndOpensNoWindow` | Welcome appears; **no** board window; a welcome row carries the loader's own sentence, naming `index.md` and saying "unparseable YAML" |
| `testMalformedBoardIsNeverRepaired` | The malformed bytes are unchanged on disk (where the runner can read the container), and a relaunch is refused identically |
The trash grammar these flows encode is the current one: delete moves a card into `<root>/.trash/`; ⌘⌫ stages by place (board → trash, no confirmation; trash → permanent, with one); ⌥⌘⌫ skips the trash from anywhere, with a confirmation; restore is an ordinary move out — a drag, or ⌘X in the trash and ⌘V into a lane. **There is no Put Back.**
Two things this pass deliberately does **not** cover, because they are not base's: any git flow (commit, pull, push — pro-m1's, and base has no git operations at all; that a `.git` folder is inert in base is pinned by `KanbanTests/InertGitTests.swift`), and the accessibility audits, which are `AccessibilityVerification.md`'s.
## Manual-only flows
Three, each with a reason it is not automated and a note on where its logic *is* covered.
**Instantiating a template (File ▸ New Board… ▸ Choose).** Choose runs an `NSSavePanel`, which in a sandboxed app is Powerbox — a separate, system-owned process (`com.apple.appkit.xpc.openAndSavePanelService`). Driving it means driving another application's UI, on a surface with no accessibility contract of its own that Apple changes between releases. A test that did it would fail for reasons that are not this app's, which is worse than no test: a flaky gate teaches people to re-run rather than to read. The automated flow stops at the panel and asserts everything before it. **Check by hand, per release:** File ▸ New Board…, pick a template, Choose, accept the suggested name, and confirm the new board opens in its own window with the template's lanes and starter cards, and that its row is on welcome afterwards. Everything behind the panel — the copy, the `template:` key, the collision ladder, cancellation — is `KanbanTests/TemplateEngineTests.swift`.
**File ▸ Duplicate's save-panel fallback.** The silent Finder-style sibling is attempted first and only a *permission refusal* raises the panel, so the automated path cannot reach the panel without first arranging an unwritable parent folder — and once it does, the panel is Powerbox again. **Check by hand** when the duplicate path changes: open a board inside a folder the app has no write grant for, File ▸ Duplicate, and confirm the panel opens pre-filled with the parent folder and the "copy" name, that Cancel says nothing at all, and that choosing a location completes the duplicate.
**File ▸ Open… (⌘O).** The open panel is Powerbox for the same reason. The whole fixture mechanism exists precisely because a sandboxed app cannot be handed a path, so there is nothing here to automate. **Check by hand** by opening a board from the Finder and from ⌘O, and confirming both land on the same window when the board is already open.
## The performance budgets
`LargeBoardPerformanceTests.swift`, on the `large` fixture (8 × 40 = 320 cards).
Each test does two things, and they do different jobs. The `measure` block **records** the metric, so the test report carries a distribution and Xcode can offer a local baseline. The explicit wall-clock `XCTAssertLessThan` is the **gate**: XCTest baselines are stored per machine in the project's `xcshareddata` and do not travel, so on a fresh clone `measure` alone would record a number and pass unconditionally — a log line, not a gate.
| Budget | Value | Covers |
| --- | --- | --- |
| `launchBudget` | 30 s | Process start, materializing 328 folders through `BoardWriter`, the load, and the first frame with a lane on it |
| `interactionBudget` | 5 s | View ▸ Show Trash on a 320-card board: the re-divide, every lane's reflow, and the column arriving |
They are **regression bounds, not targets** — roughly 5× the measured cost on the development machine. What they have to catch is a tenfold change (an O(n²) reflow, a synchronous tree walk on the main thread, a per-card watcher); what they must never do is fail because a laptop was busy. The launch budget's dominant term is the fixture *build* rather than the app: writing an `index.md` per card takes about two seconds on the development machine, which `KanbanTests ▸ UITestLargeFixtureBoardTests` measures without a window. That is deliberate — the number stays honest about what a launch on this fixture actually costs — but it is why the budget is loose, and why a launch regression shows up here as a big move rather than a small one.
Tighten a budget only with a measurement in hand, and record which machine it was measured on.
## When something fails
Read *which line* failed before reading the test:
- **A `clickMenuItem` failure is a validation failure.** The row was missing or disabled, which means a focused value did not reach it — a `focusedSceneValue` that stopped being published, a command scoped to the wrong window, a predicate that grew a clause.
- **An `awaitLane` / `awaitTrash` failure is the flow itself.** The command ran and the board did not end up where the design says. This is the interesting kind.
- **A `waitForExistence` on a card or an editor is navigation**, in the test file: the fixture's shape and the test's idea of it have drifted.
- **`testDragCardBetweenLanes` failing alone is worth one re-run before it is believed.** It is the suite's only real AppKit dragging session and the only gesture whose success depends on timing thresholds rather than on state.
- **Everything failing on the first click** is automation permission, not the app.
File anything that fails on the Redesign board, with the test name it came from.
KanbanUITests/FailFastLaunchTests.swift
+164
View File
@@ -0,0 +1,164 @@
import XCTest
/// **A board that will not load, from the outside** (01-storage-format.md § Malformed input;
/// 02-architecture.md § Launch and window lifecycle).
///
/// ### The claim
///
/// > A restored board that fails surfaces on welcome, row-level: its window doesn't open; welcome
/// > appears alongside whatever did restore, the failed board's recents row carrying fail-fast's
/// > specifics (load error) … never a silent drop.
///
/// The `malformed` fixture is a well-formed board with exactly one unparseable card `index.md`
/// (`UITestLaunch.malformedIndexText` — a frontmatter flow sequence that is never closed). Building
/// it succeeds; loading it must not, and *how* it fails is the whole of this file:
///
/// 1. **No board window.** Not an empty one, not one with the good lanes in it — fail-fast is
/// all-or-nothing, so a partial board on screen would be the worse failure.
/// 2. **Welcome, loudly.** The recents row for that board wears the loader's own sentence, which
/// names the offending file. A row that fell back to "Unavailable", or to a count, would be the
/// app declining to say what it found.
/// 3. **Nothing repaired.** The bytes on disk are the bytes the fixture wrote. The loader is a pure
/// function of the tree and writes nothing, ever — the Repair precedent — so a board it refused
/// must still be refusable, byte for byte.
///
/// ### Where each claim is checked
///
/// The first two are here, because they are about *windows* and a window is what a unit test does not
/// have. The third is checked **both** here and in `KanbanTests` — unconditionally there
/// (`UITestMalformedFixtureBoardTests`, which builds the fixture and re-reads the tree), and
/// opportunistically here, because reaching the app's container from the runner depends on how the
/// app under test was signed and installed. Where the container is not reachable this file says so
/// and leans on the unit suite rather than inventing a pass.
final class FailFastLaunchTests: XCTestCase {
override func setUp() {
super.setUp()
continueAfterFailure = false
}
/// The whole of claims 1 and 2, in one launch: no board window, welcome instead, and the row
/// carrying the loader's specifics.
@MainActor
func testMalformedBoardFailsLoudlyAndOpensNoWindow() throws {
let app = XCUIApplication.launched(with: .malformed)
// Welcome is where a failed open lands (`BoardWindowHost.start`: record the failure, refresh
// the recents, open welcome, dismiss the board window).
XCTAssertTrue(
app.windows["Welcome to Lanework"].waitForExistence(timeout: XCUIApplication.uiTimeout),
"the welcome window did not appear after a failed open"
)
// Claim 1. Checked *after* welcome has appeared, so this is "the board window never came",
// not "the board window has not come yet".
XCTAssertFalse(
app.windows[FixtureBoard.malformed.windowTitle].exists,
"a board window opened for a board the loader rejected"
)
// Claim 2. The row is one combined accessibility element — name, location, caption — and the
// caption is `BoardLoadError.description`: "⟨lane⟩/⟨card⟩/index.md: unparseable YAML at line
// N: …". The UUIDs in that path are minted at launch and unknowable here, so the assertion is
// on the parts that are the *app's* to keep stable: the offending file is named, and the
// reason is stated.
XCTAssertTrue(
app.element(labelContaining: "index.md").waitForExistence(timeout: XCUIApplication.uiTimeout),
"no welcome row named the offending index.md — fail-fast's specifics did not reach the surface"
)
XCTAssertTrue(
app.element(labelContaining: "unparseable YAML").exists,
"the welcome row did not say why the board was refused"
)
// And it is the malformed board's own row that says it.
XCTAssertTrue(
app.element(labelContaining: FixtureBoard.malformed.windowTitle).exists,
"the failure did not land on the failed board's row"
)
}
/// Claim 3, twice over: the malformed bytes survive the refusal, and a second launch is refused
/// the same way rather than opening a board the app quietly fixed.
///
/// The relaunch is not redundant with the byte check — it is what the byte check *means* from the
/// user's side, and it is the half that holds even where the container cannot be read.
@MainActor
func testMalformedBoardIsNeverRepaired() throws {
let app = XCUIApplication.launched(with: .malformed)
XCTAssertTrue(
app.windows["Welcome to Lanework"].waitForExistence(timeout: XCUIApplication.uiTimeout),
"the welcome window did not appear after a failed open"
)
// The bytes, where the runner can reach them. `NSTemporaryDirectory()` inside the sandboxed
// app resolves to its container, which an unsandboxed test runner can read — but only when
// the app under test is installed where this path expects, so a miss is reported rather than
// failed. `UITestMalformedFixtureBoardTests` makes the same claim unconditionally.
if let malformed = Self.malformedIndexOnDisk() {
XCTAssertTrue(
malformed.contains(Self.malformationMarker),
"the malformed index.md no longer carries its marker — something rewrote a file the loader refused to read"
)
XCTAssertTrue(
malformed.contains("order: [1024"),
"the malformed frontmatter was repaired — fail-fast must not write"
)
} else {
// Not a failure, and not silence either: the run says which half of the claim it made.
XCTContext.runActivity(named: "container not reachable from the runner") { _ in
print("""
The app's fixture scratch directory could not be read from the test runner, so \
the on-disk half of "nothing was repaired" was not checked here. It is pinned \
unconditionally by KanbanTests ▸ UITestMalformedFixtureBoardTests.
""")
}
}
// The relaunch. A fresh launch rebuilds the fixture from scratch (the scratch root is wiped
// per launch), so what this proves is the durable half: the app has no repair path that
// would make the second attempt succeed where the first failed.
app.terminate()
let second = XCUIApplication.launched(with: .malformed)
XCTAssertTrue(
second.windows["Welcome to Lanework"].waitForExistence(timeout: XCUIApplication.uiTimeout),
"the second launch did not reach welcome"
)
XCTAssertFalse(
second.windows[FixtureBoard.malformed.windowTitle].exists,
"the second launch opened the board the first one refused"
)
XCTAssertTrue(
second.element(labelContaining: "index.md").waitForExistence(timeout: XCUIApplication.uiTimeout),
"the second launch did not name the offending file"
)
}
// MARK: - Reading the app's container
/// The string that appears only in the malformed file — `UITestLaunch.malformationMarker`,
/// mirrored here for `FixtureBoard`'s reason.
private static let malformationMarker = "lanework-ui-test-malformed-fixture"
/// The malformed card's `index.md`, read from the app's sandbox container — or `nil` when the
/// runner cannot reach it.
///
/// The card folder's name is a UUID minted at launch, so the file is found by its content rather
/// than by its path: exactly one `index.md` under the fixture board carries the marker, which is
/// what the marker is for.
@MainActor
private static func malformedIndexOnDisk() -> String? {
let home = URL(fileURLWithPath: NSHomeDirectory(), isDirectory: true)
let board = home
.appendingPathComponent("Library/Containers/dev.rzen.indie.Kanban/Data/tmp", isDirectory: true)
.appendingPathComponent("LaneworkUITestFixture", isDirectory: true)
.appendingPathComponent("\(FixtureBoard.malformed.windowTitle).kanban", isDirectory: true)
guard let walker = FileManager.default.enumerator(atPath: board.path) else { return nil }
for case let relative as String in walker where relative.hasSuffix("index.md") {
guard let data = try? Data(contentsOf: board.appendingPathComponent(relative)) else { continue }
let text = String(decoding: data, as: UTF8.self)
if text.contains(malformationMarker) { return text }
}
return nil
}
}
KanbanUITests/KanbanUITests.swift
+13 -4
View File
@@ -1,11 +1,20 @@
import XCTest
/// The bundle's smoke test: the app launches and puts a window on screen.
///
/// It launches on the **standard fixture** rather than bare, which is a deliberate narrowing of what
/// it used to prove. A bare launch consults the registry's restore flags, so on a developer's machine
/// it opens that developer's real boards — real watchers over real documents, and a recents list
/// stamped by a test run. Every other test in this bundle goes out of its way to avoid exactly that
/// (`UITestLaunch` redirects the registry into a scratch directory for the same reason), and one test
/// quietly opting out would make "a UI run never touches your real boards" false.
///
/// What is given up is the launch-to-*welcome* path, which no longer has a UI test. It is not
/// untested: `LaunchPlanTests` pins the decision that produces it, and it is the path a developer
/// exercises every time they run the app from Xcode.
final class KanbanUITests: XCTestCase {
@MainActor
func testAppLaunches() throws {
let app = XCUIApplication()
app.launch()
XCTAssertTrue(app.wait(for: .runningForeground, timeout: 30))
XCTAssertTrue(app.windows.firstMatch.waitForExistence(timeout: 30))
_ = XCUIApplication.launchedWithFixtureBoard()
}
}
KanbanUITests/LargeBoardPerformanceTests.swift
+135
View File
@@ -0,0 +1,135 @@
import XCTest
/// **Performance sanity on a big board** — 8 lanes × 40 cards (`UITestLaunch.FixtureVariant.large`),
/// measured on launch and on one interaction.
///
/// ### Why the budgets are wall-clock assertions and not baselines
///
/// `XCTMeasureOptions` and `XCTPerformanceMetric` baselines are stored *per device* in the Xcode
/// project's `xcshareddata`, keyed by a machine identifier. They do not travel: a fresh clone, a
/// different Mac, or CI has no baseline at all, so `measure` there records a number and passes
/// unconditionally. A "performance test" that passes unconditionally is not a gate — it is a log
/// line — and this repository has no CI to grow one on.
///
/// So each test does both, and they do different jobs:
///
/// - **The `measure` block records the metric**, so a developer reading the test report can see the
/// distribution and Xcode can offer a baseline locally to whoever wants one.
/// - **The explicit wall-clock assertion is the gate.** It is a real bound, checked everywhere, with
/// no stored state behind it.
///
/// ### The budgets, and what they are budgets for
///
/// They are **generous on purpose** and they are *regression* bounds, not targets. What they have to
/// catch is a change that makes the large board an order of magnitude worse — an O(n²) reflow, a
/// synchronous tree walk on the main thread, a per-card watcher — and what they must never do is fail
/// because a laptop was busy. The numbers below are roughly 5× the measured cost on the development
/// machine, which leaves a slow or loaded machine plenty of room while still failing a tenfold
/// regression.
///
/// | Budget | Value | What it covers |
/// | --- | --- | --- |
/// | `launchBudget` | 30 s | Process start, **materializing 328 folders through `BoardWriter`**, the load, and the first frame with a lane on it |
/// | `interactionBudget` | 5 s | View ▸ Show Trash on a 320-card board: the re-divide, every lane's reflow, and the column arriving |
///
/// The launch budget's dominant term is the fixture *build*, not the app: writing an `index.md` per
/// card takes about two seconds on the development machine (`UITestLargeFixtureBoardTests` measures
/// the same work without a window). That is deliberate — the number stays honest about what a launch
/// on this fixture actually costs — but it is why the budget is not tighter, and why a launch
/// regression shows up here as a big move rather than a small one.
///
/// ### Running these
///
/// Like the rest of `KanbanUITests`: a real, unlocked display and Accessibility automation
/// permission. They are also the slowest tests in the repository — each one builds the large board
/// once per `measure` iteration — so they are excluded from the ordinary run and invoked by name.
/// `KanbanUITests/EndToEndVerification.md` carries the command.
final class LargeBoardPerformanceTests: XCTestCase {
/// Launch to a board window with the large fixture on it.
static let launchBudget: TimeInterval = 30
/// One board-wide interaction: View ▸ Show Trash, which re-divides every lane.
static let interactionBudget: TimeInterval = 5
override func setUp() {
super.setUp()
continueAfterFailure = false
}
/// **Launch-to-board**, measured with `XCTApplicationLaunchMetric` and gated by
/// `launchBudget`.
///
/// `.manuallyStart`, so the measured span is the launch and the window — not the teardown of the
/// previous iteration's app, which would otherwise be folded into the number.
///
/// The gate is timed separately, before the measure block, for two reasons: a `measure` body runs
/// five times by default and asserting inside it would report one failure per iteration, and the
/// first launch is the one a user experiences (the later ones benefit from a warm dyld cache and
/// a warm filesystem).
@MainActor
func testLargeBoardLaunchPerformance() throws {
let started = Date()
let first = XCUIApplication.launchedWithFixtureBoard(.large)
let elapsed = Date().timeIntervalSince(started)
first.terminate()
XCTAssertLessThan(
elapsed,
Self.launchBudget,
"launching onto the large fixture took \(String(format: "%.1f", elapsed))s, over the \(Self.launchBudget)s budget"
)
let options = XCTMeasureOptions()
options.invocationOptions = [.manuallyStart]
measure(metrics: [XCTApplicationLaunchMetric()], options: options) {
let app = XCUIApplication()
app.launchArguments += FixtureBoard.large.launchArguments
startMeasuring()
app.launch()
_ = app.windows[FixtureBoard.large.windowTitle].waitForExistence(timeout: XCUIApplication.uiTimeout)
stopMeasuring()
app.terminate()
}
}
/// **One interaction on a full board**: View ▸ Show Trash, which 03-board-ui.md § Trash calls "a
/// re-divide trigger, a lane add's behaviour exactly" — so every one of the eight lanes
/// recomputes its width and relays its 40 cards.
///
/// Chosen over select-all because it is *observable*: the column arriving is an element the test
/// can wait for, so the span measured is the whole interaction rather than the round trip of a
/// keystroke whose completion nothing announces. Select-all's cost is covered indirectly — it
/// runs inside the same window, over the same 320 cards, in `SelectionGrammarTests`.
///
/// The app is launched **once**, outside the measurement, and the toggle is measured over and
/// back: showing and hiding are the same re-divide in two directions, and a pair per iteration is
/// what leaves the board in the state the next iteration starts from.
@MainActor
func testLargeBoardShowTrashLatency() throws {
let app = XCUIApplication.launchedWithFixtureBoard(.large)
defer { app.terminate() }
// The first toggle is timed on its own and is the gate — it is the one that pays for whatever
// the board has not laid out yet, which is exactly the cost a user feels.
let started = Date()
app.clickMenuItem("Show Trash", in: "View")
XCTAssertTrue(
app.element(labeled: Phrase.trash).waitForExistence(timeout: XCUIApplication.uiTimeout),
"the trash column never appeared on the large board"
)
let elapsed = Date().timeIntervalSince(started)
XCTAssertLessThan(
elapsed,
Self.interactionBudget,
"showing the trash on the large board took \(String(format: "%.1f", elapsed))s, over the \(Self.interactionBudget)s budget"
)
// And the recorded metric, over the toggle in both directions.
measure(metrics: [XCTClockMetric()]) {
app.clickMenuItem("Show Trash", in: "View")
app.clickMenuItem("Show Trash", in: "View")
}
}
}
KanbanUITests/UITestSupport.swift
+343
View File
@@ -0,0 +1,343 @@
import XCTest
// MARK: - The fixture boards
/// **The three boards the app can build for a UI test**, mirrored from
/// `UITestLaunch.FixtureVariant`.
///
/// It is a *mirror* and not an import because a UI test bundle does not link the app it drives: the
/// two ends of a launch argument are always a pair of literals, and this is the one place this end of
/// the pair is spelled. The app end is `UITestLaunch.FixtureVariant.flag` / `.boardTitle`; a change
/// to either without the matching change here shows up as a launch that opens welcome and a wait that
/// times out, which is the loudest failure available for a string pair.
enum FixtureBoard {
/// The audit board — three lanes ("To Do", "Doing", "Done"), six cards, one with a rich Markdown
/// body and an attachment, one already in `.trash/`. The end-to-end flows use it too: every one
/// of them asserts on a lane's spoken card count, and a shape small enough to state in a sentence
/// is a shape those assertions can be read against.
case standard
/// 8 lanes × 40 cards, for the performance pass. Nothing in it is interesting; there is simply a
/// lot of it.
case large
/// A well-formed board with exactly one unparseable card `index.md`, for the fail-fast pass. It
/// is the only variant whose board window is *expected* never to appear.
case malformed
/// `UITestLaunch.fixtureFlag` plus this variant's own flag. Both, always: the first is what "this
/// launch is synthetic" means (it is what redirects the registry into the scratch directory), the
/// second is which board.
var launchArguments: [String] {
switch self {
case .standard: ["--ui-test-fixture-board", "--ui-test-fixture-standard"]
case .large: ["--ui-test-fixture-board", "--ui-test-fixture-large"]
case .malformed: ["--ui-test-fixture-board", "--ui-test-fixture-malformed"]
}
}
/// The board's title, which is also its window title (`BoardWindowHost.windowTitle`) and its
/// folder name.
var windowTitle: String {
switch self {
case .standard: "Audit Board"
case .large: "Large Board"
case .malformed: "Malformed Board"
}
}
}
// MARK: - The standard board's shape
/// The audit/end-to-end board as the tests address it: lane titles, card titles, and the labels
/// those produce.
///
/// Mirrored from `UITestLaunch` for `FixtureBoard`'s reason, and stated as *names* rather than
/// scattered string literals so that a test reads as a sentence about a board rather than as a
/// sentence about strings.
enum StandardBoard {
static let toDo = "To Do"
static let doing = "Doing"
static let done = "Done"
/// The first lane's cards, in card order — minus `trashed`, which the fixture deleted on the way
/// out, so the lane renders three of these four.
static let firstLaneCards = [
"Draft the release notes", "Check the trash grammar", "Confirm the rotor jumps", "Ship the audit",
]
/// The card the fixture deleted into `.trash/` — the trash column's one occupant at launch.
static let trashed = "Check the trash grammar"
/// The sole card of "Doing" — the rich one, and the destination every cross-lane flow moves into.
static let richCard = "Write the smoke script"
/// The sole card of "Done".
static let doneCard = "Wire the launch fixture"
}
// MARK: - Phrases
/// The spoken vocabulary the board's elements wear, mirrored from `AccessibilityPhrases`.
///
/// The board's containers are addressed by these labels and nothing else — a lane's element *is*
/// "⟨title⟩, lane, N cards" — which makes a lane's card count assertable without any identifier of
/// our own. That is deliberate: the count is the one thing every one of these flows changes, and
/// asserting on the label asserts the visible badge, the spoken count and the layout all at once,
/// because 10-accessibility.md makes them one number.
enum Phrase {
/// "3 cards", "1 card" — the app's one plural folding for a card count.
static func cards(_ count: Int) -> String {
"\(count) card\(count == 1 ? "" : "s")"
}
/// A lane container's label: "⟨title⟩, lane, N cards".
static func lane(_ title: String, cards count: Int) -> String {
"\(title), lane, \(cards(count))"
}
/// The trash container's label. Its *value* is `cards(count)`; the label is the bare word.
static let trash = "Trash"
/// The untitled placeholder — what a lane created by File ▸ New Lane wears until it is renamed.
static let untitled = "Untitled"
/// The inline title editor's placeholders (`NewCardStubView`, `LaneView`, `CardFaceView`).
static let cardTitlePrompt = "Card title"
}
// MARK: - Driving the app
extension XCUIApplication {
/// One timeout for the whole suite. Generous, because a cold launch here also builds a board on
/// disk, and because a UI test that is slow is not a UI test that is wrong.
static let uiTimeout: TimeInterval = 30
/// The first element anywhere in the app carrying `label`, whatever type it turned out to be.
///
/// SwiftUI decides for itself which `XCUIElement.ElementType` a flattened accessibility element
/// lands as — a card face and a lane container are both "one element" by design and neither is
/// promised to be a `staticText` — so the waits in this file match on the label, which *is*
/// specified (`AccessibilityPhrases`), rather than on a type that is not.
@MainActor
func element(labeled label: String) -> XCUIElement {
descendants(matching: .any)
.matching(NSPredicate(format: "label == %@", label))
.firstMatch
}
/// The first element anywhere in the app whose label *contains* `fragment`.
///
/// Used only where the whole label is not the test's to predict — a welcome row combines a board
/// name, a location and a caption into one element, and the caption is the part under test.
@MainActor
func element(labelContaining fragment: String) -> XCUIElement {
descendants(matching: .any)
.matching(NSPredicate(format: "label CONTAINS %@", fragment))
.firstMatch
}
/// Launches the app on `board` — the audit fixture unless told otherwise — and waits for its
/// window.
///
/// **Not for `.malformed`**, which has no window to wait for: that variant is launched with
/// `launched(with:)` and the caller waits for welcome instead.
@MainActor
static func launchedWithFixtureBoard(_ board: FixtureBoard = .standard) -> XCUIApplication {
let app = launched(with: board)
// Waiting on *this* window rather than on `windows.firstMatch` is what makes a failed fixture
// build a failure here instead of an audit that quietly passed over the welcome window.
XCTAssertTrue(
app.windows[board.windowTitle].waitForExistence(timeout: uiTimeout),
"the \(board.windowTitle) window did not open — check the app's launch log for a fixture build failure"
)
return app
}
/// Launches the app on `board` and returns as soon as it is frontmost — no window assertion, so
/// the caller decides what should have appeared.
@MainActor
static func launched(with board: FixtureBoard) -> XCUIApplication {
let app = XCUIApplication()
app.launchArguments += board.launchArguments
app.launch()
XCTAssertTrue(
app.wait(for: .runningForeground, timeout: uiTimeout),
"the app did not reach the foreground"
)
return app
}
/// Clicks a menu-bar row by its title.
///
/// **By title, because the titles are API** (04-interactions.md ▸ Configurable bindings: menu
/// items are remapped by title through `NSUserKeyEquivalents`, so the strings in
/// `KanbanApp.menuCommands` are already a published contract). Driving the menus rather than
/// typing the chords also means these tests exercise the same path a keyboard user takes, and a
/// row that silently disabled itself fails here as an un-hittable element rather than as a
/// keystroke that went nowhere.
@MainActor
func clickMenuItem(_ title: String, in menu: String) {
let bar = menuBars.firstMatch
let menuBarItem = bar.menuBarItems[menu]
XCTAssertTrue(
menuBarItem.waitForExistence(timeout: Self.uiTimeout),
"the \(menu) menu is missing from the menu bar"
)
menuBarItem.click()
let item = bar.menuItems[title]
XCTAssertTrue(
item.waitForExistence(timeout: Self.uiTimeout),
"\(menu) ▸ \(title) is missing"
)
XCTAssertTrue(item.isEnabled, "\(menu) ▸ \(title) is disabled")
item.click()
}
/// Whether a menu row is currently enabled — opened, read, and closed again with Escape.
///
/// Its own helper because *asking* is a different act from *clicking*: the template-chooser flow
/// needs to know that Choose is live without pressing it, since pressing it raises a save panel
/// this suite cannot drive.
@MainActor
func menuItemIsEnabled(_ title: String, in menu: String) -> Bool {
let bar = menuBars.firstMatch
let menuBarItem = bar.menuBarItems[menu]
guard menuBarItem.waitForExistence(timeout: Self.uiTimeout) else { return false }
menuBarItem.click()
let item = bar.menuItems[title]
let enabled = item.waitForExistence(timeout: Self.uiTimeout) && item.isEnabled
typeKey(.escape, modifierFlags: [])
return enabled
}
/// Selects the fixture's rich card and opens its window.
///
/// **The selection is seeded with one arrow press**, which is the app's own documented rule:
/// "an empty selection seeds at the first lane's first card" (04-interactions.md ▸ Grammar,
/// `BoardView.seed`). From there the walk to the rich card is ordinary arrow navigation — the
/// same keys a user without a pointer would press — so this helper needs to know no identifiers
/// and no geometry, only the fixture's shape.
///
/// The rich card is the sole card of lane 2 ("Doing"), so: one press to seed into lane 1's first
/// card, then one `→` to step into lane 2. `→` steps to the nearest card in that direction, and
/// with one card in the lane there is only one it can be.
@MainActor
func openRichCardWindow() throws {
typeKey(.downArrow, modifierFlags: [])
typeKey(.rightArrow, modifierFlags: [])
clickMenuItem("Open Card", in: "Board")
XCTAssertTrue(
windows[StandardBoard.richCard].waitForExistence(timeout: Self.uiTimeout),
"the card window did not open"
)
}
// MARK: - Board assertions
/// Waits for a lane to be rendering exactly `count` cards, by its label.
///
/// **This is how every flow in the end-to-end suite asserts that a move landed.** A card's own
/// element carries only its title, which does not change when it changes lanes; the lane's does,
/// because the label carries the count. So "the card moved from To Do to Doing" is checkable as
/// two lane labels and needs no identifier, no geometry and no reading of the disk.
@MainActor
@discardableResult
func awaitLane(_ title: String, cards count: Int, _ message: String = "") -> Bool {
let label = Phrase.lane(title, cards: count)
let found = element(labeled: label).waitForExistence(timeout: Self.uiTimeout)
XCTAssertTrue(found, "expected a lane reading \"\(label)\"\(message.isEmpty ? "" : " — \(message)")")
return found
}
/// Waits for the trash column to be rendering exactly `count` cards.
///
/// The trash's *label* is the bare word (it never states a count — 03-board-ui.md § Trash keeps
/// the header's title stable), so the count lives in its value and this matches on the pair.
@MainActor
@discardableResult
func awaitTrash(cards count: Int) -> Bool {
let predicate = NSPredicate(
format: "label == %@ AND value == %@", Phrase.trash, Phrase.cards(count)
)
let element = descendants(matching: .any).matching(predicate).firstMatch
let found = element.waitForExistence(timeout: Self.uiTimeout)
XCTAssertTrue(found, "expected the trash column to hold \(Phrase.cards(count))")
return found
}
/// Clicks a card by its title — the ordinary single click, which selects it
/// (`BoardStore.click`).
@MainActor
func selectCard(_ title: String) {
let card = element(labeled: title)
XCTAssertTrue(card.waitForExistence(timeout: Self.uiTimeout), "the card \"\(title)\" is not on the board")
card.click()
}
/// Waits for an inline title editor and answers it — the shared body of every flow that types a
/// title (`InlineTitleField`).
///
/// The editor is born focused, so the typing goes to the app rather than to a located element;
/// what is located is only the *field*, and only so that a flow which never opened one fails here
/// rather than typing its title into the board's keyboard grammar.
///
/// **Matched by placeholder first, then by "the only text field on screen".** The board window's
/// search field is an `NSSearchField` and so is not a `textField` at all, which leaves the inline
/// editor as the sole candidate; the placeholder match is the precise one and the fallback is
/// what keeps a SwiftUI change to how prompts reach the accessibility tree from failing every
/// flow at once.
@MainActor
func typeIntoInlineEditor(_ text: String, prompt: String, replacingExisting: Bool = false) {
let byPlaceholder = textFields
.matching(NSPredicate(format: "placeholderValue == %@", prompt))
.firstMatch
if !byPlaceholder.waitForExistence(timeout: 5) {
XCTAssertTrue(
textFields.firstMatch.waitForExistence(timeout: Self.uiTimeout),
"the inline title editor (\(prompt)) did not appear"
)
}
if replacingExisting {
// The rename editor opens seeded with the current title; Select All inside a focused
// field is the field editor's, not the board's (the board's own Select All disables
// while an inline editor is open — 04-interactions.md ▸ Grammar).
typeKey("a", modifierFlags: .command)
}
typeText(text)
typeKey(.return, modifierFlags: [])
}
/// Drags one card onto another — the pointer path (04-interactions.md ▸ Drag and drop), which is
/// a **real AppKit dragging session**: `CardFaceView` starts it from `.onDrag` with an
/// `NSItemProvider`, and the lane strip answers with `onDrop` delegates.
///
/// **This is the most environment-sensitive gesture in the suite**, and deliberately the slowest.
/// The press has to outlast the click-versus-drag threshold before the session arms, the movement
/// has to be slow enough that the drop delegates see intermediate positions (the reflow is
/// computed from them), and the hold at the end has to outlast the drop animation. A failure here
/// is worth re-running once before it is believed.
@MainActor
func dragCard(_ title: String, onto destinationTitle: String) {
let source = element(labeled: title)
let destination = element(labeled: destinationTitle)
XCTAssertTrue(source.waitForExistence(timeout: Self.uiTimeout), "the card \"\(title)\" is not on the board")
XCTAssertTrue(
destination.waitForExistence(timeout: Self.uiTimeout),
"the drop target \"\(destinationTitle)\" is not on the board"
)
let centre = CGVector(dx: 0.5, dy: 0.5)
source.coordinate(withNormalizedOffset: centre).press(
forDuration: 1,
thenDragTo: destination.coordinate(withNormalizedOffset: centre),
withVelocity: .slow,
thenHoldForDuration: 1
)
}
}