Build the end-to-end UI test pass
The golden-path XCUITest suite, adapted to current rulings where the card body had gone stale: trash flows follow the materialized-trash grammar (no Put Back, restore is drag or cut/paste out), git flows are pro-m1 scope and skipped, and fixtures extend m11's in-container --ui-test-fixture-board mechanism (the sandbox forbids the card's --open-board path handoff) with exact-match variant flags: standard (the three-lane audit board), large (8 lanes x 40 cards for masonry/reflow), malformed (BoardWriter-built board with one card's index.md overwritten to unterminated YAML, opened through the ORDINARY path so the failure is the loader's own). EndToEndFlowTests: create card/lane, inline rename, coordinate drag across lanes, cut/paste, undo/redo of a move, delete-to-trash / show-trash / restore-by-cut-paste / Empty Trash confirm - all asserting on lane accessibility labels. FailFastLaunchTests: welcome appears, no board window ever, a welcome row carries the loader's sentence naming the file; byte-fidelity of the malformed board pinned unconditionally in KanbanTests plus an identically-refused relaunch. Performance: launch metric plus explicit wall-clock gates (30s launch / 5s Show Trash on 320 cards) since XCTest baselines don't travel. Powerbox panels (template save panel, Duplicate fallback, Open) are documented as manual in EndToEndVerification.md, not faked. The smoke test now launches on the standard fixture (it launched bare before, opening the developer's real boards); README's everyday test command scopes to -only-testing:KanbanTests. Suite compiles on both schemes (build-for-testing verified); flows await a real display + automation permission to execute - run instructions in KanbanUITests/EndToEndVerification.md. +8 unit tests; 1669 green both schemes. Claude-Session: https://claude.ai/code/session_01SR4XGjmBE16ZUYWpfFHXwY
This commit is contained in:
@@ -100,20 +100,27 @@ struct RestoreBootstrapView: View {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The audit suite's board: built here, opened through the same `openBoard` every other path
|
/// The UI suites' board: built here, opened through the same `openBoard` every other path uses,
|
||||||
/// uses, so it registers, bookmarks and titles itself exactly like a board the user opened.
|
/// so it registers, bookmarks and titles itself exactly like a board the user opened.
|
||||||
///
|
///
|
||||||
/// **A failure lands on welcome as an ordinary launch failure**, with the fixture's own path on
|
/// **Which board is the launch arguments' to say** (`UITestLaunch.variant`), and this method does
|
||||||
/// it. That is deliberate: a suite whose fixture failed to build would otherwise audit an empty
|
/// not care: the malformed variant is built and opened exactly like the other two, and its
|
||||||
/// screen and pass, which is the one outcome an accessibility gate must never produce.
|
/// failure arrives one layer down as the *loader's* — a board window that records fail-fast's own
|
||||||
|
/// sentence and dismisses itself (`BoardWindowHost.start`). Special-casing it here would replace
|
||||||
|
/// the sentence under test with a sentence about the fixture.
|
||||||
|
///
|
||||||
|
/// **A failure to *build* lands on welcome as an ordinary launch failure**, with the fixture's own
|
||||||
|
/// path on it. That is deliberate: a suite whose fixture failed to build would otherwise audit an
|
||||||
|
/// empty screen and pass, which is the one outcome an accessibility gate must never produce.
|
||||||
private func openFixtureBoard() {
|
private func openFixtureBoard() {
|
||||||
|
let variant = UITestLaunch.variant
|
||||||
do {
|
do {
|
||||||
let url = try UITestLaunch.materializeFixtureBoard()
|
let url = try UITestLaunch.materializeFixtureBoard(variant)
|
||||||
appModel.openBoard(at: url)
|
appModel.openBoard(at: url)
|
||||||
} catch {
|
} catch {
|
||||||
Self.logger.error("the UI-test fixture board could not be built: \(error.localizedDescription, privacy: .public)")
|
Self.logger.error("the UI-test fixture board could not be built: \(error.localizedDescription, privacy: .public)")
|
||||||
appModel.recordLaunchFailure(
|
appModel.recordLaunchFailure(
|
||||||
path: UITestLaunch.fixtureBoardURL.path,
|
path: UITestLaunch.fixtureBoardURL(for: variant).path,
|
||||||
message: "The UI-test fixture board could not be built: \(error.localizedDescription)"
|
message: "The UI-test fixture board could not be built: \(error.localizedDescription)"
|
||||||
)
|
)
|
||||||
appModel.showWelcome()
|
appModel.showWelcome()
|
||||||
|
|||||||
+244
-16
@@ -52,10 +52,12 @@ enum LaunchPlan: Equatable, Sendable {
|
|||||||
|
|
||||||
// MARK: - UITestLaunch
|
// MARK: - UITestLaunch
|
||||||
|
|
||||||
/// **The accessibility audit suite's board**, and the launch argument that asks for it
|
/// **The UI suites' boards**, and the launch arguments that ask for them — the accessibility audit's
|
||||||
/// (10-accessibility.md ▸ Verification: "Xcode's accessibility audit … runs in UI tests over every
|
/// fixture (10-accessibility.md ▸ Verification: "Xcode's accessibility audit … runs in UI tests over
|
||||||
/// surface — board (trash shown and hidden), card window (Preview, Edit, raw source), welcome,
|
/// every surface — board (trash shown and hidden), card window (Preview, Edit, raw source), welcome,
|
||||||
/// template chooser, board popover").
|
/// template chooser, board popover") and, since the end-to-end pass, two more shapes that the audit
|
||||||
|
/// never needed: a **large** board for reflow and launch cost, and a **malformed** one whose only job
|
||||||
|
/// is to fail to load.
|
||||||
///
|
///
|
||||||
/// ### Why the app builds the board instead of being handed one
|
/// ### Why the app builds the board instead of being handed one
|
||||||
///
|
///
|
||||||
@@ -73,6 +75,12 @@ enum LaunchPlan: Equatable, Sendable {
|
|||||||
/// fixture is a board the app made, not a board a test file *believes* is well-formed. A format
|
/// fixture is a board the app made, not a board a test file *believes* is well-formed. A format
|
||||||
/// change that broke this would break the app first.
|
/// change that broke this would break the app first.
|
||||||
///
|
///
|
||||||
|
/// **The one deliberate exception is the malformed variant**, which builds its board through the very
|
||||||
|
/// same door and then overwrites exactly one card's `index.md` with raw bytes. That write is the
|
||||||
|
/// point of the variant — there is no Writer call that produces an unparseable file, and there should
|
||||||
|
/// not be one — and it happens *last*, so everything around the broken file is still a board the app
|
||||||
|
/// made.
|
||||||
|
///
|
||||||
/// ### It is inert without the flag
|
/// ### It is inert without the flag
|
||||||
///
|
///
|
||||||
/// Every entry point below is reached only from `LaunchPlan.uiTestFixture`, and that case is reached
|
/// Every entry point below is reached only from `LaunchPlan.uiTestFixture`, and that case is reached
|
||||||
@@ -102,20 +110,31 @@ enum UITestLaunch {
|
|||||||
|
|
||||||
// MARK: - The flag
|
// MARK: - The flag
|
||||||
|
|
||||||
/// The launch argument the audit suite passes (`KanbanUITests/AccessibilityAuditTests.swift`).
|
/// The launch argument every UI suite passes (`KanbanUITests/UITestSupport.swift`), on its own or
|
||||||
|
/// beside a variant flag. It means "this launch is synthetic" and nothing narrower — which board
|
||||||
|
/// is `FixtureVariant`'s to say.
|
||||||
///
|
///
|
||||||
/// `--`-prefixed on purpose: a single-dash `-key value` pair is swallowed by `UserDefaults`'
|
/// `--`-prefixed on purpose: a single-dash `-key value` pair is swallowed by `UserDefaults`'
|
||||||
/// `NSArgumentDomain` and would silently become a preference, which is precisely the kind of
|
/// `NSArgumentDomain` and would silently become a preference, which is precisely the kind of
|
||||||
/// side effect a test-only switch must not have.
|
/// side effect a test-only switch must not have.
|
||||||
static let fixtureFlag = "--ui-test-fixture-board"
|
static let fixtureFlag = "--ui-test-fixture-board"
|
||||||
|
|
||||||
/// Whether `arguments` asks for the fixture board — **pure**, so the rule is pinned by
|
/// Whether `arguments` asks for a fixture board — **pure**, so the rule is pinned by
|
||||||
/// `UITestLaunchTests` rather than by launching an app and looking.
|
/// `UITestLaunchTests` rather than by launching an app and looking.
|
||||||
///
|
///
|
||||||
/// Exact match, not a prefix: `--ui-test-fixture-boards-elsewhere` is not this flag, and a
|
/// Exact match, not a prefix: `--ui-test-fixture-boards-elsewhere` is not this flag, and a
|
||||||
/// `hasPrefix` check that accepted it would be a launch switch with a fuzzy edge.
|
/// `hasPrefix` check that accepted it would be a launch switch with a fuzzy edge. That is also
|
||||||
|
/// why the variants below wear *sibling* flags rather than a `=value` suffix or a
|
||||||
|
/// `--flag value` pair — a suffix would demand exactly the prefix matching this rules out, and a
|
||||||
|
/// pair is the shape `NSArgumentDomain` swallows.
|
||||||
|
///
|
||||||
|
/// **A variant flag on its own is enough**, which is belt over braces rather than a second
|
||||||
|
/// spelling: every call site passes `fixtureFlag` too (it is what "this launch is synthetic"
|
||||||
|
/// means), but a test bundle that passed only `--ui-test-fixture-large` must not get a launch
|
||||||
|
/// that reopens the developer's real boards into a scratch registry's blind spot.
|
||||||
static func isFixtureLaunch(arguments: [String]) -> Bool {
|
static func isFixtureLaunch(arguments: [String]) -> Bool {
|
||||||
arguments.contains(fixtureFlag)
|
arguments.contains(fixtureFlag)
|
||||||
|
|| FixtureVariant.allCases.contains { arguments.contains($0.flag) }
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The running process's answer to the same question.
|
/// The running process's answer to the same question.
|
||||||
@@ -123,6 +142,60 @@ enum UITestLaunch {
|
|||||||
isFixtureLaunch(arguments: ProcessInfo.processInfo.arguments)
|
isFixtureLaunch(arguments: ProcessInfo.processInfo.arguments)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MARK: - The variants
|
||||||
|
|
||||||
|
/// **Which fixture board a launch asks for.** Three shapes, because the three suites that consume
|
||||||
|
/// them are asking three different questions:
|
||||||
|
///
|
||||||
|
/// - `standard` — the audit's board (three lanes, six cards, a rich card, a trashed card). It is
|
||||||
|
/// also the end-to-end suite's board: every golden flow is expressed against a shape small
|
||||||
|
/// enough to state in a sentence, so an assertion about a lane's card count is readable.
|
||||||
|
/// - `large` — many lanes × many cards, for the masonry, the reflow, and the launch-cost
|
||||||
|
/// measurements. Nothing about it is *interesting*; the point is that there is a lot of it.
|
||||||
|
/// - `malformed` — a well-formed board with exactly one unparseable card `index.md`, for the
|
||||||
|
/// fail-fast pass (01-storage-format.md § Malformed input). It is the only variant whose
|
||||||
|
/// *successful* materialization is expected to produce a *failed* load.
|
||||||
|
///
|
||||||
|
/// The raw value is the flag's tail, so the flag and the case can never drift; the flag is
|
||||||
|
/// double-dashed for `fixtureFlag`'s reason and exact-matched for its reason too.
|
||||||
|
enum FixtureVariant: String, CaseIterable, Sendable {
|
||||||
|
case standard
|
||||||
|
case large
|
||||||
|
case malformed
|
||||||
|
|
||||||
|
/// The launch argument naming this variant. Paired with `fixtureFlag` at every call site.
|
||||||
|
var flag: String { "--ui-test-fixture-\(rawValue)" }
|
||||||
|
|
||||||
|
/// The board's title — and, through `fixtureBoardURL(for:)`, its folder name and its window
|
||||||
|
/// title, so a test can wait on `app.windows["Audit Board"]` and its neighbours.
|
||||||
|
///
|
||||||
|
/// Distinct per variant on purpose: a suite that waited on the wrong title would otherwise
|
||||||
|
/// pass against the wrong board, and the malformed variant's whole assertion is that *no*
|
||||||
|
/// window by its name ever appears.
|
||||||
|
var boardTitle: String {
|
||||||
|
switch self {
|
||||||
|
case .standard: UITestLaunch.boardTitle
|
||||||
|
case .large: "Large Board"
|
||||||
|
case .malformed: "Malformed Board"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The variant `arguments` asks for — `standard` when none is named, which is what the audit
|
||||||
|
/// suite's bare `--ui-test-fixture-board` has always meant.
|
||||||
|
///
|
||||||
|
/// **Declaration order breaks a tie**, so a launch naming two variants is deterministic rather
|
||||||
|
/// than dependent on argument order. Nothing produces that today; stating the rule is cheaper
|
||||||
|
/// than discovering it.
|
||||||
|
static func variant(arguments: [String]) -> FixtureVariant {
|
||||||
|
FixtureVariant.allCases.first { arguments.contains($0.flag) } ?? .standard
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The running process's answer to the same question.
|
||||||
|
static var variant: FixtureVariant {
|
||||||
|
variant(arguments: ProcessInfo.processInfo.arguments)
|
||||||
|
}
|
||||||
|
|
||||||
// MARK: - The scratch directory
|
// MARK: - The scratch directory
|
||||||
|
|
||||||
/// Everything a fixture launch writes, under one removable root inside the app's container.
|
/// Everything a fixture launch writes, under one removable root inside the app's container.
|
||||||
@@ -141,11 +214,20 @@ enum UITestLaunch {
|
|||||||
scratchRoot.appendingPathComponent("board-registry.json", isDirectory: false)
|
scratchRoot.appendingPathComponent("board-registry.json", isDirectory: false)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The fixture board's own folder. `.kanban`-suffixed because a board the app made through the
|
/// A fixture board's own folder. `.kanban`-suffixed because a board the app made through the
|
||||||
/// ordinary create path is a document, and the audit should be looking at the shape a user's
|
/// ordinary create path is a document, and the audit should be looking at the shape a user's
|
||||||
/// board actually has (01-storage-format.md § Document packaging).
|
/// board actually has (01-storage-format.md § Document packaging).
|
||||||
|
///
|
||||||
|
/// One folder per variant, all under the one scratch root: the root is wiped per launch anyway,
|
||||||
|
/// so the separation buys nothing at runtime — it buys a *name*, which is what a suite waits on.
|
||||||
|
static func fixtureBoardURL(for variant: FixtureVariant) -> URL {
|
||||||
|
scratchRoot.appendingPathComponent("\(variant.boardTitle).kanban", isDirectory: true)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The audit fixture's folder — `fixtureBoardURL(for: .standard)`, kept as a name because that
|
||||||
|
/// board is the one every caller predating the variants meant.
|
||||||
static var fixtureBoardURL: URL {
|
static var fixtureBoardURL: URL {
|
||||||
scratchRoot.appendingPathComponent("\(boardTitle).kanban", isDirectory: true)
|
fixtureBoardURL(for: .standard)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Wipes and recreates the scratch root, and answers the registry URL to build the app model
|
/// Wipes and recreates the scratch root, and answers the registry URL to build the app model
|
||||||
@@ -175,10 +257,10 @@ enum UITestLaunch {
|
|||||||
return registryStorageURL
|
return registryStorageURL
|
||||||
}
|
}
|
||||||
|
|
||||||
// MARK: - The fixture board's shape
|
// MARK: - The standard board's shape
|
||||||
|
|
||||||
/// The board's title — and, through `fixtureBoardURL`, its folder name and its window title, so
|
/// The standard board's title — and, through `fixtureBoardURL`, its folder name and its window
|
||||||
/// a test can wait on `app.windows["Audit Board"]`.
|
/// title, so a test can wait on `app.windows["Audit Board"]`.
|
||||||
static let boardTitle = "Audit Board"
|
static let boardTitle = "Audit Board"
|
||||||
|
|
||||||
/// The lane titles, in board order. Three because the tree the audit walks should have more than
|
/// The lane titles, in board order. Three because the tree the audit walks should have more than
|
||||||
@@ -255,18 +337,104 @@ enum UITestLaunch {
|
|||||||
/// and Reveal in Finder and never Open. An empty column audits its own label and stops there.
|
/// and Reveal in Finder and never Open. An empty column audits its own label and stops there.
|
||||||
static let trashedCardIndex = (lane: 0, card: 1)
|
static let trashedCardIndex = (lane: 0, card: 1)
|
||||||
|
|
||||||
|
// MARK: - The large board's shape
|
||||||
|
|
||||||
|
/// The large board's lanes and cards — **8 × 40**, which is 320 cards.
|
||||||
|
///
|
||||||
|
/// The numbers are a budget, not a maximum. They are large enough that the two things the large
|
||||||
|
/// board exists to exercise actually happen — a lane wide enough to lay out in several interior
|
||||||
|
/// masonry columns has plenty to lay out, and the board's reflow, scrolling and select-all all
|
||||||
|
/// have real work to do — and small enough that materializing it (a folder and an `index.md` per
|
||||||
|
/// card, through the ordinary Writer) stays in the seconds a UI test can afford. Raising them is
|
||||||
|
/// a decision about how long every performance run takes; they are stated here so that decision
|
||||||
|
/// is made in one place.
|
||||||
|
static let largeLaneCount = 8
|
||||||
|
static let largeCardsPerLane = 40
|
||||||
|
|
||||||
|
static func largeLaneTitle(_ index: Int) -> String {
|
||||||
|
"Lane \(index + 1)"
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A large-board card's title — unique across the whole board, and **deliberately of four
|
||||||
|
/// different lengths**.
|
||||||
|
///
|
||||||
|
/// A wall of identical one-line cards would lay out as a perfect grid, which is exactly the case
|
||||||
|
/// masonry has nothing to do. Cycling the length gives the layout genuinely different card
|
||||||
|
/// heights to balance, so a reflow measured against this board is measuring the work the real
|
||||||
|
/// algorithm does.
|
||||||
|
static func largeCardTitle(lane laneIndex: Int, card cardIndex: Int) -> String {
|
||||||
|
let base = "Card \(laneIndex + 1)-\(cardIndex + 1)"
|
||||||
|
let tail = String(repeating: " with a longer title that wraps", count: cardIndex % 4)
|
||||||
|
return base + tail
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - The malformed board's shape
|
||||||
|
|
||||||
|
/// The malformed board's lanes and cards — small, because nothing about this variant is about
|
||||||
|
/// size. Two lanes so the tree has a shape at all, and two cards in the first so the broken one
|
||||||
|
/// has an intact sibling the loader walked past on its way to it.
|
||||||
|
static let malformedLaneTitles = ["Intact", "Also intact"]
|
||||||
|
|
||||||
|
static let malformedCardTitles: [[String]] = [
|
||||||
|
["A good card", "The malformed card"],
|
||||||
|
["Another good card"],
|
||||||
|
]
|
||||||
|
|
||||||
|
/// Which card gets the raw overwrite, named by `(lane, card)` index.
|
||||||
|
static let malformedCardIndex = (lane: 0, card: 1)
|
||||||
|
|
||||||
|
/// A string appearing only in the malformed file, so a test can *find* that file on disk without
|
||||||
|
/// knowing the UUID the Writer minted for it — and then assert its bytes are exactly what were
|
||||||
|
/// written, which is the "nothing was silently repaired" half of fail-fast.
|
||||||
|
static let malformationMarker = "lanework-ui-test-malformed-fixture"
|
||||||
|
|
||||||
|
/// The bytes written over the doomed card's `index.md`.
|
||||||
|
///
|
||||||
|
/// **The frontmatter opens a flow sequence and never closes it**, so the YAML between the
|
||||||
|
/// delimiters cannot parse — `BoardLoader` rejects the whole board with `.unparseableYAML` naming
|
||||||
|
/// this file, which is the failure the fail-fast pass is about. Everything else in the file is
|
||||||
|
/// well-formed on purpose: the delimiters are there, the body is ordinary text, and the two
|
||||||
|
/// required fields are present in spirit. The one thing wrong with it is the one thing under
|
||||||
|
/// test.
|
||||||
|
static let malformedIndexText = """
|
||||||
|
---
|
||||||
|
schema: 1
|
||||||
|
title: The malformed card
|
||||||
|
order: [1024
|
||||||
|
---
|
||||||
|
|
||||||
|
\(malformationMarker): this card's frontmatter opens a flow sequence and never closes it.
|
||||||
|
The loader must reject the whole board rather than repair, skip, or rewrite anything — and
|
||||||
|
these bytes must still be here, unchanged, after the app has refused to open the board.
|
||||||
|
"""
|
||||||
|
|
||||||
// MARK: - Materialization
|
// MARK: - Materialization
|
||||||
|
|
||||||
/// Builds the fixture board and answers its URL — every write through `BoardWriter`, in the order
|
/// Builds the fixture board `variant` asks for and answers its URL — every write through
|
||||||
/// a user would have produced them.
|
/// `BoardWriter`, in the order a user would have produced them, with the malformed variant's one
|
||||||
|
/// raw overwrite as the documented exception (see this type's note).
|
||||||
///
|
///
|
||||||
/// Called from `RestoreBootstrapView` rather than from `KanbanApp.init()`: it is filesystem work,
|
/// Called from `RestoreBootstrapView` rather than from `KanbanApp.init()`: it is filesystem work,
|
||||||
/// and the launch path already has a place for filesystem work that has to happen before the
|
/// and the launch path already has a place for filesystem work that has to happen before the
|
||||||
/// first real window (that view's whole reason for existing). A throw surfaces as a launch
|
/// first real window (that view's whole reason for existing). A throw surfaces as a launch
|
||||||
/// failure on welcome — the same treatment a board that fails to restore gets — so a broken
|
/// failure on welcome — the same treatment a board that fails to restore gets — so a broken
|
||||||
/// fixture is visible rather than a suite that quietly audits an empty screen.
|
/// fixture is visible rather than a suite that quietly audits an empty screen.
|
||||||
static func materializeFixtureBoard() throws -> URL {
|
///
|
||||||
let root = fixtureBoardURL
|
/// **The malformed variant does not throw here.** Building it succeeds; *loading* it is what
|
||||||
|
/// fails, one layer up, through the ordinary board-window path — which is the whole point, since
|
||||||
|
/// a fixture that failed to build would surface a message about the fixture rather than the
|
||||||
|
/// loader's own sentence about the offending file.
|
||||||
|
static func materializeFixtureBoard(_ variant: FixtureVariant = .standard) throws -> URL {
|
||||||
|
switch variant {
|
||||||
|
case .standard: try materializeStandardBoard()
|
||||||
|
case .large: try materializeLargeBoard()
|
||||||
|
case .malformed: try materializeMalformedBoard()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The audit suite's board (see `boardTitle` and the constants above it).
|
||||||
|
private static func materializeStandardBoard() throws -> URL {
|
||||||
|
let root = fixtureBoardURL(for: .standard)
|
||||||
try BoardWriter.createBoard(at: root, title: boardTitle)
|
try BoardWriter.createBoard(at: root, title: boardTitle)
|
||||||
|
|
||||||
var laneURLs: [URL] = []
|
var laneURLs: [URL] = []
|
||||||
@@ -302,6 +470,66 @@ enum UITestLaunch {
|
|||||||
return root
|
return root
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The performance suite's board — `largeLaneCount` lanes of `largeCardsPerLane` cards, built
|
||||||
|
/// through the same three Writer calls the standard board uses and nothing else.
|
||||||
|
///
|
||||||
|
/// No rich body, no attachment, no trashed card: every one of those is a *feature* the audit
|
||||||
|
/// wanted a specimen of, and this board is not about features. What it is about is quantity, and
|
||||||
|
/// quantity is the only thing that differs.
|
||||||
|
private static func materializeLargeBoard() throws -> URL {
|
||||||
|
let root = fixtureBoardURL(for: .large)
|
||||||
|
try BoardWriter.createBoard(at: root, title: FixtureVariant.large.boardTitle)
|
||||||
|
|
||||||
|
for laneIndex in 0 ..< largeLaneCount {
|
||||||
|
let laneID = try BoardWriter.createLane(inBoard: root, title: largeLaneTitle(laneIndex))
|
||||||
|
let laneURL = root.appendingPathComponent(laneID.rawValue, isDirectory: true)
|
||||||
|
for cardIndex in 0 ..< largeCardsPerLane {
|
||||||
|
_ = try BoardWriter.createCard(
|
||||||
|
inLane: laneURL,
|
||||||
|
title: largeCardTitle(lane: laneIndex, card: cardIndex)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return root
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The fail-fast suite's board: an ordinary small board, with one card's `index.md` overwritten
|
||||||
|
/// by `malformedIndexText` **after** every Writer call has finished.
|
||||||
|
///
|
||||||
|
/// The order is the whole design. Building first means the board around the broken file is one
|
||||||
|
/// the app made — right frontmatter, right ranks, right stamps — so the load that follows fails
|
||||||
|
/// for exactly one reason and the loader's sentence names exactly one file. Overwriting first, or
|
||||||
|
/// hand-writing the tree, would have produced a board whose *many* problems the loader would
|
||||||
|
/// report whichever it reached first.
|
||||||
|
///
|
||||||
|
/// The write is `Data.write`, not `BoardWriter.atomicReplace`: the Writer refuses to produce a
|
||||||
|
/// file its own loader would reject, which is a guarantee worth keeping rather than a door worth
|
||||||
|
/// opening. This is the one place in the app that goes around it, and it is unreachable without
|
||||||
|
/// the launch flag.
|
||||||
|
private static func materializeMalformedBoard() throws -> URL {
|
||||||
|
let root = fixtureBoardURL(for: .malformed)
|
||||||
|
try BoardWriter.createBoard(at: root, title: FixtureVariant.malformed.boardTitle)
|
||||||
|
|
||||||
|
var cardURLs: [[URL]] = []
|
||||||
|
for (laneIndex, laneTitle) in malformedLaneTitles.enumerated() {
|
||||||
|
let laneID = try BoardWriter.createLane(inBoard: root, title: laneTitle)
|
||||||
|
let laneURL = root.appendingPathComponent(laneID.rawValue, isDirectory: true)
|
||||||
|
var lane: [URL] = []
|
||||||
|
for title in malformedCardTitles[laneIndex] {
|
||||||
|
let cardID = try BoardWriter.createCard(inLane: laneURL, title: title)
|
||||||
|
lane.append(laneURL.appendingPathComponent(cardID.rawValue, isDirectory: true))
|
||||||
|
}
|
||||||
|
cardURLs.append(lane)
|
||||||
|
}
|
||||||
|
|
||||||
|
let doomed = cardURLs[malformedCardIndex.lane][malformedCardIndex.card]
|
||||||
|
.appendingPathComponent(BoardLoader.indexFileName, isDirectory: false)
|
||||||
|
try Data(malformedIndexText.utf8).write(to: doomed, options: .atomic)
|
||||||
|
|
||||||
|
return root
|
||||||
|
}
|
||||||
|
|
||||||
/// Writes the attachment's source into the scratch root and imports it the way a Finder drop
|
/// Writes the attachment's source into the scratch root and imports it the way a Finder drop
|
||||||
/// would (`BoardWriter.importAttachments`), so the card ends up with a real `attachments/`
|
/// would (`BoardWriter.importAttachments`), so the card ends up with a real `attachments/`
|
||||||
/// folder rather than a hand-placed file the loader would have to normalize.
|
/// folder rather than a hand-placed file the loader would have to normalize.
|
||||||
|
|||||||
@@ -96,6 +96,69 @@ struct UITestLaunchFlagTests {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MARK: - The variant flags
|
||||||
|
|
||||||
|
/// The three fixture shapes and the arguments that name them (`UITestLaunch.FixtureVariant`).
|
||||||
|
///
|
||||||
|
/// The parsing is the flag's, restated one level down — exact match, double dash, no `=value` and no
|
||||||
|
/// `--flag value` pair — so the whole family has one edge rather than two, and the tie-break exists
|
||||||
|
/// so a launch naming two variants is a decided case rather than an argument-order accident.
|
||||||
|
@Suite("The UI-test fixture variants")
|
||||||
|
struct UITestFixtureVariantTests {
|
||||||
|
|
||||||
|
@Test("A bare fixture flag is the standard board", arguments: [
|
||||||
|
["--ui-test-fixture-board"],
|
||||||
|
["/path/to/Lanework", "--ui-test-fixture-board", "-NSTreatUnknownArgumentsAsOpen", "NO"],
|
||||||
|
[],
|
||||||
|
])
|
||||||
|
func standardIsTheDefault(arguments: [String]) {
|
||||||
|
#expect(UITestLaunch.variant(arguments: arguments) == .standard)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test("A variant flag names its variant", arguments: UITestLaunch.FixtureVariant.allCases)
|
||||||
|
func variantRecognized(variant: UITestLaunch.FixtureVariant) {
|
||||||
|
#expect(UITestLaunch.variant(arguments: [UITestLaunch.fixtureFlag, variant.flag]) == variant)
|
||||||
|
// The pairing every call site uses is base-flag-plus-variant, but the variant alone is
|
||||||
|
// enough to mark the launch synthetic — otherwise a bundle that forgot the base flag would
|
||||||
|
// get an ordinary launch over the developer's real boards.
|
||||||
|
#expect(UITestLaunch.isFixtureLaunch(arguments: [variant.flag]))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The same fuzzy-edge rule the base flag has, applied to the family: a near-miss is not a flag,
|
||||||
|
/// and a near-miss is therefore not a fixture launch either.
|
||||||
|
@Test("A near-miss is not a variant flag")
|
||||||
|
func variantNotMatchedLoosely() {
|
||||||
|
let large = UITestLaunch.FixtureVariant.large
|
||||||
|
#expect(UITestLaunch.variant(arguments: [UITestLaunch.fixtureFlag, "\(large.flag)r"]) == .standard)
|
||||||
|
#expect(UITestLaunch.variant(arguments: [UITestLaunch.fixtureFlag, "--ui-test-fixture-board=large"]) == .standard)
|
||||||
|
#expect(UITestLaunch.variant(arguments: [UITestLaunch.fixtureFlag, "--ui-test-fixture-variant", "large"]) == .standard)
|
||||||
|
#expect(UITestLaunch.isFixtureLaunch(arguments: ["-ui-test-fixture-large"]) == false)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Declaration order breaks a tie, whatever order the arguments arrived in.
|
||||||
|
@Test("Two variants named at once resolve in declaration order")
|
||||||
|
func variantTieBreak() {
|
||||||
|
let flags = [UITestLaunch.FixtureVariant.malformed.flag, UITestLaunch.FixtureVariant.large.flag]
|
||||||
|
#expect(UITestLaunch.variant(arguments: [UITestLaunch.fixtureFlag] + flags) == .large)
|
||||||
|
#expect(UITestLaunch.variant(arguments: [UITestLaunch.fixtureFlag] + flags.reversed()) == .large)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every variant flag is double-dashed, for the base flag's reason, and every one of them is
|
||||||
|
/// distinct from the base flag and from its siblings — the window titles below rest on the same
|
||||||
|
/// distinctness, so a duplicate would be two boards claiming one name.
|
||||||
|
@Test("The flags and titles are double-dashed and distinct")
|
||||||
|
func flagsAreWellFormed() {
|
||||||
|
let variants = UITestLaunch.FixtureVariant.allCases
|
||||||
|
#expect(variants.allSatisfy { $0.flag.hasPrefix("--") })
|
||||||
|
#expect(Set(variants.map(\.flag)).count == variants.count)
|
||||||
|
#expect(variants.allSatisfy { $0.flag != UITestLaunch.fixtureFlag })
|
||||||
|
#expect(Set(variants.map(\.boardTitle)).count == variants.count)
|
||||||
|
// The standard variant's title is the one the audit suite has always waited on.
|
||||||
|
#expect(UITestLaunch.FixtureVariant.standard.boardTitle == UITestLaunch.boardTitle)
|
||||||
|
#expect(UITestLaunch.fixtureBoardURL == UITestLaunch.fixtureBoardURL(for: .standard))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// MARK: - The fixture board
|
// MARK: - The fixture board
|
||||||
|
|
||||||
@Suite("The audit fixture board")
|
@Suite("The audit fixture board")
|
||||||
@@ -147,12 +210,17 @@ struct UITestFixtureBoardTests {
|
|||||||
/// Everything the fixture launch writes stays inside the app's own container — the sandbox
|
/// Everything the fixture launch writes stays inside the app's own container — the sandbox
|
||||||
/// constraint that decided the whole design (a path handed over on the command line would not be
|
/// constraint that decided the whole design (a path handed over on the command line would not be
|
||||||
/// readable), stated as a test so a future "just use `/tmp`" cannot land quietly.
|
/// readable), stated as a test so a future "just use `/tmp`" cannot land quietly.
|
||||||
|
///
|
||||||
|
/// Every variant's board, not just the audit's: they share one scratch root by construction, and
|
||||||
|
/// this is the assertion that keeps a future variant from inventing a second home.
|
||||||
@Test("Everything it writes is inside the app container")
|
@Test("Everything it writes is inside the app container")
|
||||||
func scratchIsContained() {
|
func scratchIsContained() {
|
||||||
let container = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true).standardizedFileURL.path
|
let container = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true).standardizedFileURL.path
|
||||||
#expect(UITestLaunch.scratchRoot.standardizedFileURL.path.hasPrefix(container))
|
#expect(UITestLaunch.scratchRoot.standardizedFileURL.path.hasPrefix(container))
|
||||||
#expect(UITestLaunch.registryStorageURL.standardizedFileURL.path.hasPrefix(container))
|
#expect(UITestLaunch.registryStorageURL.standardizedFileURL.path.hasPrefix(container))
|
||||||
#expect(UITestLaunch.fixtureBoardURL.standardizedFileURL.path.hasPrefix(container))
|
for variant in UITestLaunch.FixtureVariant.allCases {
|
||||||
|
#expect(UITestLaunch.fixtureBoardURL(for: variant).standardizedFileURL.path.hasPrefix(container))
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The fixture registry is **not** the real one — the clause that keeps an audit run out of the
|
/// The fixture registry is **not** the real one — the clause that keeps an audit run out of the
|
||||||
@@ -163,3 +231,158 @@ struct UITestFixtureBoardTests {
|
|||||||
#expect(UITestLaunch.registryStorageURL != BoardRegistry.defaultStorageURL)
|
#expect(UITestLaunch.registryStorageURL != BoardRegistry.defaultStorageURL)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MARK: - The large board
|
||||||
|
|
||||||
|
/// The performance suite's board (`UITestLaunch.FixtureVariant.large`).
|
||||||
|
///
|
||||||
|
/// Tested here for the audit fixture's reason turned up a notch: **the suite that consumes it cannot
|
||||||
|
/// be run in every environment**, and a large board that quietly came out small would turn a
|
||||||
|
/// performance measurement into a measurement of something else — one that *passes*, since a smaller
|
||||||
|
/// board is a faster one. So the counts are asserted, through the ordinary loader, where they can be
|
||||||
|
/// checked anywhere.
|
||||||
|
@Suite("The large fixture board")
|
||||||
|
struct UITestLargeFixtureBoardTests {
|
||||||
|
|
||||||
|
@Test("It builds at the stated size and loads through the ordinary loader")
|
||||||
|
func largeBoardLoads() throws {
|
||||||
|
UITestLaunch.prepareScratchDirectory()
|
||||||
|
defer { try? FileManager.default.removeItem(at: UITestLaunch.scratchRoot) }
|
||||||
|
|
||||||
|
let root = try UITestLaunch.materializeFixtureBoard(.large)
|
||||||
|
let result = try BoardLoader.load(boardRoot: root)
|
||||||
|
let model = result.model
|
||||||
|
|
||||||
|
// The window title the performance suite waits on.
|
||||||
|
#expect(model.title.value == UITestLaunch.FixtureVariant.large.boardTitle)
|
||||||
|
#expect(root == UITestLaunch.fixtureBoardURL(for: .large))
|
||||||
|
|
||||||
|
// The size the budgets in `EndToEndVerification.md` are budgets *for*. A board that came out
|
||||||
|
// a different size makes every one of them a number about a different board.
|
||||||
|
#expect(model.lanes.count == UITestLaunch.largeLaneCount)
|
||||||
|
#expect(model.lanes.allSatisfy { $0.cards.count == UITestLaunch.largeCardsPerLane })
|
||||||
|
#expect(model.lanes.map(\.cards.count).reduce(0, +) == UITestLaunch.largeLaneCount * UITestLaunch.largeCardsPerLane)
|
||||||
|
|
||||||
|
// Lanes in board order, exactly as named — the same claim the audit fixture makes, and for
|
||||||
|
// the same reason: a shuffled board would make a lane-addressed assertion meaningless.
|
||||||
|
#expect(model.lanes.map(\.title.value) == (0 ..< UITestLaunch.largeLaneCount).map(UITestLaunch.largeLaneTitle))
|
||||||
|
|
||||||
|
// Cards in card order within each lane, and every title distinct across the whole board —
|
||||||
|
// which is what lets a UI test name one card and mean one card.
|
||||||
|
for (laneIndex, lane) in model.lanes.enumerated() {
|
||||||
|
let expected = (0 ..< UITestLaunch.largeCardsPerLane).map {
|
||||||
|
UITestLaunch.largeCardTitle(lane: laneIndex, card: $0)
|
||||||
|
}
|
||||||
|
#expect(lane.cards.map(\.title.value) == expected)
|
||||||
|
}
|
||||||
|
let titles = model.lanes.flatMap { $0.cards.compactMap(\.title.value) }
|
||||||
|
#expect(Set(titles).count == titles.count)
|
||||||
|
|
||||||
|
// Four title lengths, cycled — the masonry has different card heights to balance rather than
|
||||||
|
// a perfect grid, which is the one thing about this board that is not simply "a lot of it".
|
||||||
|
let firstLane = try #require(model.lanes.first)
|
||||||
|
#expect(Set(firstLane.cards.compactMap(\.title.value).prefix(4).map(\.count)).count == 4)
|
||||||
|
|
||||||
|
// Nothing tolerated-but-notable: this board is built through the Writer alone, so a warning
|
||||||
|
// here would mean the *builder* left a stray behind.
|
||||||
|
#expect(result.warnings.isEmpty)
|
||||||
|
#expect(model.trash.isEmpty)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - The malformed board
|
||||||
|
|
||||||
|
/// The fail-fast suite's board (`UITestLaunch.FixtureVariant.malformed`) — and the two claims the UI
|
||||||
|
/// suite makes about it, pinned where they can be checked without a display.
|
||||||
|
///
|
||||||
|
/// 01-storage-format.md § Malformed input is the rule under test: a structurally broken `index.md`
|
||||||
|
/// rejects **the whole load**, loudly, naming the file — and the app never rewrites what it could not
|
||||||
|
/// read (the Repair precedent, which `BoardLoader`'s own note states as "a load is a pure function of
|
||||||
|
/// the tree and writes nothing, ever").
|
||||||
|
@Suite("The malformed fixture board")
|
||||||
|
struct UITestMalformedFixtureBoardTests {
|
||||||
|
|
||||||
|
@Test("It builds, and then fails to load — loudly, naming the offending file")
|
||||||
|
func malformedBoardFailsFast() throws {
|
||||||
|
UITestLaunch.prepareScratchDirectory()
|
||||||
|
defer { try? FileManager.default.removeItem(at: UITestLaunch.scratchRoot) }
|
||||||
|
|
||||||
|
// Building succeeds. That is the point: the failure under test is the *loader's*, so a
|
||||||
|
// fixture that threw on the way in would surface a different sentence entirely.
|
||||||
|
let root = try UITestLaunch.materializeFixtureBoard(.malformed)
|
||||||
|
#expect(root == UITestLaunch.fixtureBoardURL(for: .malformed))
|
||||||
|
|
||||||
|
do {
|
||||||
|
_ = try BoardLoader.load(boardRoot: root)
|
||||||
|
Issue.record("the malformed board loaded — the fail-fast pass would audit a board that opens")
|
||||||
|
} catch let error as BoardLoadError {
|
||||||
|
// The path is board-relative and names the *file*, which is what the welcome row's
|
||||||
|
// failure caption carries and what the UI test asserts against.
|
||||||
|
#expect(error.path.hasSuffix("/\(BoardLoader.indexFileName)"))
|
||||||
|
#expect(error.path.split(separator: "/").count == 3, "the offending path names <lane>/<card>/index.md")
|
||||||
|
|
||||||
|
// The reason is the one the bytes were written to produce — unparseable YAML, not a
|
||||||
|
// missing field. A future edit to `malformedIndexText` that accidentally produced a
|
||||||
|
// *valid* file with a missing key would still fail the load, and this line is what
|
||||||
|
// would notice.
|
||||||
|
if case .unparseableYAML = error.reason {} else {
|
||||||
|
Issue.record("expected unparseable YAML, got \(error.reason)")
|
||||||
|
}
|
||||||
|
|
||||||
|
// The whole sentence, which is what actually reaches the user: file first, then why.
|
||||||
|
#expect(error.description.contains(BoardLoader.indexFileName))
|
||||||
|
#expect(error.description.lowercased().contains("yaml"))
|
||||||
|
} catch {
|
||||||
|
Issue.record("expected a BoardLoadError, got \(error)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// **Nothing is silently repaired.** The refused load leaves the malformed bytes exactly as they
|
||||||
|
/// were written — no rewrite, no relocation into `.trash/`, no skip-and-continue — and the intact
|
||||||
|
/// siblings are untouched too.
|
||||||
|
///
|
||||||
|
/// This is the claim the UI suite can only make opportunistically (it can read the app's
|
||||||
|
/// container when the runner can reach it), so it is made unconditionally here.
|
||||||
|
@Test("A refused load repairs nothing")
|
||||||
|
func malformedBoardIsNotRepaired() throws {
|
||||||
|
UITestLaunch.prepareScratchDirectory()
|
||||||
|
defer { try? FileManager.default.removeItem(at: UITestLaunch.scratchRoot) }
|
||||||
|
|
||||||
|
let root = try UITestLaunch.materializeFixtureBoard(.malformed)
|
||||||
|
let before = try Self.tree(under: root)
|
||||||
|
|
||||||
|
// Twice, because a repair that only ran on the second attempt would be the worst kind.
|
||||||
|
for _ in 0 ..< 2 {
|
||||||
|
do {
|
||||||
|
_ = try BoardLoader.load(boardRoot: root)
|
||||||
|
Issue.record("the malformed board loaded")
|
||||||
|
} catch let error as BoardLoadError {
|
||||||
|
#expect(error.path.hasSuffix(BoardLoader.indexFileName))
|
||||||
|
} catch {
|
||||||
|
Issue.record("expected a BoardLoadError, got \(error)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#expect(try Self.tree(under: root) == before)
|
||||||
|
|
||||||
|
// And the bytes themselves are the ones the fixture wrote, marker included — the string a UI
|
||||||
|
// test searches the container for when it can reach it.
|
||||||
|
let malformed = try #require(before.first { $0.value.contains(UITestLaunch.malformationMarker) })
|
||||||
|
#expect(malformed.value == UITestLaunch.malformedIndexText)
|
||||||
|
#expect(malformed.key.hasSuffix("/\(BoardLoader.indexFileName)"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every `index.md` beneath `root`, keyed by its board-relative path, read as raw text. Hidden
|
||||||
|
/// entries included, so a relocation into `.trash/` would show up as a new key rather than as a
|
||||||
|
/// silence.
|
||||||
|
private static func tree(under root: URL) throws -> [String: String] {
|
||||||
|
let manager = FileManager.default
|
||||||
|
guard let walker = manager.enumerator(atPath: root.path) else { return [:] }
|
||||||
|
var files: [String: String] = [:]
|
||||||
|
for case let relative as String in walker where relative.hasSuffix(BoardLoader.indexFileName) {
|
||||||
|
let data = try Data(contentsOf: root.appendingPathComponent(relative))
|
||||||
|
files[relative] = String(decoding: data, as: UTF8.self)
|
||||||
|
}
|
||||||
|
return files
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -35,7 +35,9 @@ import XCTest
|
|||||||
/// Every test launches with `UITestLaunch.fixtureFlag`, which makes the app build a known board
|
/// Every test launches with `UITestLaunch.fixtureFlag`, which makes the app build a known board
|
||||||
/// inside its own container and open it (see `UITestLaunch` for why the board cannot simply be
|
/// inside its own container and open it (see `UITestLaunch` for why the board cannot simply be
|
||||||
/// handed over on the command line — the sandbox). The board is three lanes, six cards, one card
|
/// handed over on the command line — the sandbox). The board is three lanes, six cards, one card
|
||||||
/// with a rich Markdown body and an attachment, and one card already in `.trash/`.
|
/// with a rich Markdown body and an attachment, and one card already in `.trash/`. That is the
|
||||||
|
/// `standard` fixture variant; the launch helpers and the other two shapes live in
|
||||||
|
/// `UITestSupport.swift`.
|
||||||
///
|
///
|
||||||
/// ### Running these
|
/// ### Running these
|
||||||
///
|
///
|
||||||
@@ -165,100 +167,3 @@ final class AccessibilityAuditTests: XCTestCase {
|
|||||||
try app.performAccessibilityAudit()
|
try app.performAccessibilityAudit()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// MARK: - Driving the app
|
|
||||||
|
|
||||||
extension XCUIApplication {
|
|
||||||
|
|
||||||
/// One timeout for the whole suite. Generous, because a cold launch here also builds a board on
|
|
||||||
/// disk, and because a UI test that is slow is not a UI test that is wrong.
|
|
||||||
static let uiTimeout: TimeInterval = 30
|
|
||||||
|
|
||||||
/// The first element anywhere in the app carrying `label`, whatever type it turned out to be.
|
|
||||||
///
|
|
||||||
/// SwiftUI decides for itself which `XCUIElement.ElementType` a flattened accessibility element
|
|
||||||
/// lands as — a card face and a lane container are both "one element" by design and neither is
|
|
||||||
/// promised to be a `staticText` — so the waits in this file match on the label, which *is*
|
|
||||||
/// specified (`AccessibilityPhrases`), rather than on a type that is not.
|
|
||||||
@MainActor
|
|
||||||
func element(labeled label: String) -> XCUIElement {
|
|
||||||
descendants(matching: .any)
|
|
||||||
.matching(NSPredicate(format: "label == %@", label))
|
|
||||||
.firstMatch
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Launches the app on the audit fixture board and waits for its window.
|
|
||||||
///
|
|
||||||
/// The flag's spelling is `UITestLaunch.fixtureFlag` in the app target; it is written out here
|
|
||||||
/// because a UI test bundle does not link the app it drives — the two ends of a launch argument
|
|
||||||
/// are always a pair of literals, and this is the one place this end of the pair is spelled.
|
|
||||||
@MainActor
|
|
||||||
static func launchedWithFixtureBoard() -> XCUIApplication {
|
|
||||||
let app = XCUIApplication()
|
|
||||||
app.launchArguments += ["--ui-test-fixture-board"]
|
|
||||||
app.launch()
|
|
||||||
XCTAssertTrue(
|
|
||||||
app.wait(for: .runningForeground, timeout: uiTimeout),
|
|
||||||
"the app did not reach the foreground"
|
|
||||||
)
|
|
||||||
// `UITestLaunch.boardTitle`, which is also the window title (`BoardWindowHost.windowTitle`).
|
|
||||||
// Waiting on *this* window rather than on `windows.firstMatch` is what makes a failed fixture
|
|
||||||
// build a failure here instead of an audit that quietly passed over the welcome window.
|
|
||||||
XCTAssertTrue(
|
|
||||||
app.windows["Audit Board"].waitForExistence(timeout: uiTimeout),
|
|
||||||
"the fixture board window did not open — check the app's launch log for a fixture build failure"
|
|
||||||
)
|
|
||||||
return app
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Clicks a menu-bar row by its title.
|
|
||||||
///
|
|
||||||
/// **By title, because the titles are API** (04-interactions.md ▸ Configurable bindings: menu
|
|
||||||
/// items are remapped by title through `NSUserKeyEquivalents`, so the strings in
|
|
||||||
/// `KanbanApp.menuCommands` are already a published contract). Driving the menus rather than
|
|
||||||
/// typing the chords also means these tests exercise the same path a keyboard user takes, and a
|
|
||||||
/// row that silently disabled itself fails here as an un-hittable element rather than as a
|
|
||||||
/// keystroke that went nowhere.
|
|
||||||
@MainActor
|
|
||||||
func clickMenuItem(_ title: String, in menu: String) {
|
|
||||||
let bar = menuBars.firstMatch
|
|
||||||
let menuBarItem = bar.menuBarItems[menu]
|
|
||||||
XCTAssertTrue(
|
|
||||||
menuBarItem.waitForExistence(timeout: Self.uiTimeout),
|
|
||||||
"the \(menu) menu is missing from the menu bar"
|
|
||||||
)
|
|
||||||
menuBarItem.click()
|
|
||||||
|
|
||||||
let item = bar.menuItems[title]
|
|
||||||
XCTAssertTrue(
|
|
||||||
item.waitForExistence(timeout: Self.uiTimeout),
|
|
||||||
"\(menu) ▸ \(title) is missing"
|
|
||||||
)
|
|
||||||
XCTAssertTrue(item.isEnabled, "\(menu) ▸ \(title) is disabled")
|
|
||||||
item.click()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Selects the fixture's rich card and opens its window.
|
|
||||||
///
|
|
||||||
/// **The selection is seeded with one arrow press**, which is the app's own documented rule:
|
|
||||||
/// "an empty selection seeds at the first lane's first card" (04-interactions.md ▸ Grammar,
|
|
||||||
/// `BoardView.seed`). From there the walk to the rich card is ordinary arrow navigation — the
|
|
||||||
/// same keys a user without a pointer would press — so this helper needs to know no identifiers
|
|
||||||
/// and no geometry, only the fixture's shape.
|
|
||||||
///
|
|
||||||
/// The rich card is the sole card of lane 2 ("Doing"), so: one press to seed into lane 1's first
|
|
||||||
/// card, then one `→` to step into lane 2. `→` steps to the nearest card in that direction, and
|
|
||||||
/// with one card in the lane there is only one it can be.
|
|
||||||
@MainActor
|
|
||||||
func openRichCardWindow() throws {
|
|
||||||
typeKey(.downArrow, modifierFlags: [])
|
|
||||||
typeKey(.rightArrow, modifierFlags: [])
|
|
||||||
clickMenuItem("Open Card", in: "Board")
|
|
||||||
// `UITestLaunch.cardTitles[1][0]`, which is the card window's title
|
|
||||||
// (`CardWindowHost.windowTitle`).
|
|
||||||
XCTAssertTrue(
|
|
||||||
windows["Write the smoke script"].waitForExistence(timeout: Self.uiTimeout),
|
|
||||||
"the card window did not open"
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -4,6 +4,8 @@ The whole of DESIGN/10-accessibility.md § Verification, in one place: the autom
|
|||||||
|
|
||||||
It lives beside `AccessibilityAuditTests.swift` because the design says it should — "a manual VoiceOver smoke script lives with the test plan" — and because the automated and manual halves are one pass, not two.
|
It lives beside `AccessibilityAuditTests.swift` because the design says it should — "a manual VoiceOver smoke script lives with the test plan" — and because the automated and manual halves are one pass, not two.
|
||||||
|
|
||||||
|
This is the **accessibility** gate. The golden-path pass — the flows a user performs, the fail-fast launch, and the performance budgets — is `EndToEndVerification.md`, beside it, sharing this document's prerequisites and the same fixture mechanism.
|
||||||
|
|
||||||
## Contents
|
## Contents
|
||||||
|
|
||||||
1. [Before you start](#before-you-start)
|
1. [Before you start](#before-you-start)
|
||||||
@@ -49,7 +51,7 @@ The eight surfaces, and how each test gets there:
|
|||||||
| `testTemplateChooser` | Template chooser | File ▸ New Board… |
|
| `testTemplateChooser` | Template chooser | File ▸ New Board… |
|
||||||
| `testBoardInfoPopover` | Board popover | File ▸ Board Info |
|
| `testBoardInfoPopover` | Board popover | File ▸ Board Info |
|
||||||
|
|
||||||
Every test launches the app with `--ui-test-fixture-board`, which makes the app build a known board inside its own container and open it — three lanes ("To Do", "Doing", "Done"), six cards, one card with a rich Markdown body and an attachment, one card already in the trash. The board and the registry both live in a scratch directory that is wiped on every launch, so an audit run never touches your real boards or your recents list. See `Kanban/App/UITestLaunch.swift` for why the board cannot simply be handed to the app on the command line (the sandbox).
|
Every test launches the app with `--ui-test-fixture-board`, which makes the app build a known board inside its own container and open it — three lanes ("To Do", "Doing", "Done"), six cards, one card with a rich Markdown body and an attachment, one card already in the trash. That is the `standard` fixture variant; the bare flag means it, and the other two shapes (`large`, `malformed`) belong to the end-to-end pass. The board and the registry both live in a scratch directory that is wiped on every launch, so an audit run never touches your real boards or your recents list. See `Kanban/App/UITestLaunch.swift` for why the board cannot simply be handed to the app on the command line (the sandbox).
|
||||||
|
|
||||||
If a test fails, read the issue's `compactDescription` and fix the app. Adding a waiver is a design change and needs an entry on the Redesign board first.
|
If a test fails, read the issue's `compactDescription` and fix the app. Adding a waiver is a design change and needs an entry on the Redesign board first.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,312 @@
|
|||||||
|
import XCTest
|
||||||
|
|
||||||
|
/// **The golden paths, driven through the shipping app** — create, rename, move, cut and paste,
|
||||||
|
/// delete, restore, empty the trash, undo and redo.
|
||||||
|
///
|
||||||
|
/// ### What this suite is for, and what it is not
|
||||||
|
///
|
||||||
|
/// Every rule these flows exercise is already pinned somewhere in `KanbanTests`: the write paths by
|
||||||
|
/// `*WriteTests`, the grammar by `SelectionGrammarTests` and `KeyboardGrammarTests`, the trash by
|
||||||
|
/// `TrashModelTests` and `TrashWriteTests`, the stack by `UndoWriteTests`. Not one assertion below is
|
||||||
|
/// a fact those suites cannot state faster and more precisely.
|
||||||
|
///
|
||||||
|
/// What they cannot state is that the pieces are *wired together* — that File ▸ New Card reaches
|
||||||
|
/// `beginNewCard`, that the editor it opens is focused, that Return commits it, that the write lands,
|
||||||
|
/// that the watcher rounds it back, and that the lane the user is looking at now says "4 cards". This
|
||||||
|
/// suite is that sentence, once per flow, and nothing else. When one of these fails and the unit
|
||||||
|
/// suites stay green, the defect is in the wiring — a menu item scoped to the wrong focus value, a
|
||||||
|
/// command that lost its shortcut, a reload that never arrived.
|
||||||
|
///
|
||||||
|
/// ### How a flow asserts
|
||||||
|
///
|
||||||
|
/// **On lane labels**, almost always (`XCUIApplication.awaitLane`). A lane's accessibility label is
|
||||||
|
/// "⟨title⟩, lane, N cards" and the count is the *rendered* one, so one label assertion covers the
|
||||||
|
/// visible badge, the spoken count and the layout at once — 10-accessibility.md ▸ Trash lane makes
|
||||||
|
/// them one number, and this is where that pays. A card's own element carries only its title, which
|
||||||
|
/// does not change when the card changes lanes, so asserting on the card would assert nothing about
|
||||||
|
/// where it went.
|
||||||
|
///
|
||||||
|
/// The fixture is the `standard` board: three lanes — **To Do** (3 cards), **Doing** (1), **Done**
|
||||||
|
/// (1) — plus one card already in `.trash/`. Every flow below starts from exactly that and states
|
||||||
|
/// its own arithmetic against it.
|
||||||
|
///
|
||||||
|
/// ### Running these
|
||||||
|
///
|
||||||
|
/// They drive the real app through the real menu bar, so the machine running them must have granted
|
||||||
|
/// the test runner Accessibility control (System Settings ▸ Privacy & Security ▸ Accessibility) and
|
||||||
|
/// must not be locked or headless — the whole of `KanbanUITests/EndToEndVerification.md`'s
|
||||||
|
/// prerequisites section, which is where the run command lives too.
|
||||||
|
///
|
||||||
|
/// ### Reading a failure
|
||||||
|
///
|
||||||
|
/// A failure on a `clickMenuItem` is a *validation* failure: the row was missing or disabled, which
|
||||||
|
/// means a focused value did not reach it. A failure on an `awaitLane` is the flow itself: the
|
||||||
|
/// command ran and the board did not end up where the design says it should. A failure on a
|
||||||
|
/// `waitForExistence` for a card or an editor is navigation, in this file.
|
||||||
|
final class EndToEndFlowTests: XCTestCase {
|
||||||
|
|
||||||
|
override func setUp() {
|
||||||
|
super.setUp()
|
||||||
|
// A flow is a sequence: once a step has failed, every later assertion is about a board in a
|
||||||
|
// state nobody wrote down.
|
||||||
|
continueAfterFailure = false
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - Creating
|
||||||
|
|
||||||
|
/// **File ▸ New Card (⌘N)** — the placeholder opens focused in the target lane, Return commits,
|
||||||
|
/// and the card is on the board (04-interactions.md ▸ Grammar: "Return commits and re-selects the
|
||||||
|
/// lane").
|
||||||
|
///
|
||||||
|
/// The target is made explicit by selecting a card in "To Do" first. ⌘N with nothing selected
|
||||||
|
/// would also land there (`NewCardTarget.resolve` falls back to the first lane), but a flow that
|
||||||
|
/// relies on a fallback is a flow that would pass for the wrong reason.
|
||||||
|
@MainActor
|
||||||
|
func testCreateCard() throws {
|
||||||
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
||||||
|
app.selectCard(StandardBoard.firstLaneCards[0])
|
||||||
|
|
||||||
|
app.clickMenuItem("New Card", in: "File")
|
||||||
|
app.typeIntoInlineEditor("A card the suite made", prompt: Phrase.cardTitlePrompt)
|
||||||
|
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.element(labeled: "A card the suite made").waitForExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"the new card never appeared on the board"
|
||||||
|
)
|
||||||
|
app.awaitLane(StandardBoard.toDo, cards: 4, "the new card should have landed in To Do")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// **File ▸ New Lane (⇧⌘N)** — a lane with no title and no editor (`BoardStore.createLane`), so
|
||||||
|
/// it arrives reading the untitled placeholder with an empty count.
|
||||||
|
///
|
||||||
|
/// The count is the assertion that matters: a new lane that arrived holding cards would mean the
|
||||||
|
/// create had landed somewhere it should not have.
|
||||||
|
@MainActor
|
||||||
|
func testCreateLane() throws {
|
||||||
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
||||||
|
|
||||||
|
app.clickMenuItem("New Lane", in: "File")
|
||||||
|
|
||||||
|
app.awaitLane(Phrase.untitled, cards: 0, "File ▸ New Lane should add one empty, untitled lane")
|
||||||
|
// The three lanes that were already there are untouched — a create is not a re-divide of the
|
||||||
|
// board's contents.
|
||||||
|
app.awaitLane(StandardBoard.toDo, cards: 3)
|
||||||
|
app.awaitLane(StandardBoard.doing, cards: 1)
|
||||||
|
app.awaitLane(StandardBoard.done, cards: 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - Renaming
|
||||||
|
|
||||||
|
/// **Board ▸ Rename** on a card — the inline editor, seeded with the current title, committed
|
||||||
|
/// with Return (04-interactions.md ▸ Selection; the menu row exists "for completeness and
|
||||||
|
/// remapping", the pointer path being a slow double click).
|
||||||
|
///
|
||||||
|
/// Select All then type is how the seeded title is replaced: inside a focused field that chord
|
||||||
|
/// belongs to the field editor, because the board's own Select All disables while an inline
|
||||||
|
/// editor is open.
|
||||||
|
@MainActor
|
||||||
|
func testRenameCardInline() throws {
|
||||||
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
||||||
|
let original = StandardBoard.firstLaneCards[3]
|
||||||
|
let renamed = "Ship the audit, renamed"
|
||||||
|
|
||||||
|
app.selectCard(original)
|
||||||
|
app.clickMenuItem("Rename", in: "Board")
|
||||||
|
app.typeIntoInlineEditor(renamed, prompt: Phrase.cardTitlePrompt, replacingExisting: true)
|
||||||
|
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.element(labeled: renamed).waitForExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"the renamed card never appeared"
|
||||||
|
)
|
||||||
|
// A rename is not a move: the lane still holds the same three cards.
|
||||||
|
app.awaitLane(StandardBoard.toDo, cards: 3)
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - Moving
|
||||||
|
|
||||||
|
/// **A pointer drag across lanes** (04-interactions.md ▸ Drag and drop) — the card leaves "To Do"
|
||||||
|
/// and lands in "Doing", which the two lanes' counts say and nothing else has to.
|
||||||
|
///
|
||||||
|
/// This is the suite's one gesture that goes through a real AppKit dragging session rather than
|
||||||
|
/// through a menu, and it is correspondingly the most environment-sensitive
|
||||||
|
/// (`XCUIApplication.dragCard` carries the details). It earns its place anyway: the drag is the
|
||||||
|
/// board's headline interaction, its reflow is the largest piece of geometry in the app, and no
|
||||||
|
/// unit test can press a mouse button.
|
||||||
|
@MainActor
|
||||||
|
func testDragCardBetweenLanes() throws {
|
||||||
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
||||||
|
|
||||||
|
app.dragCard(StandardBoard.firstLaneCards[0], onto: StandardBoard.richCard)
|
||||||
|
|
||||||
|
app.awaitLane(StandardBoard.doing, cards: 2, "the dragged card should have landed in Doing")
|
||||||
|
app.awaitLane(StandardBoard.toDo, cards: 2, "the dragged card should have left To Do")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// **Cut and paste across lanes (⌘X / ⌘V)** — the keyboard's own move (04-interactions.md
|
||||||
|
/// ▸ Clipboard), driven through the Edit menu because the board answers `cut:`/`paste:` as a
|
||||||
|
/// responder and the menu is where that wiring is visible.
|
||||||
|
///
|
||||||
|
/// The paste lands the card **after** the anchor card, which is why the destination is selected
|
||||||
|
/// by clicking a card in it rather than by clicking the lane.
|
||||||
|
@MainActor
|
||||||
|
func testCutAndPasteCardAcrossLanes() throws {
|
||||||
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
||||||
|
|
||||||
|
app.selectCard(StandardBoard.firstLaneCards[0])
|
||||||
|
app.clickMenuItem("Cut", in: "Edit")
|
||||||
|
app.selectCard(StandardBoard.richCard)
|
||||||
|
app.clickMenuItem("Paste", in: "Edit")
|
||||||
|
|
||||||
|
app.awaitLane(StandardBoard.doing, cards: 2, "the cut card should have landed in Doing")
|
||||||
|
app.awaitLane(StandardBoard.toDo, cards: 2, "the cut card should have left To Do")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// **⌘Z and ⇧⌘Z over a move** (13-native-undo.md ▸ Rules) — the chords rather than the menu rows,
|
||||||
|
/// because those rows are titled dynamically ("Undo Move Card") by `NSUndoManager` and a test
|
||||||
|
/// that spelled the title would be asserting the platform's composition rather than the app's
|
||||||
|
/// stack.
|
||||||
|
///
|
||||||
|
/// The move is the cut/paste above, so what undo has to put back is a folder that physically
|
||||||
|
/// changed lanes — the interesting case, and the one the native provider registers an inverse
|
||||||
|
/// for at the Writer boundary.
|
||||||
|
@MainActor
|
||||||
|
func testUndoAndRedoOfAMove() throws {
|
||||||
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
||||||
|
|
||||||
|
app.selectCard(StandardBoard.firstLaneCards[0])
|
||||||
|
app.clickMenuItem("Cut", in: "Edit")
|
||||||
|
app.selectCard(StandardBoard.richCard)
|
||||||
|
app.clickMenuItem("Paste", in: "Edit")
|
||||||
|
app.awaitLane(StandardBoard.doing, cards: 2)
|
||||||
|
|
||||||
|
app.typeKey("z", modifierFlags: .command)
|
||||||
|
app.awaitLane(StandardBoard.toDo, cards: 3, "⌘Z should have returned the card to To Do")
|
||||||
|
app.awaitLane(StandardBoard.doing, cards: 1)
|
||||||
|
|
||||||
|
app.typeKey("z", modifierFlags: [.shift, .command])
|
||||||
|
app.awaitLane(StandardBoard.doing, cards: 2, "⇧⌘Z should have moved the card back into Doing")
|
||||||
|
app.awaitLane(StandardBoard.toDo, cards: 2)
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - The trash
|
||||||
|
|
||||||
|
/// **Delete, then restore** — the whole of the current trash grammar's board side
|
||||||
|
/// (03-board-ui.md § Trash, resettled 2026-07-28 for the materialized trash).
|
||||||
|
///
|
||||||
|
/// Four claims in one flow, because they are one story:
|
||||||
|
///
|
||||||
|
/// 1. **File ▸ Delete on a board card does not confirm.** The card moves into `<root>/.trash/`
|
||||||
|
/// and that is recoverable, so there is no alert — this flow would hang on one if there were.
|
||||||
|
/// 2. **View ▸ Show Trash** puts the column on the board as the last container, holding what was
|
||||||
|
/// already in it plus what just arrived.
|
||||||
|
/// 3. **Restore is an ordinary move out**: ⌘X in the trash, ⌘V into a lane. There is no Put Back
|
||||||
|
/// and there is nothing else to press.
|
||||||
|
/// 4. The counts on both sides move, both times.
|
||||||
|
@MainActor
|
||||||
|
func testDeleteToTrashAndRestore() throws {
|
||||||
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
||||||
|
let doomed = StandardBoard.firstLaneCards[0]
|
||||||
|
|
||||||
|
app.selectCard(doomed)
|
||||||
|
app.clickMenuItem("Delete", in: "File")
|
||||||
|
app.awaitLane(StandardBoard.toDo, cards: 2, "the deleted card should have left the lane")
|
||||||
|
|
||||||
|
app.clickMenuItem("Show Trash", in: "View")
|
||||||
|
app.awaitTrash(cards: 2)
|
||||||
|
|
||||||
|
// The restore: cut in the trash, paste into a live lane.
|
||||||
|
app.selectCard(doomed)
|
||||||
|
app.clickMenuItem("Cut", in: "Edit")
|
||||||
|
app.selectCard(StandardBoard.doneCard)
|
||||||
|
app.clickMenuItem("Paste", in: "Edit")
|
||||||
|
|
||||||
|
app.awaitLane(StandardBoard.done, cards: 2, "the restored card should have landed in Done")
|
||||||
|
app.awaitTrash(cards: 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// **File ▸ Empty Trash… (⇧⌘⌫)** — "the alert always appears" (03-board-ui.md § Trash), naming
|
||||||
|
/// the true count, and confirming it purges the container.
|
||||||
|
///
|
||||||
|
/// The trash has to be *shown* for the row to be enabled at all ("hidden, it is invisible to
|
||||||
|
/// every gesture"), which is the first click here and is itself part of the claim.
|
||||||
|
@MainActor
|
||||||
|
func testEmptyTrash() throws {
|
||||||
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
||||||
|
|
||||||
|
app.clickMenuItem("Show Trash", in: "View")
|
||||||
|
app.awaitTrash(cards: 1)
|
||||||
|
|
||||||
|
app.clickMenuItem("Empty Trash…", in: "File")
|
||||||
|
|
||||||
|
// The prompt is `TrashModel.emptyTrashPrompt`: the count in the title, Delete and Cancel as
|
||||||
|
// the two answers. It is asserted rather than merely dismissed, because "the alert always
|
||||||
|
// appears" is the design rule under test.
|
||||||
|
//
|
||||||
|
// Queried app-wide rather than through `dialogs`/`sheets`, deliberately: which of those a
|
||||||
|
// SwiftUI `.alert` surfaces as on macOS is the framework's choice and not a contract, while
|
||||||
|
// the button titles and the sentence are `TrashConfirmations`' and `TrashModel`'s.
|
||||||
|
let confirm = app.buttons["Delete"]
|
||||||
|
XCTAssertTrue(
|
||||||
|
confirm.waitForExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"Empty Trash… did not raise its confirmation"
|
||||||
|
)
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.element(labelContaining: "Permanently delete \(Phrase.cards(1))").exists,
|
||||||
|
"the confirmation did not name the trash's count"
|
||||||
|
)
|
||||||
|
XCTAssertTrue(app.buttons["Cancel"].exists, "the confirmation had no Cancel")
|
||||||
|
|
||||||
|
confirm.click()
|
||||||
|
|
||||||
|
app.awaitTrash(cards: 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - Templates
|
||||||
|
|
||||||
|
/// **File ▸ New Board… (⌥⌘N) as far as this suite can drive it** — the chooser opens, the bundled
|
||||||
|
/// templates are listed and selectable, and Choose is live.
|
||||||
|
///
|
||||||
|
/// ### Why it stops there
|
||||||
|
///
|
||||||
|
/// Choose runs an `NSSavePanel`, and in a sandboxed app that panel is **Powerbox** — a separate,
|
||||||
|
/// system-owned process (`com.apple.appkit.xpc.openAndSavePanelService`). Driving it means
|
||||||
|
/// driving another application's UI from inside this one's test, on a surface Apple changes
|
||||||
|
/// between releases and which has no accessibility contract of its own. A test that did it would
|
||||||
|
/// fail for reasons that have nothing to do with this app, which is worse than not having the
|
||||||
|
/// test: a flaky gate teaches people to re-run rather than to read.
|
||||||
|
///
|
||||||
|
/// So the automated half ends at the panel, and **instantiating a template is a manual step** —
|
||||||
|
/// written down in `KanbanUITests/EndToEndVerification.md` ▸ Manual-only flows rather than faked
|
||||||
|
/// here. Everything behind the panel is unit-tested (`TemplateEngineTests` covers the copy, the
|
||||||
|
/// `template:` key, the collision ladder and the cancellation).
|
||||||
|
@MainActor
|
||||||
|
func testTemplateChooserUpToTheSavePanel() throws {
|
||||||
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
||||||
|
|
||||||
|
app.clickMenuItem("New Board…", in: "File")
|
||||||
|
let chooser = app.windows["New Board"]
|
||||||
|
XCTAssertTrue(
|
||||||
|
chooser.waitForExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"the template chooser did not appear"
|
||||||
|
)
|
||||||
|
|
||||||
|
// "Basic" is the bundled tier's lowest `template.order`, so it is the row the chooser opens
|
||||||
|
// on — and a tile is a button to the accessibility tree, labeled by its name
|
||||||
|
// (10-accessibility.md ▸ Template chooser).
|
||||||
|
let basic = app.element(labeled: "Basic")
|
||||||
|
XCTAssertTrue(basic.waitForExistence(timeout: XCUIApplication.uiTimeout), "the Basic template is not listed")
|
||||||
|
basic.click()
|
||||||
|
|
||||||
|
let choose = chooser.buttons["Choose"]
|
||||||
|
XCTAssertTrue(choose.exists, "the chooser has no Choose button")
|
||||||
|
XCTAssertTrue(choose.isEnabled, "Choose is disabled on a loadable template")
|
||||||
|
|
||||||
|
// Cancel rather than Choose: pressing Choose hands the flow to Powerbox (see above). The
|
||||||
|
// chooser closing is the last thing this test can honestly assert.
|
||||||
|
chooser.buttons["Cancel"].click()
|
||||||
|
XCTAssertTrue(
|
||||||
|
chooser.waitForNonExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"Cancel did not dismiss the template chooser"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,152 @@
|
|||||||
|
# End-to-end verification
|
||||||
|
|
||||||
|
The golden-path UI pass: the flows a user actually performs, driven through the shipping app. It sits beside `AccessibilityVerification.md` — that document is the accessibility gate, this one is the wiring gate — and shares its prerequisites, because both drive the real app through the real menu bar.
|
||||||
|
|
||||||
|
Three suites, one fixture mechanism, and a short list of flows that stayed manual and say so.
|
||||||
|
|
||||||
|
## Contents
|
||||||
|
|
||||||
|
1. [Before you start](#before-you-start)
|
||||||
|
2. [The fixture boards](#the-fixture-boards)
|
||||||
|
3. [Running the suites](#running-the-suites)
|
||||||
|
4. [The flow inventory](#the-flow-inventory)
|
||||||
|
5. [Manual-only flows](#manual-only-flows)
|
||||||
|
6. [The performance budgets](#the-performance-budgets)
|
||||||
|
7. [When something fails](#when-something-fails)
|
||||||
|
|
||||||
|
## Before you start
|
||||||
|
|
||||||
|
Same prerequisites as the accessibility pass, and for the same reason — these tests move a real pointer and open real menus:
|
||||||
|
|
||||||
|
- A **real, unlocked display** and a real keyboard. Nothing here survives a headless session, a locked screen, or screen sharing.
|
||||||
|
- **Accessibility automation permission** for the test runner: System Settings ▸ Privacy & Security ▸ Accessibility. Without it every test fails on its first `click()`, for a reason that has nothing to do with the app. Xcode itself usually needs to be listed; so does the terminal, if you are running `xcodebuild` from one.
|
||||||
|
- Nothing else driving the pointer while a run is in progress.
|
||||||
|
|
||||||
|
A run that is interrupted can leave a stray `Lanework` process behind — `killall Lanework` between runs.
|
||||||
|
|
||||||
|
Everything a run writes goes into the app's sandbox container (`~/Library/Containers/dev.rzen.indie.Kanban/Data/tmp/LaneworkUITestFixture/`), which is wiped at the start of every launch. **A run never touches your real boards or your recents list**: the fixture flag redirects the board registry into the same scratch directory (`Kanban/App/UITestLaunch.swift`).
|
||||||
|
|
||||||
|
## The fixture boards
|
||||||
|
|
||||||
|
The app builds its own fixture, inside its own container, through its own `BoardWriter`. It cannot be handed a board on the command line: the sandbox grants `files.user-selected.read-write` and nothing else, so a path arriving as an argument is a path the app may not read. `UITestLaunch` carries the full argument.
|
||||||
|
|
||||||
|
Three variants, selected by a pair of launch arguments — the base flag, which means "this launch is synthetic", plus the variant's own:
|
||||||
|
|
||||||
|
| Variant | Launch arguments | Board | Window title |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `standard` | `--ui-test-fixture-board --ui-test-fixture-standard` | 3 lanes (To Do 3 · Doing 1 · Done 1), one rich card with an attachment, one card in `.trash/` | `Audit Board` |
|
||||||
|
| `large` | `--ui-test-fixture-board --ui-test-fixture-large` | 8 lanes × 40 cards, titles cycling through four lengths so masonry has heights to balance | `Large Board` |
|
||||||
|
| `malformed` | `--ui-test-fixture-board --ui-test-fixture-malformed` | 2 lanes, 3 cards — one card's `index.md` overwritten with unparseable frontmatter | `Malformed Board` (never appears) |
|
||||||
|
|
||||||
|
`--ui-test-fixture-board` on its own still means `standard`, which is what the accessibility suite has always passed. The flags are exact-matched and double-dashed for the base flag's reasons: a launch switch with a fuzzy edge can be tripped by accident, and a `-key value` pair would be swallowed by `UserDefaults`' argument domain and become a preference.
|
||||||
|
|
||||||
|
Every variant is built through `BoardWriter` — `createBoard`, `createLane`, `createCard`, and for the standard board `writeBody`, `importAttachments` and `deleteCardToTrash`. The **one** exception is the malformed variant's final act, a raw `Data.write` over one card's `index.md`: no Writer call produces a file its own loader would reject, and none should.
|
||||||
|
|
||||||
|
The builders themselves are unit-tested (`KanbanTests/UITestLaunchTests.swift`): the flag and variant parsing, the large board's exact counts and ordering, and the malformed board's fail-fast plus its byte fidelity after a refused load. **Those run anywhere** — no display, no permission — and they are what stops a fixture that quietly came out wrong from turning a UI run into a pass over the wrong board.
|
||||||
|
|
||||||
|
## Running the suites
|
||||||
|
|
||||||
|
The whole UI bundle, accessibility audits included:
|
||||||
|
|
||||||
|
```
|
||||||
|
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
|
||||||
|
-destination 'platform=macOS,arch=arm64' \
|
||||||
|
-only-testing:KanbanUITests
|
||||||
|
```
|
||||||
|
|
||||||
|
The golden flows on their own — the fastest useful run, and the one to reach for after a change to a command, a focused value, or the store's write path:
|
||||||
|
|
||||||
|
```
|
||||||
|
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
|
||||||
|
-destination 'platform=macOS,arch=arm64' \
|
||||||
|
-only-testing:KanbanUITests/EndToEndFlowTests
|
||||||
|
```
|
||||||
|
|
||||||
|
Fail-fast on a malformed board:
|
||||||
|
|
||||||
|
```
|
||||||
|
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
|
||||||
|
-destination 'platform=macOS,arch=arm64' \
|
||||||
|
-only-testing:KanbanUITests/FailFastLaunchTests
|
||||||
|
```
|
||||||
|
|
||||||
|
The performance pass, which is slow — each measured iteration builds the 320-card board from scratch — and is therefore worth invoking on its own rather than leaving in a routine run:
|
||||||
|
|
||||||
|
```
|
||||||
|
xcodebuild test -project Kanban.xcodeproj -scheme Kanban \
|
||||||
|
-destination 'platform=macOS,arch=arm64' \
|
||||||
|
-only-testing:KanbanUITests/LargeBoardPerformanceTests
|
||||||
|
```
|
||||||
|
|
||||||
|
To check only that the suites still *compile* — which is all that can be done on a machine without a display or without automation permission:
|
||||||
|
|
||||||
|
```
|
||||||
|
xcodebuild build-for-testing -project Kanban.xcodeproj -scheme Kanban \
|
||||||
|
-destination 'platform=macOS,arch=arm64'
|
||||||
|
```
|
||||||
|
|
||||||
|
The UI bundle is base-only by design (`project.yml`): these tests launch the real app and drive its menu bar, and a second copy would double the slowest part of the suite to re-assert something edition-blind. The Pro app's own launch is already exercised on every `LaneworkPro` unit run, which uses it as the test host.
|
||||||
|
|
||||||
|
## The flow inventory
|
||||||
|
|
||||||
|
`EndToEndFlowTests.swift`, on the `standard` fixture. Every flow asserts on **lane labels** — a lane's accessibility label is "⟨title⟩, lane, N cards" and the count is the rendered one, so one assertion covers the visible badge, the spoken count and the layout at once.
|
||||||
|
|
||||||
|
| Test | Flow | Driven by | Asserted by |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `testCreateCard` | Create a card | File ▸ New Card, type, Return | The new card's element, and To Do at 4 cards |
|
||||||
|
| `testCreateLane` | Create a lane | File ▸ New Lane | An "Untitled, lane, 0 cards" container; the other three lanes unchanged |
|
||||||
|
| `testRenameCardInline` | Inline rename | Board ▸ Rename, ⌘A, type, Return | The new title's element; the lane count unchanged |
|
||||||
|
| `testDragCardBetweenLanes` | Drag a card across lanes | `XCUICoordinate` press + drag + hold | Both lanes' counts |
|
||||||
|
| `testCutAndPasteCardAcrossLanes` | Cut/paste across lanes | Edit ▸ Cut, select, Edit ▸ Paste | Both lanes' counts |
|
||||||
|
| `testUndoAndRedoOfAMove` | Undo and redo a move | ⌘Z / ⇧⌘Z after a cut/paste | Counts back, then forward again |
|
||||||
|
| `testDeleteToTrashAndRestore` | Delete, show trash, restore | File ▸ Delete (no confirmation), View ▸ Show Trash, Edit ▸ Cut in the trash, Edit ▸ Paste into a lane | Lane counts and the trash column's value, both directions |
|
||||||
|
| `testEmptyTrash` | Empty Trash | View ▸ Show Trash, File ▸ Empty Trash…, confirm | The alert's sentence and both buttons; the trash at 0 cards |
|
||||||
|
| `testTemplateChooserUpToTheSavePanel` | Template chooser | File ▸ New Board…, select Basic, read Choose, Cancel | The chooser window, the Basic tile, Choose enabled — **and no further** (see below) |
|
||||||
|
|
||||||
|
`FailFastLaunchTests.swift`, on the `malformed` fixture:
|
||||||
|
|
||||||
|
| Test | Claim |
|
||||||
|
| --- | --- |
|
||||||
|
| `testMalformedBoardFailsLoudlyAndOpensNoWindow` | Welcome appears; **no** board window; a welcome row carries the loader's own sentence, naming `index.md` and saying "unparseable YAML" |
|
||||||
|
| `testMalformedBoardIsNeverRepaired` | The malformed bytes are unchanged on disk (where the runner can read the container), and a relaunch is refused identically |
|
||||||
|
|
||||||
|
The trash grammar these flows encode is the current one: delete moves a card into `<root>/.trash/`; ⌘⌫ stages by place (board → trash, no confirmation; trash → permanent, with one); ⌥⌘⌫ skips the trash from anywhere, with a confirmation; restore is an ordinary move out — a drag, or ⌘X in the trash and ⌘V into a lane. **There is no Put Back.**
|
||||||
|
|
||||||
|
Two things this pass deliberately does **not** cover, because they are not base's: any git flow (commit, pull, push — pro-m1's, and base has no git operations at all; that a `.git` folder is inert in base is pinned by `KanbanTests/InertGitTests.swift`), and the accessibility audits, which are `AccessibilityVerification.md`'s.
|
||||||
|
|
||||||
|
## Manual-only flows
|
||||||
|
|
||||||
|
Three, each with a reason it is not automated and a note on where its logic *is* covered.
|
||||||
|
|
||||||
|
**Instantiating a template (File ▸ New Board… ▸ Choose).** Choose runs an `NSSavePanel`, which in a sandboxed app is Powerbox — a separate, system-owned process (`com.apple.appkit.xpc.openAndSavePanelService`). Driving it means driving another application's UI, on a surface with no accessibility contract of its own that Apple changes between releases. A test that did it would fail for reasons that are not this app's, which is worse than no test: a flaky gate teaches people to re-run rather than to read. The automated flow stops at the panel and asserts everything before it. **Check by hand, per release:** File ▸ New Board…, pick a template, Choose, accept the suggested name, and confirm the new board opens in its own window with the template's lanes and starter cards, and that its row is on welcome afterwards. Everything behind the panel — the copy, the `template:` key, the collision ladder, cancellation — is `KanbanTests/TemplateEngineTests.swift`.
|
||||||
|
|
||||||
|
**File ▸ Duplicate's save-panel fallback.** The silent Finder-style sibling is attempted first and only a *permission refusal* raises the panel, so the automated path cannot reach the panel without first arranging an unwritable parent folder — and once it does, the panel is Powerbox again. **Check by hand** when the duplicate path changes: open a board inside a folder the app has no write grant for, File ▸ Duplicate, and confirm the panel opens pre-filled with the parent folder and the "copy" name, that Cancel says nothing at all, and that choosing a location completes the duplicate.
|
||||||
|
|
||||||
|
**File ▸ Open… (⌘O).** The open panel is Powerbox for the same reason. The whole fixture mechanism exists precisely because a sandboxed app cannot be handed a path, so there is nothing here to automate. **Check by hand** by opening a board from the Finder and from ⌘O, and confirming both land on the same window when the board is already open.
|
||||||
|
|
||||||
|
## The performance budgets
|
||||||
|
|
||||||
|
`LargeBoardPerformanceTests.swift`, on the `large` fixture (8 × 40 = 320 cards).
|
||||||
|
|
||||||
|
Each test does two things, and they do different jobs. The `measure` block **records** the metric, so the test report carries a distribution and Xcode can offer a local baseline. The explicit wall-clock `XCTAssertLessThan` is the **gate**: XCTest baselines are stored per machine in the project's `xcshareddata` and do not travel, so on a fresh clone `measure` alone would record a number and pass unconditionally — a log line, not a gate.
|
||||||
|
|
||||||
|
| Budget | Value | Covers |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `launchBudget` | 30 s | Process start, materializing 328 folders through `BoardWriter`, the load, and the first frame with a lane on it |
|
||||||
|
| `interactionBudget` | 5 s | View ▸ Show Trash on a 320-card board: the re-divide, every lane's reflow, and the column arriving |
|
||||||
|
|
||||||
|
They are **regression bounds, not targets** — roughly 5× the measured cost on the development machine. What they have to catch is a tenfold change (an O(n²) reflow, a synchronous tree walk on the main thread, a per-card watcher); what they must never do is fail because a laptop was busy. The launch budget's dominant term is the fixture *build* rather than the app: writing an `index.md` per card takes about two seconds on the development machine, which `KanbanTests ▸ UITestLargeFixtureBoardTests` measures without a window. That is deliberate — the number stays honest about what a launch on this fixture actually costs — but it is why the budget is loose, and why a launch regression shows up here as a big move rather than a small one.
|
||||||
|
|
||||||
|
Tighten a budget only with a measurement in hand, and record which machine it was measured on.
|
||||||
|
|
||||||
|
## When something fails
|
||||||
|
|
||||||
|
Read *which line* failed before reading the test:
|
||||||
|
|
||||||
|
- **A `clickMenuItem` failure is a validation failure.** The row was missing or disabled, which means a focused value did not reach it — a `focusedSceneValue` that stopped being published, a command scoped to the wrong window, a predicate that grew a clause.
|
||||||
|
- **An `awaitLane` / `awaitTrash` failure is the flow itself.** The command ran and the board did not end up where the design says. This is the interesting kind.
|
||||||
|
- **A `waitForExistence` on a card or an editor is navigation**, in the test file: the fixture's shape and the test's idea of it have drifted.
|
||||||
|
- **`testDragCardBetweenLanes` failing alone is worth one re-run before it is believed.** It is the suite's only real AppKit dragging session and the only gesture whose success depends on timing thresholds rather than on state.
|
||||||
|
- **Everything failing on the first click** is automation permission, not the app.
|
||||||
|
|
||||||
|
File anything that fails on the Redesign board, with the test name it came from.
|
||||||
@@ -0,0 +1,164 @@
|
|||||||
|
import XCTest
|
||||||
|
|
||||||
|
/// **A board that will not load, from the outside** (01-storage-format.md § Malformed input;
|
||||||
|
/// 02-architecture.md § Launch and window lifecycle).
|
||||||
|
///
|
||||||
|
/// ### The claim
|
||||||
|
///
|
||||||
|
/// > A restored board that fails surfaces on welcome, row-level: its window doesn't open; welcome
|
||||||
|
/// > appears alongside whatever did restore, the failed board's recents row carrying fail-fast's
|
||||||
|
/// > specifics (load error) … never a silent drop.
|
||||||
|
///
|
||||||
|
/// The `malformed` fixture is a well-formed board with exactly one unparseable card `index.md`
|
||||||
|
/// (`UITestLaunch.malformedIndexText` — a frontmatter flow sequence that is never closed). Building
|
||||||
|
/// it succeeds; loading it must not, and *how* it fails is the whole of this file:
|
||||||
|
///
|
||||||
|
/// 1. **No board window.** Not an empty one, not one with the good lanes in it — fail-fast is
|
||||||
|
/// all-or-nothing, so a partial board on screen would be the worse failure.
|
||||||
|
/// 2. **Welcome, loudly.** The recents row for that board wears the loader's own sentence, which
|
||||||
|
/// names the offending file. A row that fell back to "Unavailable", or to a count, would be the
|
||||||
|
/// app declining to say what it found.
|
||||||
|
/// 3. **Nothing repaired.** The bytes on disk are the bytes the fixture wrote. The loader is a pure
|
||||||
|
/// function of the tree and writes nothing, ever — the Repair precedent — so a board it refused
|
||||||
|
/// must still be refusable, byte for byte.
|
||||||
|
///
|
||||||
|
/// ### Where each claim is checked
|
||||||
|
///
|
||||||
|
/// The first two are here, because they are about *windows* and a window is what a unit test does not
|
||||||
|
/// have. The third is checked **both** here and in `KanbanTests` — unconditionally there
|
||||||
|
/// (`UITestMalformedFixtureBoardTests`, which builds the fixture and re-reads the tree), and
|
||||||
|
/// opportunistically here, because reaching the app's container from the runner depends on how the
|
||||||
|
/// app under test was signed and installed. Where the container is not reachable this file says so
|
||||||
|
/// and leans on the unit suite rather than inventing a pass.
|
||||||
|
final class FailFastLaunchTests: XCTestCase {
|
||||||
|
|
||||||
|
override func setUp() {
|
||||||
|
super.setUp()
|
||||||
|
continueAfterFailure = false
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The whole of claims 1 and 2, in one launch: no board window, welcome instead, and the row
|
||||||
|
/// carrying the loader's specifics.
|
||||||
|
@MainActor
|
||||||
|
func testMalformedBoardFailsLoudlyAndOpensNoWindow() throws {
|
||||||
|
let app = XCUIApplication.launched(with: .malformed)
|
||||||
|
|
||||||
|
// Welcome is where a failed open lands (`BoardWindowHost.start`: record the failure, refresh
|
||||||
|
// the recents, open welcome, dismiss the board window).
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.windows["Welcome to Lanework"].waitForExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"the welcome window did not appear after a failed open"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Claim 1. Checked *after* welcome has appeared, so this is "the board window never came",
|
||||||
|
// not "the board window has not come yet".
|
||||||
|
XCTAssertFalse(
|
||||||
|
app.windows[FixtureBoard.malformed.windowTitle].exists,
|
||||||
|
"a board window opened for a board the loader rejected"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Claim 2. The row is one combined accessibility element — name, location, caption — and the
|
||||||
|
// caption is `BoardLoadError.description`: "⟨lane⟩/⟨card⟩/index.md: unparseable YAML at line
|
||||||
|
// N: …". The UUIDs in that path are minted at launch and unknowable here, so the assertion is
|
||||||
|
// on the parts that are the *app's* to keep stable: the offending file is named, and the
|
||||||
|
// reason is stated.
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.element(labelContaining: "index.md").waitForExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"no welcome row named the offending index.md — fail-fast's specifics did not reach the surface"
|
||||||
|
)
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.element(labelContaining: "unparseable YAML").exists,
|
||||||
|
"the welcome row did not say why the board was refused"
|
||||||
|
)
|
||||||
|
// And it is the malformed board's own row that says it.
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.element(labelContaining: FixtureBoard.malformed.windowTitle).exists,
|
||||||
|
"the failure did not land on the failed board's row"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Claim 3, twice over: the malformed bytes survive the refusal, and a second launch is refused
|
||||||
|
/// the same way rather than opening a board the app quietly fixed.
|
||||||
|
///
|
||||||
|
/// The relaunch is not redundant with the byte check — it is what the byte check *means* from the
|
||||||
|
/// user's side, and it is the half that holds even where the container cannot be read.
|
||||||
|
@MainActor
|
||||||
|
func testMalformedBoardIsNeverRepaired() throws {
|
||||||
|
let app = XCUIApplication.launched(with: .malformed)
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.windows["Welcome to Lanework"].waitForExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"the welcome window did not appear after a failed open"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The bytes, where the runner can reach them. `NSTemporaryDirectory()` inside the sandboxed
|
||||||
|
// app resolves to its container, which an unsandboxed test runner can read — but only when
|
||||||
|
// the app under test is installed where this path expects, so a miss is reported rather than
|
||||||
|
// failed. `UITestMalformedFixtureBoardTests` makes the same claim unconditionally.
|
||||||
|
if let malformed = Self.malformedIndexOnDisk() {
|
||||||
|
XCTAssertTrue(
|
||||||
|
malformed.contains(Self.malformationMarker),
|
||||||
|
"the malformed index.md no longer carries its marker — something rewrote a file the loader refused to read"
|
||||||
|
)
|
||||||
|
XCTAssertTrue(
|
||||||
|
malformed.contains("order: [1024"),
|
||||||
|
"the malformed frontmatter was repaired — fail-fast must not write"
|
||||||
|
)
|
||||||
|
} else {
|
||||||
|
// Not a failure, and not silence either: the run says which half of the claim it made.
|
||||||
|
XCTContext.runActivity(named: "container not reachable from the runner") { _ in
|
||||||
|
print("""
|
||||||
|
The app's fixture scratch directory could not be read from the test runner, so \
|
||||||
|
the on-disk half of "nothing was repaired" was not checked here. It is pinned \
|
||||||
|
unconditionally by KanbanTests ▸ UITestMalformedFixtureBoardTests.
|
||||||
|
""")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The relaunch. A fresh launch rebuilds the fixture from scratch (the scratch root is wiped
|
||||||
|
// per launch), so what this proves is the durable half: the app has no repair path that
|
||||||
|
// would make the second attempt succeed where the first failed.
|
||||||
|
app.terminate()
|
||||||
|
let second = XCUIApplication.launched(with: .malformed)
|
||||||
|
XCTAssertTrue(
|
||||||
|
second.windows["Welcome to Lanework"].waitForExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"the second launch did not reach welcome"
|
||||||
|
)
|
||||||
|
XCTAssertFalse(
|
||||||
|
second.windows[FixtureBoard.malformed.windowTitle].exists,
|
||||||
|
"the second launch opened the board the first one refused"
|
||||||
|
)
|
||||||
|
XCTAssertTrue(
|
||||||
|
second.element(labelContaining: "index.md").waitForExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"the second launch did not name the offending file"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - Reading the app's container
|
||||||
|
|
||||||
|
/// The string that appears only in the malformed file — `UITestLaunch.malformationMarker`,
|
||||||
|
/// mirrored here for `FixtureBoard`'s reason.
|
||||||
|
private static let malformationMarker = "lanework-ui-test-malformed-fixture"
|
||||||
|
|
||||||
|
/// The malformed card's `index.md`, read from the app's sandbox container — or `nil` when the
|
||||||
|
/// runner cannot reach it.
|
||||||
|
///
|
||||||
|
/// The card folder's name is a UUID minted at launch, so the file is found by its content rather
|
||||||
|
/// than by its path: exactly one `index.md` under the fixture board carries the marker, which is
|
||||||
|
/// what the marker is for.
|
||||||
|
@MainActor
|
||||||
|
private static func malformedIndexOnDisk() -> String? {
|
||||||
|
let home = URL(fileURLWithPath: NSHomeDirectory(), isDirectory: true)
|
||||||
|
let board = home
|
||||||
|
.appendingPathComponent("Library/Containers/dev.rzen.indie.Kanban/Data/tmp", isDirectory: true)
|
||||||
|
.appendingPathComponent("LaneworkUITestFixture", isDirectory: true)
|
||||||
|
.appendingPathComponent("\(FixtureBoard.malformed.windowTitle).kanban", isDirectory: true)
|
||||||
|
|
||||||
|
guard let walker = FileManager.default.enumerator(atPath: board.path) else { return nil }
|
||||||
|
for case let relative as String in walker where relative.hasSuffix("index.md") {
|
||||||
|
guard let data = try? Data(contentsOf: board.appendingPathComponent(relative)) else { continue }
|
||||||
|
let text = String(decoding: data, as: UTF8.self)
|
||||||
|
if text.contains(malformationMarker) { return text }
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,11 +1,20 @@
|
|||||||
import XCTest
|
import XCTest
|
||||||
|
|
||||||
|
/// The bundle's smoke test: the app launches and puts a window on screen.
|
||||||
|
///
|
||||||
|
/// It launches on the **standard fixture** rather than bare, which is a deliberate narrowing of what
|
||||||
|
/// it used to prove. A bare launch consults the registry's restore flags, so on a developer's machine
|
||||||
|
/// it opens that developer's real boards — real watchers over real documents, and a recents list
|
||||||
|
/// stamped by a test run. Every other test in this bundle goes out of its way to avoid exactly that
|
||||||
|
/// (`UITestLaunch` redirects the registry into a scratch directory for the same reason), and one test
|
||||||
|
/// quietly opting out would make "a UI run never touches your real boards" false.
|
||||||
|
///
|
||||||
|
/// What is given up is the launch-to-*welcome* path, which no longer has a UI test. It is not
|
||||||
|
/// untested: `LaunchPlanTests` pins the decision that produces it, and it is the path a developer
|
||||||
|
/// exercises every time they run the app from Xcode.
|
||||||
final class KanbanUITests: XCTestCase {
|
final class KanbanUITests: XCTestCase {
|
||||||
@MainActor
|
@MainActor
|
||||||
func testAppLaunches() throws {
|
func testAppLaunches() throws {
|
||||||
let app = XCUIApplication()
|
_ = XCUIApplication.launchedWithFixtureBoard()
|
||||||
app.launch()
|
|
||||||
XCTAssertTrue(app.wait(for: .runningForeground, timeout: 30))
|
|
||||||
XCTAssertTrue(app.windows.firstMatch.waitForExistence(timeout: 30))
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,135 @@
|
|||||||
|
import XCTest
|
||||||
|
|
||||||
|
/// **Performance sanity on a big board** — 8 lanes × 40 cards (`UITestLaunch.FixtureVariant.large`),
|
||||||
|
/// measured on launch and on one interaction.
|
||||||
|
///
|
||||||
|
/// ### Why the budgets are wall-clock assertions and not baselines
|
||||||
|
///
|
||||||
|
/// `XCTMeasureOptions` and `XCTPerformanceMetric` baselines are stored *per device* in the Xcode
|
||||||
|
/// project's `xcshareddata`, keyed by a machine identifier. They do not travel: a fresh clone, a
|
||||||
|
/// different Mac, or CI has no baseline at all, so `measure` there records a number and passes
|
||||||
|
/// unconditionally. A "performance test" that passes unconditionally is not a gate — it is a log
|
||||||
|
/// line — and this repository has no CI to grow one on.
|
||||||
|
///
|
||||||
|
/// So each test does both, and they do different jobs:
|
||||||
|
///
|
||||||
|
/// - **The `measure` block records the metric**, so a developer reading the test report can see the
|
||||||
|
/// distribution and Xcode can offer a baseline locally to whoever wants one.
|
||||||
|
/// - **The explicit wall-clock assertion is the gate.** It is a real bound, checked everywhere, with
|
||||||
|
/// no stored state behind it.
|
||||||
|
///
|
||||||
|
/// ### The budgets, and what they are budgets for
|
||||||
|
///
|
||||||
|
/// They are **generous on purpose** and they are *regression* bounds, not targets. What they have to
|
||||||
|
/// catch is a change that makes the large board an order of magnitude worse — an O(n²) reflow, a
|
||||||
|
/// synchronous tree walk on the main thread, a per-card watcher — and what they must never do is fail
|
||||||
|
/// because a laptop was busy. The numbers below are roughly 5× the measured cost on the development
|
||||||
|
/// machine, which leaves a slow or loaded machine plenty of room while still failing a tenfold
|
||||||
|
/// regression.
|
||||||
|
///
|
||||||
|
/// | Budget | Value | What it covers |
|
||||||
|
/// | --- | --- | --- |
|
||||||
|
/// | `launchBudget` | 30 s | Process start, **materializing 328 folders through `BoardWriter`**, the load, and the first frame with a lane on it |
|
||||||
|
/// | `interactionBudget` | 5 s | View ▸ Show Trash on a 320-card board: the re-divide, every lane's reflow, and the column arriving |
|
||||||
|
///
|
||||||
|
/// The launch budget's dominant term is the fixture *build*, not the app: writing an `index.md` per
|
||||||
|
/// card takes about two seconds on the development machine (`UITestLargeFixtureBoardTests` measures
|
||||||
|
/// the same work without a window). That is deliberate — the number stays honest about what a launch
|
||||||
|
/// on this fixture actually costs — but it is why the budget is not tighter, and why a launch
|
||||||
|
/// regression shows up here as a big move rather than a small one.
|
||||||
|
///
|
||||||
|
/// ### Running these
|
||||||
|
///
|
||||||
|
/// Like the rest of `KanbanUITests`: a real, unlocked display and Accessibility automation
|
||||||
|
/// permission. They are also the slowest tests in the repository — each one builds the large board
|
||||||
|
/// once per `measure` iteration — so they are excluded from the ordinary run and invoked by name.
|
||||||
|
/// `KanbanUITests/EndToEndVerification.md` carries the command.
|
||||||
|
final class LargeBoardPerformanceTests: XCTestCase {
|
||||||
|
|
||||||
|
/// Launch to a board window with the large fixture on it.
|
||||||
|
static let launchBudget: TimeInterval = 30
|
||||||
|
|
||||||
|
/// One board-wide interaction: View ▸ Show Trash, which re-divides every lane.
|
||||||
|
static let interactionBudget: TimeInterval = 5
|
||||||
|
|
||||||
|
override func setUp() {
|
||||||
|
super.setUp()
|
||||||
|
continueAfterFailure = false
|
||||||
|
}
|
||||||
|
|
||||||
|
/// **Launch-to-board**, measured with `XCTApplicationLaunchMetric` and gated by
|
||||||
|
/// `launchBudget`.
|
||||||
|
///
|
||||||
|
/// `.manuallyStart`, so the measured span is the launch and the window — not the teardown of the
|
||||||
|
/// previous iteration's app, which would otherwise be folded into the number.
|
||||||
|
///
|
||||||
|
/// The gate is timed separately, before the measure block, for two reasons: a `measure` body runs
|
||||||
|
/// five times by default and asserting inside it would report one failure per iteration, and the
|
||||||
|
/// first launch is the one a user experiences (the later ones benefit from a warm dyld cache and
|
||||||
|
/// a warm filesystem).
|
||||||
|
@MainActor
|
||||||
|
func testLargeBoardLaunchPerformance() throws {
|
||||||
|
let started = Date()
|
||||||
|
let first = XCUIApplication.launchedWithFixtureBoard(.large)
|
||||||
|
let elapsed = Date().timeIntervalSince(started)
|
||||||
|
first.terminate()
|
||||||
|
|
||||||
|
XCTAssertLessThan(
|
||||||
|
elapsed,
|
||||||
|
Self.launchBudget,
|
||||||
|
"launching onto the large fixture took \(String(format: "%.1f", elapsed))s, over the \(Self.launchBudget)s budget"
|
||||||
|
)
|
||||||
|
|
||||||
|
let options = XCTMeasureOptions()
|
||||||
|
options.invocationOptions = [.manuallyStart]
|
||||||
|
measure(metrics: [XCTApplicationLaunchMetric()], options: options) {
|
||||||
|
let app = XCUIApplication()
|
||||||
|
app.launchArguments += FixtureBoard.large.launchArguments
|
||||||
|
startMeasuring()
|
||||||
|
app.launch()
|
||||||
|
_ = app.windows[FixtureBoard.large.windowTitle].waitForExistence(timeout: XCUIApplication.uiTimeout)
|
||||||
|
stopMeasuring()
|
||||||
|
app.terminate()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// **One interaction on a full board**: View ▸ Show Trash, which 03-board-ui.md § Trash calls "a
|
||||||
|
/// re-divide trigger, a lane add's behaviour exactly" — so every one of the eight lanes
|
||||||
|
/// recomputes its width and relays its 40 cards.
|
||||||
|
///
|
||||||
|
/// Chosen over select-all because it is *observable*: the column arriving is an element the test
|
||||||
|
/// can wait for, so the span measured is the whole interaction rather than the round trip of a
|
||||||
|
/// keystroke whose completion nothing announces. Select-all's cost is covered indirectly — it
|
||||||
|
/// runs inside the same window, over the same 320 cards, in `SelectionGrammarTests`.
|
||||||
|
///
|
||||||
|
/// The app is launched **once**, outside the measurement, and the toggle is measured over and
|
||||||
|
/// back: showing and hiding are the same re-divide in two directions, and a pair per iteration is
|
||||||
|
/// what leaves the board in the state the next iteration starts from.
|
||||||
|
@MainActor
|
||||||
|
func testLargeBoardShowTrashLatency() throws {
|
||||||
|
let app = XCUIApplication.launchedWithFixtureBoard(.large)
|
||||||
|
defer { app.terminate() }
|
||||||
|
|
||||||
|
// The first toggle is timed on its own and is the gate — it is the one that pays for whatever
|
||||||
|
// the board has not laid out yet, which is exactly the cost a user feels.
|
||||||
|
let started = Date()
|
||||||
|
app.clickMenuItem("Show Trash", in: "View")
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.element(labeled: Phrase.trash).waitForExistence(timeout: XCUIApplication.uiTimeout),
|
||||||
|
"the trash column never appeared on the large board"
|
||||||
|
)
|
||||||
|
let elapsed = Date().timeIntervalSince(started)
|
||||||
|
|
||||||
|
XCTAssertLessThan(
|
||||||
|
elapsed,
|
||||||
|
Self.interactionBudget,
|
||||||
|
"showing the trash on the large board took \(String(format: "%.1f", elapsed))s, over the \(Self.interactionBudget)s budget"
|
||||||
|
)
|
||||||
|
|
||||||
|
// And the recorded metric, over the toggle in both directions.
|
||||||
|
measure(metrics: [XCTClockMetric()]) {
|
||||||
|
app.clickMenuItem("Show Trash", in: "View")
|
||||||
|
app.clickMenuItem("Show Trash", in: "View")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,343 @@
|
|||||||
|
import XCTest
|
||||||
|
|
||||||
|
// MARK: - The fixture boards
|
||||||
|
|
||||||
|
/// **The three boards the app can build for a UI test**, mirrored from
|
||||||
|
/// `UITestLaunch.FixtureVariant`.
|
||||||
|
///
|
||||||
|
/// It is a *mirror* and not an import because a UI test bundle does not link the app it drives: the
|
||||||
|
/// two ends of a launch argument are always a pair of literals, and this is the one place this end of
|
||||||
|
/// the pair is spelled. The app end is `UITestLaunch.FixtureVariant.flag` / `.boardTitle`; a change
|
||||||
|
/// to either without the matching change here shows up as a launch that opens welcome and a wait that
|
||||||
|
/// times out, which is the loudest failure available for a string pair.
|
||||||
|
enum FixtureBoard {
|
||||||
|
|
||||||
|
/// The audit board — three lanes ("To Do", "Doing", "Done"), six cards, one with a rich Markdown
|
||||||
|
/// body and an attachment, one already in `.trash/`. The end-to-end flows use it too: every one
|
||||||
|
/// of them asserts on a lane's spoken card count, and a shape small enough to state in a sentence
|
||||||
|
/// is a shape those assertions can be read against.
|
||||||
|
case standard
|
||||||
|
|
||||||
|
/// 8 lanes × 40 cards, for the performance pass. Nothing in it is interesting; there is simply a
|
||||||
|
/// lot of it.
|
||||||
|
case large
|
||||||
|
|
||||||
|
/// A well-formed board with exactly one unparseable card `index.md`, for the fail-fast pass. It
|
||||||
|
/// is the only variant whose board window is *expected* never to appear.
|
||||||
|
case malformed
|
||||||
|
|
||||||
|
/// `UITestLaunch.fixtureFlag` plus this variant's own flag. Both, always: the first is what "this
|
||||||
|
/// launch is synthetic" means (it is what redirects the registry into the scratch directory), the
|
||||||
|
/// second is which board.
|
||||||
|
var launchArguments: [String] {
|
||||||
|
switch self {
|
||||||
|
case .standard: ["--ui-test-fixture-board", "--ui-test-fixture-standard"]
|
||||||
|
case .large: ["--ui-test-fixture-board", "--ui-test-fixture-large"]
|
||||||
|
case .malformed: ["--ui-test-fixture-board", "--ui-test-fixture-malformed"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The board's title, which is also its window title (`BoardWindowHost.windowTitle`) and its
|
||||||
|
/// folder name.
|
||||||
|
var windowTitle: String {
|
||||||
|
switch self {
|
||||||
|
case .standard: "Audit Board"
|
||||||
|
case .large: "Large Board"
|
||||||
|
case .malformed: "Malformed Board"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - The standard board's shape
|
||||||
|
|
||||||
|
/// The audit/end-to-end board as the tests address it: lane titles, card titles, and the labels
|
||||||
|
/// those produce.
|
||||||
|
///
|
||||||
|
/// Mirrored from `UITestLaunch` for `FixtureBoard`'s reason, and stated as *names* rather than
|
||||||
|
/// scattered string literals so that a test reads as a sentence about a board rather than as a
|
||||||
|
/// sentence about strings.
|
||||||
|
enum StandardBoard {
|
||||||
|
|
||||||
|
static let toDo = "To Do"
|
||||||
|
static let doing = "Doing"
|
||||||
|
static let done = "Done"
|
||||||
|
|
||||||
|
/// The first lane's cards, in card order — minus `trashed`, which the fixture deleted on the way
|
||||||
|
/// out, so the lane renders three of these four.
|
||||||
|
static let firstLaneCards = [
|
||||||
|
"Draft the release notes", "Check the trash grammar", "Confirm the rotor jumps", "Ship the audit",
|
||||||
|
]
|
||||||
|
|
||||||
|
/// The card the fixture deleted into `.trash/` — the trash column's one occupant at launch.
|
||||||
|
static let trashed = "Check the trash grammar"
|
||||||
|
|
||||||
|
/// The sole card of "Doing" — the rich one, and the destination every cross-lane flow moves into.
|
||||||
|
static let richCard = "Write the smoke script"
|
||||||
|
|
||||||
|
/// The sole card of "Done".
|
||||||
|
static let doneCard = "Wire the launch fixture"
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - Phrases
|
||||||
|
|
||||||
|
/// The spoken vocabulary the board's elements wear, mirrored from `AccessibilityPhrases`.
|
||||||
|
///
|
||||||
|
/// The board's containers are addressed by these labels and nothing else — a lane's element *is*
|
||||||
|
/// "⟨title⟩, lane, N cards" — which makes a lane's card count assertable without any identifier of
|
||||||
|
/// our own. That is deliberate: the count is the one thing every one of these flows changes, and
|
||||||
|
/// asserting on the label asserts the visible badge, the spoken count and the layout all at once,
|
||||||
|
/// because 10-accessibility.md makes them one number.
|
||||||
|
enum Phrase {
|
||||||
|
|
||||||
|
/// "3 cards", "1 card" — the app's one plural folding for a card count.
|
||||||
|
static func cards(_ count: Int) -> String {
|
||||||
|
"\(count) card\(count == 1 ? "" : "s")"
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A lane container's label: "⟨title⟩, lane, N cards".
|
||||||
|
static func lane(_ title: String, cards count: Int) -> String {
|
||||||
|
"\(title), lane, \(cards(count))"
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The trash container's label. Its *value* is `cards(count)`; the label is the bare word.
|
||||||
|
static let trash = "Trash"
|
||||||
|
|
||||||
|
/// The untitled placeholder — what a lane created by File ▸ New Lane wears until it is renamed.
|
||||||
|
static let untitled = "Untitled"
|
||||||
|
|
||||||
|
/// The inline title editor's placeholders (`NewCardStubView`, `LaneView`, `CardFaceView`).
|
||||||
|
static let cardTitlePrompt = "Card title"
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - Driving the app
|
||||||
|
|
||||||
|
extension XCUIApplication {
|
||||||
|
|
||||||
|
/// One timeout for the whole suite. Generous, because a cold launch here also builds a board on
|
||||||
|
/// disk, and because a UI test that is slow is not a UI test that is wrong.
|
||||||
|
static let uiTimeout: TimeInterval = 30
|
||||||
|
|
||||||
|
/// The first element anywhere in the app carrying `label`, whatever type it turned out to be.
|
||||||
|
///
|
||||||
|
/// SwiftUI decides for itself which `XCUIElement.ElementType` a flattened accessibility element
|
||||||
|
/// lands as — a card face and a lane container are both "one element" by design and neither is
|
||||||
|
/// promised to be a `staticText` — so the waits in this file match on the label, which *is*
|
||||||
|
/// specified (`AccessibilityPhrases`), rather than on a type that is not.
|
||||||
|
@MainActor
|
||||||
|
func element(labeled label: String) -> XCUIElement {
|
||||||
|
descendants(matching: .any)
|
||||||
|
.matching(NSPredicate(format: "label == %@", label))
|
||||||
|
.firstMatch
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The first element anywhere in the app whose label *contains* `fragment`.
|
||||||
|
///
|
||||||
|
/// Used only where the whole label is not the test's to predict — a welcome row combines a board
|
||||||
|
/// name, a location and a caption into one element, and the caption is the part under test.
|
||||||
|
@MainActor
|
||||||
|
func element(labelContaining fragment: String) -> XCUIElement {
|
||||||
|
descendants(matching: .any)
|
||||||
|
.matching(NSPredicate(format: "label CONTAINS %@", fragment))
|
||||||
|
.firstMatch
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Launches the app on `board` — the audit fixture unless told otherwise — and waits for its
|
||||||
|
/// window.
|
||||||
|
///
|
||||||
|
/// **Not for `.malformed`**, which has no window to wait for: that variant is launched with
|
||||||
|
/// `launched(with:)` and the caller waits for welcome instead.
|
||||||
|
@MainActor
|
||||||
|
static func launchedWithFixtureBoard(_ board: FixtureBoard = .standard) -> XCUIApplication {
|
||||||
|
let app = launched(with: board)
|
||||||
|
// Waiting on *this* window rather than on `windows.firstMatch` is what makes a failed fixture
|
||||||
|
// build a failure here instead of an audit that quietly passed over the welcome window.
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.windows[board.windowTitle].waitForExistence(timeout: uiTimeout),
|
||||||
|
"the \(board.windowTitle) window did not open — check the app's launch log for a fixture build failure"
|
||||||
|
)
|
||||||
|
return app
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Launches the app on `board` and returns as soon as it is frontmost — no window assertion, so
|
||||||
|
/// the caller decides what should have appeared.
|
||||||
|
@MainActor
|
||||||
|
static func launched(with board: FixtureBoard) -> XCUIApplication {
|
||||||
|
let app = XCUIApplication()
|
||||||
|
app.launchArguments += board.launchArguments
|
||||||
|
app.launch()
|
||||||
|
XCTAssertTrue(
|
||||||
|
app.wait(for: .runningForeground, timeout: uiTimeout),
|
||||||
|
"the app did not reach the foreground"
|
||||||
|
)
|
||||||
|
return app
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Clicks a menu-bar row by its title.
|
||||||
|
///
|
||||||
|
/// **By title, because the titles are API** (04-interactions.md ▸ Configurable bindings: menu
|
||||||
|
/// items are remapped by title through `NSUserKeyEquivalents`, so the strings in
|
||||||
|
/// `KanbanApp.menuCommands` are already a published contract). Driving the menus rather than
|
||||||
|
/// typing the chords also means these tests exercise the same path a keyboard user takes, and a
|
||||||
|
/// row that silently disabled itself fails here as an un-hittable element rather than as a
|
||||||
|
/// keystroke that went nowhere.
|
||||||
|
@MainActor
|
||||||
|
func clickMenuItem(_ title: String, in menu: String) {
|
||||||
|
let bar = menuBars.firstMatch
|
||||||
|
let menuBarItem = bar.menuBarItems[menu]
|
||||||
|
XCTAssertTrue(
|
||||||
|
menuBarItem.waitForExistence(timeout: Self.uiTimeout),
|
||||||
|
"the \(menu) menu is missing from the menu bar"
|
||||||
|
)
|
||||||
|
menuBarItem.click()
|
||||||
|
|
||||||
|
let item = bar.menuItems[title]
|
||||||
|
XCTAssertTrue(
|
||||||
|
item.waitForExistence(timeout: Self.uiTimeout),
|
||||||
|
"\(menu) ▸ \(title) is missing"
|
||||||
|
)
|
||||||
|
XCTAssertTrue(item.isEnabled, "\(menu) ▸ \(title) is disabled")
|
||||||
|
item.click()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a menu row is currently enabled — opened, read, and closed again with Escape.
|
||||||
|
///
|
||||||
|
/// Its own helper because *asking* is a different act from *clicking*: the template-chooser flow
|
||||||
|
/// needs to know that Choose is live without pressing it, since pressing it raises a save panel
|
||||||
|
/// this suite cannot drive.
|
||||||
|
@MainActor
|
||||||
|
func menuItemIsEnabled(_ title: String, in menu: String) -> Bool {
|
||||||
|
let bar = menuBars.firstMatch
|
||||||
|
let menuBarItem = bar.menuBarItems[menu]
|
||||||
|
guard menuBarItem.waitForExistence(timeout: Self.uiTimeout) else { return false }
|
||||||
|
menuBarItem.click()
|
||||||
|
let item = bar.menuItems[title]
|
||||||
|
let enabled = item.waitForExistence(timeout: Self.uiTimeout) && item.isEnabled
|
||||||
|
typeKey(.escape, modifierFlags: [])
|
||||||
|
return enabled
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Selects the fixture's rich card and opens its window.
|
||||||
|
///
|
||||||
|
/// **The selection is seeded with one arrow press**, which is the app's own documented rule:
|
||||||
|
/// "an empty selection seeds at the first lane's first card" (04-interactions.md ▸ Grammar,
|
||||||
|
/// `BoardView.seed`). From there the walk to the rich card is ordinary arrow navigation — the
|
||||||
|
/// same keys a user without a pointer would press — so this helper needs to know no identifiers
|
||||||
|
/// and no geometry, only the fixture's shape.
|
||||||
|
///
|
||||||
|
/// The rich card is the sole card of lane 2 ("Doing"), so: one press to seed into lane 1's first
|
||||||
|
/// card, then one `→` to step into lane 2. `→` steps to the nearest card in that direction, and
|
||||||
|
/// with one card in the lane there is only one it can be.
|
||||||
|
@MainActor
|
||||||
|
func openRichCardWindow() throws {
|
||||||
|
typeKey(.downArrow, modifierFlags: [])
|
||||||
|
typeKey(.rightArrow, modifierFlags: [])
|
||||||
|
clickMenuItem("Open Card", in: "Board")
|
||||||
|
XCTAssertTrue(
|
||||||
|
windows[StandardBoard.richCard].waitForExistence(timeout: Self.uiTimeout),
|
||||||
|
"the card window did not open"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - Board assertions
|
||||||
|
|
||||||
|
/// Waits for a lane to be rendering exactly `count` cards, by its label.
|
||||||
|
///
|
||||||
|
/// **This is how every flow in the end-to-end suite asserts that a move landed.** A card's own
|
||||||
|
/// element carries only its title, which does not change when it changes lanes; the lane's does,
|
||||||
|
/// because the label carries the count. So "the card moved from To Do to Doing" is checkable as
|
||||||
|
/// two lane labels and needs no identifier, no geometry and no reading of the disk.
|
||||||
|
@MainActor
|
||||||
|
@discardableResult
|
||||||
|
func awaitLane(_ title: String, cards count: Int, _ message: String = "") -> Bool {
|
||||||
|
let label = Phrase.lane(title, cards: count)
|
||||||
|
let found = element(labeled: label).waitForExistence(timeout: Self.uiTimeout)
|
||||||
|
XCTAssertTrue(found, "expected a lane reading \"\(label)\"\(message.isEmpty ? "" : " — \(message)")")
|
||||||
|
return found
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Waits for the trash column to be rendering exactly `count` cards.
|
||||||
|
///
|
||||||
|
/// The trash's *label* is the bare word (it never states a count — 03-board-ui.md § Trash keeps
|
||||||
|
/// the header's title stable), so the count lives in its value and this matches on the pair.
|
||||||
|
@MainActor
|
||||||
|
@discardableResult
|
||||||
|
func awaitTrash(cards count: Int) -> Bool {
|
||||||
|
let predicate = NSPredicate(
|
||||||
|
format: "label == %@ AND value == %@", Phrase.trash, Phrase.cards(count)
|
||||||
|
)
|
||||||
|
let element = descendants(matching: .any).matching(predicate).firstMatch
|
||||||
|
let found = element.waitForExistence(timeout: Self.uiTimeout)
|
||||||
|
XCTAssertTrue(found, "expected the trash column to hold \(Phrase.cards(count))")
|
||||||
|
return found
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Clicks a card by its title — the ordinary single click, which selects it
|
||||||
|
/// (`BoardStore.click`).
|
||||||
|
@MainActor
|
||||||
|
func selectCard(_ title: String) {
|
||||||
|
let card = element(labeled: title)
|
||||||
|
XCTAssertTrue(card.waitForExistence(timeout: Self.uiTimeout), "the card \"\(title)\" is not on the board")
|
||||||
|
card.click()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Waits for an inline title editor and answers it — the shared body of every flow that types a
|
||||||
|
/// title (`InlineTitleField`).
|
||||||
|
///
|
||||||
|
/// The editor is born focused, so the typing goes to the app rather than to a located element;
|
||||||
|
/// what is located is only the *field*, and only so that a flow which never opened one fails here
|
||||||
|
/// rather than typing its title into the board's keyboard grammar.
|
||||||
|
///
|
||||||
|
/// **Matched by placeholder first, then by "the only text field on screen".** The board window's
|
||||||
|
/// search field is an `NSSearchField` and so is not a `textField` at all, which leaves the inline
|
||||||
|
/// editor as the sole candidate; the placeholder match is the precise one and the fallback is
|
||||||
|
/// what keeps a SwiftUI change to how prompts reach the accessibility tree from failing every
|
||||||
|
/// flow at once.
|
||||||
|
@MainActor
|
||||||
|
func typeIntoInlineEditor(_ text: String, prompt: String, replacingExisting: Bool = false) {
|
||||||
|
let byPlaceholder = textFields
|
||||||
|
.matching(NSPredicate(format: "placeholderValue == %@", prompt))
|
||||||
|
.firstMatch
|
||||||
|
if !byPlaceholder.waitForExistence(timeout: 5) {
|
||||||
|
XCTAssertTrue(
|
||||||
|
textFields.firstMatch.waitForExistence(timeout: Self.uiTimeout),
|
||||||
|
"the inline title editor (\(prompt)) did not appear"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if replacingExisting {
|
||||||
|
// The rename editor opens seeded with the current title; Select All inside a focused
|
||||||
|
// field is the field editor's, not the board's (the board's own Select All disables
|
||||||
|
// while an inline editor is open — 04-interactions.md ▸ Grammar).
|
||||||
|
typeKey("a", modifierFlags: .command)
|
||||||
|
}
|
||||||
|
typeText(text)
|
||||||
|
typeKey(.return, modifierFlags: [])
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Drags one card onto another — the pointer path (04-interactions.md ▸ Drag and drop), which is
|
||||||
|
/// a **real AppKit dragging session**: `CardFaceView` starts it from `.onDrag` with an
|
||||||
|
/// `NSItemProvider`, and the lane strip answers with `onDrop` delegates.
|
||||||
|
///
|
||||||
|
/// **This is the most environment-sensitive gesture in the suite**, and deliberately the slowest.
|
||||||
|
/// The press has to outlast the click-versus-drag threshold before the session arms, the movement
|
||||||
|
/// has to be slow enough that the drop delegates see intermediate positions (the reflow is
|
||||||
|
/// computed from them), and the hold at the end has to outlast the drop animation. A failure here
|
||||||
|
/// is worth re-running once before it is believed.
|
||||||
|
@MainActor
|
||||||
|
func dragCard(_ title: String, onto destinationTitle: String) {
|
||||||
|
let source = element(labeled: title)
|
||||||
|
let destination = element(labeled: destinationTitle)
|
||||||
|
XCTAssertTrue(source.waitForExistence(timeout: Self.uiTimeout), "the card \"\(title)\" is not on the board")
|
||||||
|
XCTAssertTrue(
|
||||||
|
destination.waitForExistence(timeout: Self.uiTimeout),
|
||||||
|
"the drop target \"\(destinationTitle)\" is not on the board"
|
||||||
|
)
|
||||||
|
|
||||||
|
let centre = CGVector(dx: 0.5, dy: 0.5)
|
||||||
|
source.coordinate(withNormalizedOffset: centre).press(
|
||||||
|
forDuration: 1,
|
||||||
|
thenDragTo: destination.coordinate(withNormalizedOffset: centre),
|
||||||
|
withVelocity: .slow,
|
||||||
|
thenHoldForDuration: 1
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -59,14 +59,24 @@ The Xcode project is generated — `project.yml` is the source of truth, not the
|
|||||||
```sh
|
```sh
|
||||||
xcodegen generate
|
xcodegen generate
|
||||||
xcodebuild build -project Kanban.xcodeproj -scheme Kanban -destination 'platform=macOS'
|
xcodebuild build -project Kanban.xcodeproj -scheme Kanban -destination 'platform=macOS'
|
||||||
xcodebuild test -project Kanban.xcodeproj -scheme Kanban -destination 'platform=macOS,arch=arm64'
|
xcodebuild test -project Kanban.xcodeproj -scheme Kanban -destination 'platform=macOS,arch=arm64' -only-testing:KanbanTests
|
||||||
xcodebuild build -project Kanban.xcodeproj -scheme LaneworkPro -destination 'platform=macOS'
|
xcodebuild build -project Kanban.xcodeproj -scheme LaneworkPro -destination 'platform=macOS'
|
||||||
xcodebuild test -project Kanban.xcodeproj -scheme LaneworkPro -destination 'platform=macOS,arch=arm64'
|
xcodebuild test -project Kanban.xcodeproj -scheme LaneworkPro -destination 'platform=macOS,arch=arm64'
|
||||||
scripts/verify-editions.sh
|
scripts/verify-editions.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`-only-testing:KanbanTests` is the everyday run: the unit suite needs nothing but a compiler. Dropping it also runs `KanbanUITests`, which launches the real app and drives its menu bar — that needs a real, unlocked display and Accessibility automation permission, and on a machine without them every test in it fails on its first click for a reason that is not the app's. On such a machine, check the UI suites compile instead:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
xcodebuild build-for-testing -project Kanban.xcodeproj -scheme Kanban -destination 'platform=macOS,arch=arm64'
|
||||||
|
```
|
||||||
|
|
||||||
macOS 26+, Swift 6 (strict concurrency), SwiftUI, sandboxed. Internal codename `Kanban` (target, scheme, bundle id `dev.rzen.indie.Kanban`); the app ships under the display name **Lanework**.
|
macOS 26+, Swift 6 (strict concurrency), SwiftUI, sandboxed. Internal codename `Kanban` (target, scheme, bundle id `dev.rzen.indie.Kanban`); the app ships under the display name **Lanework**.
|
||||||
|
|
||||||
Two app targets are built from one source tree (DESIGN/12-editions.md): **Lanework** compiles `Kanban/` alone, **Lanework Pro** (target `KanbanPro`, scheme `LaneworkPro`, bundle id `dev.rzen.indie.KanbanPro`) compiles `Kanban/` plus the Pro-only source root `KanbanPro/`. There is no edition flag and no `#if` in shared code — an edition difference is a file one target builds and the other does not — and the difference is checkable on the signed products: base carries no libgit2 and no network-client entitlement, which `scripts/verify-editions.sh` asserts against the built bundles. The unit suite is edition-agnostic and runs twice, once hosted by each app (`KanbanTests`, `KanbanProTests` — the same sources, bound to the Pro module by `-module-alias`). Both editions declare the same `dev.rzen.indie.kanban-board` UTI, so any board opens in either app.
|
Two app targets are built from one source tree (DESIGN/12-editions.md): **Lanework** compiles `Kanban/` alone, **Lanework Pro** (target `KanbanPro`, scheme `LaneworkPro`, bundle id `dev.rzen.indie.KanbanPro`) compiles `Kanban/` plus the Pro-only source root `KanbanPro/`. There is no edition flag and no `#if` in shared code — an edition difference is a file one target builds and the other does not — and the difference is checkable on the signed products: base carries no libgit2 and no network-client entitlement, which `scripts/verify-editions.sh` asserts against the built bundles. The unit suite is edition-agnostic and runs twice, once hosted by each app (`KanbanTests`, `KanbanProTests` — the same sources, bound to the Pro module by `-module-alias`). Both editions declare the same `dev.rzen.indie.kanban-board` UTI, so any board opens in either app.
|
||||||
|
|
||||||
**Accessibility is verified, not assumed** (DESIGN/10-accessibility.md § Verification). `KanbanUITests/AccessibilityAuditTests.swift` runs Xcode's accessibility audit over all eight surfaces the design names — the board with the trash shown and hidden, the card window in Preview, Edit and raw source, welcome, the template chooser, the board popover — and every violation is a test failure with nothing waived. Each test launches the app with `--ui-test-fixture-board`, a test-only argument that makes the app build a known three-lane board through its own `BoardWriter` inside its sandbox container, with its own scratch registry, so a run never touches real boards or real recents (`Kanban/App/UITestLaunch.swift`). The manual half — the per-release VoiceOver smoke script and the consolidated accessibility checklist — is `KanbanUITests/AccessibilityVerification.md`. Both halves need a real, unlocked display and Accessibility automation permission.
|
**Accessibility is verified, not assumed** (DESIGN/10-accessibility.md § Verification). `KanbanUITests/AccessibilityAuditTests.swift` runs Xcode's accessibility audit over all eight surfaces the design names — the board with the trash shown and hidden, the card window in Preview, Edit and raw source, welcome, the template chooser, the board popover — and every violation is a test failure with nothing waived. The manual half — the per-release VoiceOver smoke script and the consolidated accessibility checklist — is `KanbanUITests/AccessibilityVerification.md`.
|
||||||
|
|
||||||
|
**The golden paths are verified end to end too.** `KanbanUITests/EndToEndFlowTests.swift` drives create card, create lane, inline rename, a pointer drag across lanes, cut/paste across lanes, undo and redo of a move, and the whole current trash grammar — delete into `.trash/` with no confirmation, View ▸ Show Trash, restore by ⌘X/⌘V back out, and Empty Trash… with its confirmation. `FailFastLaunchTests.swift` launches onto a board with one unparseable `index.md` and asserts the loud failure: no board window, welcome carrying the loader's own sentence naming the offending file, and nothing on disk repaired. `LargeBoardPerformanceTests.swift` measures launch and one interaction against an 8 × 40 board under explicit wall-clock budgets (XCTest baselines do not travel between machines, so the gate is an assertion rather than a baseline). The three flows that stay manual — instantiating a template, Duplicate's save-panel fallback, and File ▸ Open… — are the sandbox's Powerbox panels, which live in another process; they are written down as manual steps in `KanbanUITests/EndToEndVerification.md` rather than automated flakily.
|
||||||
|
|
||||||
|
Every UI test launches the app with `--ui-test-fixture-board` plus a variant flag (`--ui-test-fixture-standard`, `--ui-test-fixture-large`, `--ui-test-fixture-malformed`), a test-only argument pair that makes the app build a known board through its own `BoardWriter` inside its sandbox container, with its own scratch registry, so a run never touches real boards or real recents (`Kanban/App/UITestLaunch.swift`). The fixture builders are themselves unit-tested, so a fixture that came out wrong fails somewhere that runs everywhere. All of the UI suites need a real, unlocked display and Accessibility automation permission; run commands and prerequisites are in `KanbanUITests/EndToEndVerification.md`.
|
||||||
|
|||||||
+8
-4
@@ -197,15 +197,19 @@ targets:
|
|||||||
type: bundle.ui-testing
|
type: bundle.ui-testing
|
||||||
platform: macOS
|
platform: macOS
|
||||||
sources:
|
sources:
|
||||||
# The manual verification document lives with the tests it belongs to (10-accessibility.md
|
# The verification documents live with the tests they belong to (10-accessibility.md
|
||||||
# ▸ Verification: "A manual VoiceOver smoke script lives with the test plan"), so it is listed
|
# ▸ Verification: "A manual VoiceOver smoke script lives with the test plan"), so they are
|
||||||
# in the project — visible where the suite is — but with no build phase: it is documentation,
|
# listed in the project — visible where the suite is — but with no build phase: they are
|
||||||
# not a resource the bundle should carry.
|
# documentation, not resources the bundle should carry. `EndToEndVerification.md` is the same
|
||||||
|
# arrangement for the golden-flow, fail-fast and performance suites: the run command, the
|
||||||
|
# prerequisites, and the flows that stayed manual.
|
||||||
- path: KanbanUITests
|
- path: KanbanUITests
|
||||||
excludes:
|
excludes:
|
||||||
- "**/*.md"
|
- "**/*.md"
|
||||||
- path: KanbanUITests/AccessibilityVerification.md
|
- path: KanbanUITests/AccessibilityVerification.md
|
||||||
buildPhase: none
|
buildPhase: none
|
||||||
|
- path: KanbanUITests/EndToEndVerification.md
|
||||||
|
buildPhase: none
|
||||||
dependencies:
|
dependencies:
|
||||||
- target: Kanban
|
- target: Kanban
|
||||||
settings:
|
settings:
|
||||||
|
|||||||
Reference in New Issue
Block a user