The automated half: AccessibilityAuditTests runs performAccessibilityAudit over all eight surfaces DESIGN/10 names — board with trash hidden and shown, card window in Preview/Edit/raw source, welcome, template chooser, board popover. One audit per test, .all audit types, no issue handler — nothing waived; a future false-positive excusal must match one element on one surface with its reason beside it. Navigation is menu-bar titles and the arrow grammar; no accessibility identifiers added to production code. The suite launches with --ui-test-fixture-board: the sandbox forbids handing the app a temp-folder path (no bookmark behind it), so the flag carries no payload and the app builds a known board inside its own container through the ordinary BoardWriter door — three lanes, six cards, a rich Markdown body with attachment, one card already in .trash/ — with the registry redirected to the same scratch directory so audit runs never pollute real recents. LaunchPlan replaces the restore Bool (welcome / restoreBoards / uiTestFixture, fixture wins outright), decided once in KanbanApp.init and dispatched by RestoreBootstrapView; pure and pinned by UITestLaunchTests, and the fixture itself is materialized and read back through BoardLoader in units — the only proof available headlessly. The manual half: KanbanUITests/AccessibilityVerification.md is the one document — the audit suite at the top (it needs a real display and Accessibility permission), the per-release VoiceOver smoke script with expected utterances quoted from AccessibilityPhrases, and the consolidated m11 checklist from all four implementation cards. 1588 unit tests green, UI target compiles, both schemes build. The audit run and smoke script await a real display — the manual pass is the user's. Claude-Session: https://claude.ai/code/session_01SR4XGjmBE16ZUYWpfFHXwY
265 lines
13 KiB
Swift
265 lines
13 KiB
Swift
import XCTest
|
|
|
|
/// **The automated half of 10-accessibility.md ▸ Verification**, whose first clause is the reason
|
|
/// this file is a set of tests and not a checklist:
|
|
///
|
|
/// > **Automated audits are test failures**: Xcode's accessibility audit (`performAccessibilityAudit`)
|
|
/// > runs in UI tests over every surface — board (trash shown and hidden), card window (Preview, Edit,
|
|
/// > raw source), welcome, template chooser, board popover.
|
|
///
|
|
/// Eight surfaces, eight tests, one audit call each. `performAccessibilityAudit` audits **the app's
|
|
/// currently displayed UI** rather than a subtree, so each test's job is entirely navigation: get the
|
|
/// surface on screen, then let the audit look at whatever is there.
|
|
///
|
|
/// ### No waiving
|
|
///
|
|
/// The audit's issue handler is where a false positive would be excused, and every test here passes
|
|
/// `nil` — no handler, nothing excused. That is the design's own posture ("violations are test
|
|
/// failures, not warnings"), and it is worth keeping literal: a handler that swallowed one issue type
|
|
/// app-wide would also swallow the next real one of that type, on a surface nobody was thinking about
|
|
/// when the waiver was written. Should a genuine platform false positive ever need excusing, it goes
|
|
/// in as a closure that matches **that one element on that one surface** and carries the reason in a
|
|
/// comment beside it — never a bare `return true`.
|
|
///
|
|
/// ### `.all`, not a narrowed set
|
|
///
|
|
/// `XCUIAccessibilityAuditType.all` is the default and stays the default. The narrower types
|
|
/// (`.contrast`, `.elementDetection`, `.hitRegion`, `.sufficientElementDescription`, `.textClipped`,
|
|
/// `.trait`) each map onto a rule 10-accessibility.md states — contrast is its ≥ 4.5:1 clause,
|
|
/// sufficient-description is its labels, trait is its selection and heading traits — so scoping any
|
|
/// of them out would be scoping out a design rule. They are named here only so a future narrowing has
|
|
/// to argue with this paragraph first.
|
|
///
|
|
/// ### The fixture board
|
|
///
|
|
/// Every test launches with `UITestLaunch.fixtureFlag`, which makes the app build a known board
|
|
/// inside its own container and open it (see `UITestLaunch` for why the board cannot simply be
|
|
/// handed over on the command line — the sandbox). The board is three lanes, six cards, one card
|
|
/// with a rich Markdown body and an attachment, and one card already in `.trash/`.
|
|
///
|
|
/// ### Running these
|
|
///
|
|
/// They drive the real app through the real menu bar, so the machine running them must have granted
|
|
/// the test runner Accessibility control (System Settings ▸ Privacy & Security ▸ Accessibility) and
|
|
/// must not be locked or headless. A run that cannot get automation permission fails on the first
|
|
/// `click()`, not on an accessibility defect — see `KanbanUITests/AccessibilityVerification.md`,
|
|
/// which puts this suite at the top of the manual pass for exactly that reason.
|
|
///
|
|
/// ### Reading a failure
|
|
///
|
|
/// The navigation waits are deliberately loud, and they are also the part most likely to need
|
|
/// adjusting: a window is identified by its **title** (`app.windows["…"]`, which is what
|
|
/// `navigationTitle` produces) and an on-board element by its **label**. Both are specified —
|
|
/// 11-command-nexus.md fixes the menu titles, `AccessibilityPhrases` fixes the labels — but neither
|
|
/// says which `XCUIElement.ElementType` SwiftUI will choose, and a hidden-title-bar window (welcome)
|
|
/// is the one place a title might not surface at all. So: a failure on `performAccessibilityAudit`
|
|
/// is an accessibility defect and is what this suite is for; a failure on a `waitForExistence` above
|
|
/// it is a navigation problem in *this file*, and the audit never ran.
|
|
final class AccessibilityAuditTests: XCTestCase {
|
|
|
|
override func setUp() {
|
|
super.setUp()
|
|
// A failed navigation step makes every later step in that test meaningless — and an audit
|
|
// that ran against the wrong surface would report a *pass*, which is worse than a failure.
|
|
continueAfterFailure = false
|
|
}
|
|
|
|
// MARK: - The board window
|
|
|
|
/// The board as it opens: lanes, cards, the toolbar, the search field — trash hidden, which is
|
|
/// the board's default state (03-board-ui.md § Trash).
|
|
@MainActor
|
|
func testBoardWindowWithTrashHidden() throws {
|
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
|
try app.performAccessibilityAudit()
|
|
}
|
|
|
|
/// The same board with View ▸ Show Trash on — "when shown, it is the last container, labeled as
|
|
/// Trash with its count" (10-accessibility.md ▸ Trash lane), holding the fixture's one trashed
|
|
/// card so the column's own card elements are audited and not just its header.
|
|
@MainActor
|
|
func testBoardWindowWithTrashShown() throws {
|
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
|
app.clickMenuItem("Show Trash", in: "View")
|
|
XCTAssertTrue(
|
|
app.element(labeled: "Trash").waitForExistence(timeout: XCUIApplication.uiTimeout),
|
|
"the trash column did not appear"
|
|
)
|
|
try app.performAccessibilityAudit()
|
|
}
|
|
|
|
// MARK: - The card window
|
|
|
|
/// Preview mode — the card window's default. The fixture's rich card carries the structures 10
|
|
/// makes claims about (headings, lists, a task list, a table, a code block, a link, an image with
|
|
/// alt text), so this is the audit of "Preview renders to the accessibility tree as structured
|
|
/// text" rather than of an empty body.
|
|
@MainActor
|
|
func testCardWindowPreviewMode() throws {
|
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
|
try app.openRichCardWindow()
|
|
try app.performAccessibilityAudit()
|
|
}
|
|
|
|
/// Edit mode — View ▸ Edit Body (⌘E), "an ordinary accessible text editor" (10 ▸ Card window).
|
|
@MainActor
|
|
func testCardWindowEditMode() throws {
|
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
|
try app.openRichCardWindow()
|
|
app.clickMenuItem("Edit Body", in: "View")
|
|
try app.performAccessibilityAudit()
|
|
}
|
|
|
|
/// The raw-source outlet — View ▸ Raw Source (⌥⌘E), the whole `index.md` as text.
|
|
///
|
|
/// Entered from Preview rather than from Edit, because the two are mutually exclusive by design
|
|
/// ("Edit Body disables while Raw Source is active" — 05-card-window.md ▸ Raw source outlet) and
|
|
/// stacking them would be auditing a state the app does not have.
|
|
@MainActor
|
|
func testCardWindowRawSourceMode() throws {
|
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
|
try app.openRichCardWindow()
|
|
app.clickMenuItem("Raw Source", in: "View")
|
|
try app.performAccessibilityAudit()
|
|
}
|
|
|
|
// MARK: - Welcome, the template chooser, the board popover
|
|
|
|
/// The welcome window, reached by its own Window-menu row — and reached *after* the fixture board
|
|
/// has opened, so its recents list has a row in it. An empty welcome would audit the empty state
|
|
/// and miss the rows 10 specifies ("⟨name⟩, ⟨location⟩, N lanes, M cards").
|
|
@MainActor
|
|
func testWelcomeWindow() throws {
|
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
|
app.clickMenuItem("Welcome to Lanework", in: "Window")
|
|
XCTAssertTrue(
|
|
app.windows["Welcome to Lanework"].waitForExistence(timeout: XCUIApplication.uiTimeout),
|
|
"the welcome window did not appear"
|
|
)
|
|
try app.performAccessibilityAudit()
|
|
}
|
|
|
|
/// The template chooser — File ▸ New Board… (⌥⌘N), the ten bundled templates as elements
|
|
/// "labeled by title", their mini previews hidden (10 ▸ Template chooser).
|
|
@MainActor
|
|
func testTemplateChooser() throws {
|
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
|
app.clickMenuItem("New Board…", in: "File")
|
|
XCTAssertTrue(
|
|
app.windows["New Board"].waitForExistence(timeout: XCUIApplication.uiTimeout),
|
|
"the template chooser did not appear"
|
|
)
|
|
try app.performAccessibilityAudit()
|
|
}
|
|
|
|
/// The board popover — File ▸ Board Info (⌘I): "labeled controls throughout", with the git slot's
|
|
/// information readable as text and never by colour or shape alone (10 ▸ Board popover).
|
|
@MainActor
|
|
func testBoardInfoPopover() throws {
|
|
let app = XCUIApplication.launchedWithFixtureBoard()
|
|
app.clickMenuItem("Board Info", in: "File")
|
|
XCTAssertTrue(
|
|
app.popovers.firstMatch.waitForExistence(timeout: XCUIApplication.uiTimeout),
|
|
"the board popover did not appear"
|
|
)
|
|
try app.performAccessibilityAudit()
|
|
}
|
|
}
|
|
|
|
// MARK: - Driving the app
|
|
|
|
extension XCUIApplication {
|
|
|
|
/// One timeout for the whole suite. Generous, because a cold launch here also builds a board on
|
|
/// disk, and because a UI test that is slow is not a UI test that is wrong.
|
|
static let uiTimeout: TimeInterval = 30
|
|
|
|
/// The first element anywhere in the app carrying `label`, whatever type it turned out to be.
|
|
///
|
|
/// SwiftUI decides for itself which `XCUIElement.ElementType` a flattened accessibility element
|
|
/// lands as — a card face and a lane container are both "one element" by design and neither is
|
|
/// promised to be a `staticText` — so the waits in this file match on the label, which *is*
|
|
/// specified (`AccessibilityPhrases`), rather than on a type that is not.
|
|
@MainActor
|
|
func element(labeled label: String) -> XCUIElement {
|
|
descendants(matching: .any)
|
|
.matching(NSPredicate(format: "label == %@", label))
|
|
.firstMatch
|
|
}
|
|
|
|
/// Launches the app on the audit fixture board and waits for its window.
|
|
///
|
|
/// The flag's spelling is `UITestLaunch.fixtureFlag` in the app target; it is written out here
|
|
/// because a UI test bundle does not link the app it drives — the two ends of a launch argument
|
|
/// are always a pair of literals, and this is the one place this end of the pair is spelled.
|
|
@MainActor
|
|
static func launchedWithFixtureBoard() -> XCUIApplication {
|
|
let app = XCUIApplication()
|
|
app.launchArguments += ["--ui-test-fixture-board"]
|
|
app.launch()
|
|
XCTAssertTrue(
|
|
app.wait(for: .runningForeground, timeout: uiTimeout),
|
|
"the app did not reach the foreground"
|
|
)
|
|
// `UITestLaunch.boardTitle`, which is also the window title (`BoardWindowHost.windowTitle`).
|
|
// Waiting on *this* window rather than on `windows.firstMatch` is what makes a failed fixture
|
|
// build a failure here instead of an audit that quietly passed over the welcome window.
|
|
XCTAssertTrue(
|
|
app.windows["Audit Board"].waitForExistence(timeout: uiTimeout),
|
|
"the fixture board window did not open — check the app's launch log for a fixture build failure"
|
|
)
|
|
return app
|
|
}
|
|
|
|
/// Clicks a menu-bar row by its title.
|
|
///
|
|
/// **By title, because the titles are API** (04-interactions.md ▸ Configurable bindings: menu
|
|
/// items are remapped by title through `NSUserKeyEquivalents`, so the strings in
|
|
/// `KanbanApp.menuCommands` are already a published contract). Driving the menus rather than
|
|
/// typing the chords also means these tests exercise the same path a keyboard user takes, and a
|
|
/// row that silently disabled itself fails here as an un-hittable element rather than as a
|
|
/// keystroke that went nowhere.
|
|
@MainActor
|
|
func clickMenuItem(_ title: String, in menu: String) {
|
|
let bar = menuBars.firstMatch
|
|
let menuBarItem = bar.menuBarItems[menu]
|
|
XCTAssertTrue(
|
|
menuBarItem.waitForExistence(timeout: Self.uiTimeout),
|
|
"the \(menu) menu is missing from the menu bar"
|
|
)
|
|
menuBarItem.click()
|
|
|
|
let item = bar.menuItems[title]
|
|
XCTAssertTrue(
|
|
item.waitForExistence(timeout: Self.uiTimeout),
|
|
"\(menu) ▸ \(title) is missing"
|
|
)
|
|
XCTAssertTrue(item.isEnabled, "\(menu) ▸ \(title) is disabled")
|
|
item.click()
|
|
}
|
|
|
|
/// Selects the fixture's rich card and opens its window.
|
|
///
|
|
/// **The selection is seeded with one arrow press**, which is the app's own documented rule:
|
|
/// "an empty selection seeds at the first lane's first card" (04-interactions.md ▸ Grammar,
|
|
/// `BoardView.seed`). From there the walk to the rich card is ordinary arrow navigation — the
|
|
/// same keys a user without a pointer would press — so this helper needs to know no identifiers
|
|
/// and no geometry, only the fixture's shape.
|
|
///
|
|
/// The rich card is the sole card of lane 2 ("Doing"), so: one press to seed into lane 1's first
|
|
/// card, then one `→` to step into lane 2. `→` steps to the nearest card in that direction, and
|
|
/// with one card in the lane there is only one it can be.
|
|
@MainActor
|
|
func openRichCardWindow() throws {
|
|
typeKey(.downArrow, modifierFlags: [])
|
|
typeKey(.rightArrow, modifierFlags: [])
|
|
clickMenuItem("Open Card", in: "Board")
|
|
// `UITestLaunch.cardTitles[1][0]`, which is the card window's title
|
|
// (`CardWindowHost.windowTitle`).
|
|
XCTAssertTrue(
|
|
windows["Write the smoke script"].waitForExistence(timeout: Self.uiTimeout),
|
|
"the card window did not open"
|
|
)
|
|
}
|
|
}
|