Test suite tour
Test suite tour
Lettuce ships with ~800 tests across three test targets. This is the map.
Targets
| Target | What it covers | Approx. tests |
|---|---|---|
LettuceKitTests |
Renderer, link index, tag index, frontmatter, DocsVault, schema | ~500 |
LettuceServerTests |
MCP server, REST mirror, auth, concurrency, soft locks | ~200 |
LettuceAppTests |
CLI subcommands, init scaffolds, end-to-end flows | ~100 |
Running
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer swift test
Filters:
swift test --filter DocsVault # one test class
swift test --filter SuggestionsModeTests
swift test --filter "test.*Concurrency" # regex
With code coverage:
swift test --enable-code-coverage
xcrun llvm-cov report .build/arm64-apple-macosx/debug/LettucePackageTests.xctest/Contents/MacOS/LettucePackageTests \
-instr-profile=.build/arm64-apple-macosx/debug/codecov/default.profdata \
-ignore-filename-regex=".build|Tests"
What’s in LettuceKitTests/
RendererTests.swift— Markdown → HTML round-trips for ~50 inputs.LinkIndexTests.swift— wikilink parsing, edge cases (aliases, anchors, block refs).TagIndexTests.swift—#tagparsing, frontmatter merge, nested tags.FrontmatterTests.swift— YAML parsing, normalisation, schema validation.DocsVaultTests.swift— bundled docs presence and version stamp.DocsVaultEdgeCaseTests.swift— regeneration semantics on fresh/stale dirs.HTMLSanitizerTests.swift— XSS prevention on user content (live-tag escaping).Snapshots/— golden-master HTML outputs for canonical Markdown inputs.
What’s in LettuceServerTests/
MCPTests.swift— every tool’s happy path.MCPErrorTests.swift— every errorkindreturned correctly.MCPCoverageTests.swift— read-tool coverage (added to fix the 61% gap).RESTScenarioTests.swift,ReadEndpointTests.swift,WriteEndpointTests.swift— REST surface parity with MCP.AuthTests.swift— token validation, scope enforcement.ConcurrencyTests.swift,ConcurrencyStressTests.swift,SuggestionsConcurrencyTests.swift— soft locks, multi-agent races, proposal contention.MCPProposalsTests.swift,ProposalsServiceTests.swift,ProposalsEdgeCaseTests.swift—write_noteunder a propose-only key → queued proposal → proposals UI → accept flow.SchemaContractTests.swift— on-disk file format stability.
What’s in LettuceAppTests/
MainTests.swift— CLI subcommand dispatch and argument parsing.PackageScriptTests.swift— bundling, signing flow (no actual signing in CI).WikiInitTests.swift— scaffold writes expected files.IntegrationTests.swift— end-to-end: open vault, write note, agent reads it.
Property-based tests
A few tests use SwiftCheck-style property testing (vendored, lightweight):
LinkIndexPropertyTests.swift— for any set of notes, the link graph round-trips.RendererPropertyTests.swift— rendering never panics on arbitrary Markdown.HTMLSanitizerPropertyTests.swift— sanitised output never contains live disallowed tags.
Mutation tests
Scripts/test-mutations.sh performs simple AST-level mutations (e.g. flip == to !=, negate boolean returns) and runs the test suite to verify the change is caught. Current score: 10/10 caught on a representative sample.
Public API snapshot
Tests/LettuceKitTests/Snapshots/PublicAPI/LettuceKit.symbols.txt is a sorted list of every public symbol. Any change to it requires updating the snapshot, which forces a deliberate review of API surface changes.
swift test --filter PublicAPISnapshotTests
Diffs are surfaced in failures.
Performance tests
Two complementary files:
Tests/LettuceKitTests/ScalePerfTests.swift— a profiler on a synthetic ~5800-note vault. It times the hot paths (link-index build,MentionScanner, etc.) and printsPERF …lines to stdout; it doesn’t assert, so it’s for eyeballing scaling.Tests/LettuceKitTests/PerformanceBudgetTests.swift— the wall-time budgets that actually fail CI. Each test measures a known workload once and asserts it stays under a generous budget (with a warning when it crosses half the budget), so the suite catches real regressions rather than noise.
Doc-completeness
swift test --filter DocCompletenessTests
Compares the current set of undocumented public top-level symbols against Tests/LettuceKitTests/Snapshots/PublicDocs/LettuceKit.undocumented-toplevels.txt. New undocumented symbols must be either documented or added to the snapshot (forcing a deliberate “okay, this is fine” decision).
Currently 74 grandfathered. New code aspires to zero.