mqtt-5-test/docs/sprints/01-foundation.md

Sprint 1 — Go foundation, configuration, contracts and event model

Status: complete; verified locally on 2026-10-06. Read the plan, sprint index and shared rules, and README.

Goal and scope

Produce a runnable configuration-checking CLI and validated message/event contracts without connecting to a broker.

Deliverables and likely files

• go.mod, go.sum, .gitignore.
• cmd/test-mqtt5/main.go with configuration validation and help.
• internal/config/, internal/contracts/, internal/events/ and focused tests.
• config.example.toml, README startup/configuration instructions, docs/CONTRACTS.md.

Tasks

1. Verify the proposed Go/koanf/parser/provider pins against official metadata and establish the module. Keep MQTT dependency installation until needed if that avoids unused modules.
2. Implement defaults < TOML < environment < explicitly visited flags. Map TEST_MQTT5_ with double-underscore nesting; support --config, --device-id, --log-level.
3. Decode typed durations/numbers/bools, reject unknown settings, and validate TLS URL, credentials source, namespace/IDs, QoS, deadlines, certificate pair, limits, simulator and optional web settings.
4. Define telemetry/status/command/ACK structs and validation; allow only ping/get_version. Define explicit ACK stages and application deadlines.
5. Define sanitized event fields, bounded previews, secret redaction, and direction/outcome vocabulary. Do not echo raw malformed or binary payloads.
6. Add signal-aware command structure, contextual errors, and text/JSON slog output. Document examples without secrets and ignore local config/secret/data artifacts.

Dependencies and required inputs

Sprint authorization; an available compatible Go toolchain and dependency access. No server/hardware input is required.

Acceptance criteria

• CLI validates example configuration and exits successfully without MQTT or HTTP connections.
• Explicit CLI overrides win; unspecified flags do not override TOML/environment. Invalid configuration fails with key-specific errors and no secrets.
• Payload examples validate; wrong identity, stage, command, timestamp, size, or missing ID fails.
• Safe event previews preserve useful ping/version input/output while redacting sensitive fields.
• No files in existing projects are changed.

Automated checks

gofmt, go vet, focused config/contracts/events tests; precedence table including empty values and invalid durations; redaction and truncation tests. Run race detection where concurrent code exists.

Manual verification

Run help and configuration validation with defaults/example/local overrides. Inspect text and JSON error output for clarity and secret absence.

Risks and unresolved decisions

Exact toolchain availability and dependency resolution; generated module metadata may reveal incompatible pins. Report and resolve before claiming compatibility.

Mandatory end-of-sprint testing and completion gate

After implementing this sprint, run its automated checks and manual verification above. Inspect the final changes for correctness, scope, secret exposure, and compatibility with completed sprints. Fix failures and rerun affected checks before declaring completion.

Record the exact commands/checks, actual outcomes, verification environment, and each acceptance criterion as PASS, FAIL, or BLOCKED. Include applicable regression checks for previously completed behavior. A skipped/unavailable required check is BLOCKED, never PASS; do not claim full sprint completion while required checks fail or remain blocked. Do not run unrelated or production-project tests.

Report the results and remaining limitations to the user. Wait for authorization before the next sprint unless continued execution was explicitly authorized. Even with permission to continue, do not bypass this verification gate.

Copy-ready implementation prompt

Implement Sprint 1 only in /home/ryan/Media/Code/Code/robot-market/test-mqtt-5 after the user authorizes implementation. Read applicable AGENTS.md, /home/ryan/.codex/RTK.md, README.md, docs/PLAN.md and docs/sprints/01-foundation.md. Prefix shell commands with rtk. Existing sibling projects are read-only; do not deploy or contact a production broker.

Build a small Go CLI that validates configuration without networking. Verify proposed pins (Go 1.27.1; koanf/v2 v2.3.7; TOML v2 parser v2.2.2; file provider v1.2.1; env provider v1.1.0) and report required deviations. Implement typed configuration with defaults < TOML < TEST_MQTT5_ environment < explicitly supplied flags; use __ for nesting and --config. Validate durations, TLS, identities, QoS, session/deadline settings and bounded resource limits. Add versioned telemetry/status/ping/get_version/ACK contracts and validation. Define shared sanitized input/output events for logs and a future browser viewer, including redaction and bounded previews. Add example TOML, secret/data ignore rules and focused contract/configuration docs.

Use idiomatic Go, slog, contextual errors and signal cancellation. Run formatting, vet and meaningful config/contract/redaction tests. Demonstrate successful config validation and actionable invalid-config errors without leaking secrets. Do not implement MQTT, the viewer, remote deployment or device integration. Report actual checks, limitations and acceptance results, then wait for the next sprint authorization.

ESP32 reference: read docs/PLAN.md section 6.1 for the supplied Esp32MqttConfig. It is connection configuration, not a payload schema or a blocker for Go implementation. Preserve the proposed Go test contracts. Device-specific MQTT 5 selection depends on the actual C library/build/init; inspect TLS/CA handling, role-specific credentials and single publish_topic limitations during device integration. Do not modify existing C projects. Separate MQTT 3.1.1 payload interoperability from native MQTT 5 feature evidence.

Mandatory completion gate: after implementing this sprint, run its listed automated and manual checks plus relevant regression checks, inspect the final changes for correctness/scope/secrets/compatibility, fix failures and rerun affected checks. Report exact checks, real outcomes, environment, and every acceptance criterion as PASS/FAIL/BLOCKED. Missing required checks are BLOCKED, never a pass. Do not mark the sprint complete with unresolved failures/blockers or bypass verification when continuing to another sprint.

Completion evidence

Environment: local Linux/amd64, Go 1.27.1. All authored files are confined to test-mqtt-5. No existing server/device project was built, tested or modified. No remote broker or physical device was contacted.

Pinned Go configuration modules resolved without version deviations; go.mod/go.sum record the resolved versions in this project (no Git commit was created). Paho remains deferred to the MQTT sprint.

Acceptance criterion	Result	Evidence
Example configuration validates without MQTT/HTTP connections	PASS	Built CLI and go run check-config both succeed; output says network_checked=false
Explicit flag precedence; unspecified flags preserve file/env	PASS	Configuration precedence tests and CLI override smoke check
Invalid config produces key-specific errors without secrets	PASS	Validation table, malformed TOML, unknown keys, invalid identity and secret sentinel smoke checks
Valid/invalid payload contracts behave as specified	PASS	Telemetry/status/command/ACK tests; duplicate keys, nulls, deadline boundaries, identity/stage/version/size checks
Safe previews preserve useful fields and redact sensitive input	PASS	Preview tests for ping, free-form error redaction, malformed/binary/unknown fields and bounds
Existing projects remain unchanged by this work	PASS	All edit commands target the new project; caches and test binary use /tmp

Actual checks:

• rtk proxy gofmt -l cmd internal: PASS, no unformatted files.
• rtk proxy env GOMODCACHE=/tmp/test-mqtt5-mod GOCACHE=/tmp/test-mqtt5-build go vet ./...: PASS.
• Same cache settings with go test ./...: PASS, all four packages.
• Same cache settings with go test -race ./...: PASS, all four packages.
• Same cache settings with go mod verify: PASS.
• Same cache settings with go build -buildvcs=false -o /tmp/test-mqtt5-cli ./cmd/test-mqtt5: PASS.
• Same cache settings with go run ./cmd/test-mqtt5 --config config.example.toml check-config: PASS.
• Built CLI help/default/example/explicit flag checks: PASS.
• Built CLI JSON format with secret-bearing environment, invalid identity and malformed secret-bearing TOML: PASS; expected exit codes, values absent from output.
• Canceled-context CLI test: PASS.

The initial standalone build could not obtain VCS metadata from the surrounding checkout; disabling VCS stamping fixed the build without changing Git state. Initial compilation caught a koanf API mismatch; it was corrected before passing all checks.

Limits: file validation checks existence/readability/size, not certificate contents or credential correctness. No network/session/ACK correlation/idempotent execution/hardware verification is claimed. Future event producers must sanitize and bound transport property metadata. Remote and physical-device checks: NOT VERIFIED.

Next: Sprint 2 isolated local TLS broker, after authorization.