Deploy MQTT test backend with private viewer and Gitea CI/CD
This commit is contained in:
@@ -0,0 +1,104 @@
|
||||
# Sprint 1 — Go foundation, configuration, contracts and event model
|
||||
|
||||
Status: complete; verified locally on 2026-10-06. Read [the plan](../PLAN.md), [sprint index and shared rules](../SPRINTS.md), and [README](../../README.md).
|
||||
|
||||
|
||||
### Goal and scope
|
||||
|
||||
Produce a runnable configuration-checking CLI and validated message/event contracts without connecting to a broker.
|
||||
|
||||
### Deliverables and likely files
|
||||
|
||||
- `go.mod`, `go.sum`, `.gitignore`.
|
||||
- `cmd/test-mqtt5/main.go` with configuration validation and help.
|
||||
- `internal/config/`, `internal/contracts/`, `internal/events/` and focused tests.
|
||||
- `config.example.toml`, README startup/configuration instructions, `docs/CONTRACTS.md`.
|
||||
|
||||
### Tasks
|
||||
|
||||
1. Verify the proposed Go/koanf/parser/provider pins against official metadata and establish the module. Keep MQTT dependency installation until needed if that avoids unused modules.
|
||||
2. Implement defaults < TOML < environment < explicitly visited flags. Map `TEST_MQTT5_` with double-underscore nesting; support `--config`, `--device-id`, `--log-level`.
|
||||
3. Decode typed durations/numbers/bools, reject unknown settings, and validate TLS URL, credentials source, namespace/IDs, QoS, deadlines, certificate pair, limits, simulator and optional web settings.
|
||||
4. Define telemetry/status/command/ACK structs and validation; allow only ping/get_version. Define explicit ACK stages and application deadlines.
|
||||
5. Define sanitized event fields, bounded previews, secret redaction, and direction/outcome vocabulary. Do not echo raw malformed or binary payloads.
|
||||
6. Add signal-aware command structure, contextual errors, and text/JSON slog output. Document examples without secrets and ignore local config/secret/data artifacts.
|
||||
|
||||
### Dependencies and required inputs
|
||||
|
||||
Sprint authorization; an available compatible Go toolchain and dependency access. No server/hardware input is required.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- CLI validates example configuration and exits successfully without MQTT or HTTP connections.
|
||||
- Explicit CLI overrides win; unspecified flags do not override TOML/environment. Invalid configuration fails with key-specific errors and no secrets.
|
||||
- Payload examples validate; wrong identity, stage, command, timestamp, size, or missing ID fails.
|
||||
- Safe event previews preserve useful ping/version input/output while redacting sensitive fields.
|
||||
- No files in existing projects are changed.
|
||||
|
||||
### Automated checks
|
||||
|
||||
gofmt, go vet, focused config/contracts/events tests; precedence table including empty values and invalid durations; redaction and truncation tests. Run race detection where concurrent code exists.
|
||||
|
||||
### Manual verification
|
||||
|
||||
Run help and configuration validation with defaults/example/local overrides. Inspect text and JSON error output for clarity and secret absence.
|
||||
|
||||
### Risks and unresolved decisions
|
||||
|
||||
Exact toolchain availability and dependency resolution; generated module metadata may reveal incompatible pins. Report and resolve before claiming compatibility.
|
||||
|
||||
### Mandatory end-of-sprint testing and completion gate
|
||||
|
||||
After implementing this sprint, run its automated checks and manual verification above. Inspect the final changes for correctness, scope, secret exposure, and compatibility with completed sprints. Fix failures and rerun affected checks before declaring completion.
|
||||
|
||||
Record the exact commands/checks, actual outcomes, verification environment, and each acceptance criterion as PASS, FAIL, or BLOCKED. Include applicable regression checks for previously completed behavior. A skipped/unavailable required check is BLOCKED, never PASS; do not claim full sprint completion while required checks fail or remain blocked. Do not run unrelated or production-project tests.
|
||||
|
||||
Report the results and remaining limitations to the user. Wait for authorization before the next sprint unless continued execution was explicitly authorized. Even with permission to continue, do not bypass this verification gate.
|
||||
|
||||
### Copy-ready implementation prompt
|
||||
|
||||
```text
|
||||
Implement Sprint 1 only in /home/ryan/Media/Code/Code/robot-market/test-mqtt-5 after the user authorizes implementation. Read applicable AGENTS.md, /home/ryan/.codex/RTK.md, README.md, docs/PLAN.md and docs/sprints/01-foundation.md. Prefix shell commands with rtk. Existing sibling projects are read-only; do not deploy or contact a production broker.
|
||||
|
||||
Build a small Go CLI that validates configuration without networking. Verify proposed pins (Go 1.27.1; koanf/v2 v2.3.7; TOML v2 parser v2.2.2; file provider v1.2.1; env provider v1.1.0) and report required deviations. Implement typed configuration with defaults < TOML < TEST_MQTT5_ environment < explicitly supplied flags; use __ for nesting and --config. Validate durations, TLS, identities, QoS, session/deadline settings and bounded resource limits. Add versioned telemetry/status/ping/get_version/ACK contracts and validation. Define shared sanitized input/output events for logs and a future browser viewer, including redaction and bounded previews. Add example TOML, secret/data ignore rules and focused contract/configuration docs.
|
||||
|
||||
Use idiomatic Go, slog, contextual errors and signal cancellation. Run formatting, vet and meaningful config/contract/redaction tests. Demonstrate successful config validation and actionable invalid-config errors without leaking secrets. Do not implement MQTT, the viewer, remote deployment or device integration. Report actual checks, limitations and acceptance results, then wait for the next sprint authorization.
|
||||
|
||||
ESP32 reference: read docs/PLAN.md section 6.1 for the supplied Esp32MqttConfig. It is connection configuration, not a payload schema or a blocker for Go implementation. Preserve the proposed Go test contracts. Device-specific MQTT 5 selection depends on the actual C library/build/init; inspect TLS/CA handling, role-specific credentials and single publish_topic limitations during device integration. Do not modify existing C projects. Separate MQTT 3.1.1 payload interoperability from native MQTT 5 feature evidence.
|
||||
|
||||
Mandatory completion gate: after implementing this sprint, run its listed automated and manual checks plus relevant regression checks, inspect the final changes for correctness/scope/secrets/compatibility, fix failures and rerun affected checks. Report exact checks, real outcomes, environment, and every acceptance criterion as PASS/FAIL/BLOCKED. Missing required checks are BLOCKED, never a pass. Do not mark the sprint complete with unresolved failures/blockers or bypass verification when continuing to another sprint.
|
||||
```
|
||||
|
||||
## Completion evidence
|
||||
|
||||
Environment: local Linux/amd64, Go 1.27.1. All authored files are confined to test-mqtt-5. No existing server/device project was built, tested or modified. No remote broker or physical device was contacted.
|
||||
|
||||
Pinned Go configuration modules resolved without version deviations; go.mod/go.sum record the resolved versions in this project (no Git commit was created). Paho remains deferred to the MQTT sprint.
|
||||
|
||||
| Acceptance criterion | Result | Evidence |
|
||||
| --- | --- | --- |
|
||||
| Example configuration validates without MQTT/HTTP connections | PASS | Built CLI and go run check-config both succeed; output says network_checked=false |
|
||||
| Explicit flag precedence; unspecified flags preserve file/env | PASS | Configuration precedence tests and CLI override smoke check |
|
||||
| Invalid config produces key-specific errors without secrets | PASS | Validation table, malformed TOML, unknown keys, invalid identity and secret sentinel smoke checks |
|
||||
| Valid/invalid payload contracts behave as specified | PASS | Telemetry/status/command/ACK tests; duplicate keys, nulls, deadline boundaries, identity/stage/version/size checks |
|
||||
| Safe previews preserve useful fields and redact sensitive input | PASS | Preview tests for ping, free-form error redaction, malformed/binary/unknown fields and bounds |
|
||||
| Existing projects remain unchanged by this work | PASS | All edit commands target the new project; caches and test binary use /tmp |
|
||||
|
||||
Actual checks:
|
||||
|
||||
- `rtk proxy gofmt -l cmd internal`: PASS, no unformatted files.
|
||||
- `rtk proxy env GOMODCACHE=/tmp/test-mqtt5-mod GOCACHE=/tmp/test-mqtt5-build go vet ./...`: PASS.
|
||||
- Same cache settings with `go test ./...`: PASS, all four packages.
|
||||
- Same cache settings with `go test -race ./...`: PASS, all four packages.
|
||||
- Same cache settings with `go mod verify`: PASS.
|
||||
- Same cache settings with `go build -buildvcs=false -o /tmp/test-mqtt5-cli ./cmd/test-mqtt5`: PASS.
|
||||
- Same cache settings with `go run ./cmd/test-mqtt5 --config config.example.toml check-config`: PASS.
|
||||
- Built CLI help/default/example/explicit flag checks: PASS.
|
||||
- Built CLI JSON format with secret-bearing environment, invalid identity and malformed secret-bearing TOML: PASS; expected exit codes, values absent from output.
|
||||
- Canceled-context CLI test: PASS.
|
||||
|
||||
The initial standalone build could not obtain VCS metadata from the surrounding checkout; disabling VCS stamping fixed the build without changing Git state. Initial compilation caught a koanf API mismatch; it was corrected before passing all checks.
|
||||
|
||||
Limits: file validation checks existence/readability/size, not certificate contents or credential correctness. No network/session/ACK correlation/idempotent execution/hardware verification is claimed. Future event producers must sanitize and bound transport property metadata. Remote and physical-device checks: NOT VERIFIED.
|
||||
|
||||
Next: Sprint 2 isolated local TLS broker, after authorization.
|
||||
@@ -0,0 +1,96 @@
|
||||
# Sprint 2 — Server broker deployment package
|
||||
|
||||
Status: COMPLETE. Artifacts, server-container runtime and external TLS/plaintext connection checks passed on 2026-10-06. Read [the plan](../PLAN.md), [sprint index and shared rules](../SPRINTS.md), and [README](../../README.md).
|
||||
|
||||
|
||||
### Goal and scope
|
||||
|
||||
Prepare an independent server-deployable Mosquitto Compose package for the user to deploy. Include TLS and an optional authenticated plaintext listener per the user's updated request. Validate artifacts locally; defer runtime checks to the deployed server.
|
||||
|
||||
### Deliverables and likely files
|
||||
|
||||
- `deploy/compose.yaml`, broker configuration/ACL templates and health-check helper.
|
||||
- Local-only certificate/credential provisioning helper; generated secrets ignored.
|
||||
- `docs/DEPLOYMENT.md` server setup and verification instructions.
|
||||
|
||||
### Tasks
|
||||
|
||||
1. Verify Mosquitto 2.1.2-alpine image/platform availability and pin its digest; validate version-specific password/ACL plugin configuration.
|
||||
2. Publish TLS in the base server Compose file and plaintext only with an explicit override; both remain authenticated with identical ACLs. Enable persistence, bounded queues/packets/logs, and correct container ownership/modes.
|
||||
3. Generate a dedicated local test CA and server certificate with matching server hostname/IP and localhost SANs. Keep keys and generated credentials outside tracked files.
|
||||
4. Provision separate backend, device and health users. Restrict each device to its own topics; deny production/cross-device access.
|
||||
5. Add authenticated TLS subscribe/publish/receive health checking with a deadline and dedicated topic. Keep secrets out of arguments/output.
|
||||
6. Document start/stop/inspect, certificate trust, persistence cleanup, backup, update and rollback. Destructive reset remains explicit and test-instance-only.
|
||||
|
||||
### Dependencies and required inputs
|
||||
|
||||
Sprint 1; available Docker/Compose and registry access. If unavailable, report the blocker and propose an isolated native alternative. Remote server access is not needed.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- Deployed server broker starts and passes an actual authenticated MQTT health exchange.
|
||||
- Anonymous access, bad credentials, bad CA/hostname and forbidden topics fail as expected.
|
||||
- Per-device ACLs prevent cross-device publishing/subscription. Commands cannot reach production namespaces.
|
||||
- Graceful restart preserves selected retained/session state; evidence identifies graceful restart rather than crash durability.
|
||||
- Configuration and examples contain no real credentials; plaintext host access is enabled only through the explicit override and remains authenticated.
|
||||
|
||||
### Automated checks
|
||||
|
||||
Compose validation, broker config/startup validation using the pinned image, health exchange, positive/negative TLS/auth/ACL smoke checks. Do not print resolved secret-bearing Compose configuration.
|
||||
|
||||
### Manual verification
|
||||
|
||||
Inspect port binding, mounts, ownership, bounded logs and restart behavior. Verify server hostname trust and localhost health checking with the generated CA.
|
||||
|
||||
### Risks and unresolved decisions
|
||||
|
||||
Host container permissions, plugin paths in the exact image, architecture support, port conflicts. Remote OS/DNS/certificate policy remains deferred.
|
||||
|
||||
### Mandatory end-of-sprint testing and completion gate
|
||||
|
||||
After implementing this sprint, run its automated checks and manual verification above. Inspect the final changes for correctness, scope, secret exposure, and compatibility with completed sprints. Fix failures and rerun affected checks before declaring completion.
|
||||
|
||||
Record the exact commands/checks, actual outcomes, verification environment, and each acceptance criterion as PASS, FAIL, or BLOCKED. Include applicable regression checks for previously completed behavior. A skipped/unavailable required check is BLOCKED, never PASS; do not claim full sprint completion while required checks fail or remain blocked. Do not run unrelated or production-project tests.
|
||||
|
||||
Report the results and remaining limitations to the user. Wait for authorization before the next sprint unless continued execution was explicitly authorized. Even with permission to continue, do not bypass this verification gate.
|
||||
|
||||
### Copy-ready implementation prompt
|
||||
|
||||
```text
|
||||
Implement Sprint 2 only in /home/ryan/Media/Code/Code/robot-market/test-mqtt-5 after authorization. Read applicable AGENTS.md, /home/ryan/.codex/RTK.md, README.md, docs/PLAN.md and docs/sprints/02-local-broker.md. Prefix shell commands with rtk. Existing projects and production brokers are read-only. Do not deploy remotely.
|
||||
|
||||
Build/maintain the separate server Mosquitto deployment package, using verified 2.1.2-alpine and its platform-compatible immutable digest. The user will deploy it and provide endpoint/credential-file details. Check exact 2.1 password/ACL plugin syntax and image paths. Publish TLS in the base Compose file and optional authenticated plaintext in a separate override; do not weaken TLS certificate verification. Disable anonymous access. Add dedicated local test certificate generation, ignored secrets, separate backend/device/health identities, least-privilege per-device ACLs, persistence, bounded broker queues/logs and correct file ownership. Implement an authenticated TLS MQTT subscribe/publish/receive health check with safe secret handling. Document server start/stop/inspect, backup/update/rollback and remote prerequisites.
|
||||
|
||||
Verify real health exchange, invalid credentials/CA/hostname, forbidden/cross-device topics and graceful broker restart persistence. If container tooling is unavailable, report it and propose a native isolated alternative rather than claiming tests passed. Do not implement the Go MQTT application or viewer. Report acceptance results and remaining inputs, then wait.
|
||||
|
||||
ESP32 reference: read docs/PLAN.md section 6.1 for the supplied Esp32MqttConfig. It is connection configuration, not a payload schema or a blocker for Go implementation. Preserve the proposed Go test contracts. Device-specific MQTT 5 selection depends on the actual C library/build/init; inspect TLS/CA handling, role-specific credentials and single publish_topic limitations during device integration. Do not modify existing C projects. Separate MQTT 3.1.1 payload interoperability from native MQTT 5 feature evidence.
|
||||
|
||||
Mandatory completion gate: after implementing this sprint, run its listed automated and manual checks plus relevant regression checks, inspect the final changes for correctness/scope/secrets/compatibility, fix failures and rerun affected checks. Report exact checks, real outcomes, environment, and every acceptance criterion as PASS/FAIL/BLOCKED. Missing required checks are BLOCKED, never a pass. Do not mark the sprint complete with unresolved failures/blockers or bypass verification when continuing to another sprint.
|
||||
```
|
||||
|
||||
## Completion evidence and remaining server checks
|
||||
|
||||
User steering: deploy the broker on the server; provide a Compose package for operator deployment; support plaintext alongside TLS for testing. No server changes or permanent local broker were performed.
|
||||
|
||||
| Acceptance criterion | Status | Evidence / remaining action |
|
||||
| --- | --- | --- |
|
||||
| Deployable pinned broker config | PASS for artifacts | Verified official multi-platform digest; Mosquitto 2.1.2 --test-config exits 0 in a network-disabled disposable container |
|
||||
| Health MQTT exchange | PASS | User ran the verifier on the deployed server: authenticated TLS exchange passed |
|
||||
| Bad credentials / CA / hostname rejected | PASS | Deployed verifier rejected anonymous access, invalid credentials, untrusted CA and wrong hostname |
|
||||
| Per-device ACL isolation / production topic denial | PASS | Deployed verifier confirmed cross-device publish/read blocking and production topic denial |
|
||||
| Graceful retained/session persistence | PASS | Deployed verifier confirmed retained status and offline session recovery after graceful restart; crash durability not tested |
|
||||
| No tracked credentials; optional plaintext explicit | PASS | Generated secrets confined to /tmp for validation; ignored server paths; merged Compose ports/auth configuration checked |
|
||||
|
||||
Checks actually run locally:
|
||||
|
||||
- docker buildx imagetools inspect: 2.1.2-alpine digest verified; bare 2.1.2 tag was absent and corrected.
|
||||
- docker compose config --quiet for TLS-only and TLS+plaintext: PASS.
|
||||
- Compose JSON assertions: ports, non-root runtime, read-only mounts/filesystem, immutable image and log limits PASS; no credential environment values.
|
||||
- Python AST/helper --help and sh -n healthcheck: PASS.
|
||||
- prepare.py with throwaway /tmp output and owner 1000:1000: PASS, real OpenSSL generation and container-based password hashing.
|
||||
- Generated hashed passwords/secret modes/CA-key isolation: PASS without displaying values.
|
||||
- OpenSSL chain, configured hostname, localhost and 127.0.0.1 checks: PASS; wrong hostname rejected.
|
||||
- Repeat setup refuses overwrite; invalid hostname inputs rejected: PASS.
|
||||
- Exact pinned image --test-config with temporary mounts and --network none: PASS. No listening broker started.
|
||||
|
||||
On 2026-10-06 the user ran the corrected verify.py --insecure --restart on 109.122.252.129. All server-container checks passed, including optional authenticated plaintext reception. The initial cross-device failure was a verifier exit-code bug; the corrected verifier requires MQTT PUBACK Not authorized (135). Server runtime acceptance is complete. External authenticated MQTT 5 device telemetry publication/backend reception passed from the development computer on TLS 8884 with CA/hostname verification and plaintext 1884, using unique non-retained probes. Physical hardware remains NOT VERIFIED. See DEPLOYMENT.md for operator commands.
|
||||
@@ -0,0 +1,99 @@
|
||||
# Sprint 3 — MQTT 5 backend, simulator and visible message flow
|
||||
|
||||
Status: COMPLETE on 2026-10-06; local unit/race, CLI and dedicated remote-broker checks passed. Read [the plan](../PLAN.md), [sprint index and shared rules](../SPRINTS.md), and [README](../../README.md).
|
||||
|
||||
|
||||
### Goal and scope
|
||||
|
||||
Run the complete harmless local command/telemetry loop with structured input/output logs and counters.
|
||||
|
||||
### Deliverables and likely files
|
||||
|
||||
- `cmd/device-sim/main.go`; `cmd/test-mqtt5` observe/command modes.
|
||||
- `internal/mqtt/`, `internal/backend/`, `internal/simulator/`; event buffer and counters.
|
||||
- Focused unit/integration tests, README quick start and contract updates.
|
||||
|
||||
### Tasks
|
||||
|
||||
- Support the user-requested plaintext test connection through explicit mqtt.allow_plaintext=false-by-default and a mqtt:// URL only when enabled. Keep TLS verification intact, preserve test namespace/ACLs, and update configuration tests/example/docs.
|
||||
|
||||
1. Integrate `github.com/eclipse/paho.golang v0.23.0`/autopaho with verified TLS, deadlines, bounded jittered backoff and configured role/instance IDs.
|
||||
2. Implement explicit initial Clean Start/session expiry and subscription readiness/restoration. Copy callback input into bounded workers; define visible overflow behavior.
|
||||
3. Simulator publishes telemetry, retained online status and an offline Will; handles ping/get_version and emits accepted/completed ACKs.
|
||||
4. Register pending correlation before publish; use exact Response Topic and native Correlation Data; validate device/topic/request ID/properties and transport reason codes.
|
||||
5. Enforce application deadline plus MQTT Message Expiry; implement bounded deduplication with in-progress entries, cached ACK replay and conflicting-ID rejection.
|
||||
6. Produce readable text/JSON events for inbound/outbound payload previews/properties and lifecycle outcomes. Separate attempted publication from transport ACK and device completion.
|
||||
7. Add own-request latency/counters, late ACK classification, bounded buffers and graceful shutdown. No offline command outbox or automatic ambiguous retries.
|
||||
|
||||
### Dependencies and required inputs
|
||||
|
||||
Sprint 1, Sprint 2 server package deployed and verified, broker hostname/ports, private test backend/device credential files and public CA. No physical device is required.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- Observer receives valid telemetry/status; command mode receives correlated ping/version results from the simulator.
|
||||
- Logs show safe payload preview, topic, direction, QoS, request ID and native properties; no credentials leak.
|
||||
- Wrong-topic/device/correlation ACKs cannot complete a pending request; late/out-of-order ACKs do not corrupt state.
|
||||
- An intentional repeated request ID does not repeat simulated execution; conflicting reuse is rejected.
|
||||
- Missing ACK and disconnect produce truthful timeout/unknown outcomes. SIGINT/SIGTERM exits within the configured bound.
|
||||
- Session settings, queue/dedup limits and restart limitations are documented.
|
||||
|
||||
### Automated checks
|
||||
|
||||
gofmt, go vet, unit tests and race detector. Real-broker integration for initial exchange, correlation, timeout, duplicate/conflicting request and shutdown. Scope full fault matrix to Sprint 5.
|
||||
|
||||
### Manual verification
|
||||
|
||||
Run observer and simulator in separate terminals, then run command mode with a distinct client ID. Follow one request through publish_attempt, transport_ack, accepted and completed. Compare safe previews to actual test payloads.
|
||||
|
||||
### Risks and unresolved decisions
|
||||
|
||||
Autopaho callback/ACK timing and concurrent state ownership; inspect pinned behavior instead of assuming backpressure guarantees. Transport success is never execution proof.
|
||||
|
||||
### Mandatory end-of-sprint testing and completion gate
|
||||
|
||||
After implementing this sprint, run its automated checks and manual verification above. Inspect the final changes for correctness, scope, secret exposure, and compatibility with completed sprints. Fix failures and rerun affected checks before declaring completion.
|
||||
|
||||
Record the exact commands/checks, actual outcomes, verification environment, and each acceptance criterion as PASS, FAIL, or BLOCKED. Include applicable regression checks for previously completed behavior. A skipped/unavailable required check is BLOCKED, never PASS; do not claim full sprint completion while required checks fail or remain blocked. Do not run unrelated or production-project tests.
|
||||
|
||||
Report the results and remaining limitations to the user. Wait for authorization before the next sprint unless continued execution was explicitly authorized. Even with permission to continue, do not bypass this verification gate.
|
||||
|
||||
### Copy-ready implementation prompt
|
||||
|
||||
```text
|
||||
Implement Sprint 3 only in /home/ryan/Media/Code/Code/robot-market/test-mqtt-5 after authorization. Read applicable AGENTS.md, /home/ryan/.codex/RTK.md, README.md, docs/PLAN.md and docs/sprints/03-backend-simulator.md. Use rtk for shell commands. Do not edit siblings, production infrastructure, or physical firmware.
|
||||
|
||||
Using the existing configuration/contracts and isolated deployed test broker, implement a Go MQTT 5 observer/command CLI and device simulator with github.com/eclipse/paho.golang v0.23.0/autopaho, never paho.mqtt.golang. Support only ping/get_version plus telemetry/status. Use distinct predictable client IDs, TLS hostname verification, bounded workers/state, initial Clean Start=true, 300-second session expiry, subscription readiness/restoration, bounded jittered reconnect, retained online/offline status and explicit Will semantics. Commands are non-retained QoS 1 with a 15-second default deadline and MQTT Message Expiry. Correlate exact device/topic/request ID and native Response Topic/Correlation Data; inspect MQTT reason codes. Implement bounded request deduplication, cached ACK replay, conflict rejection, late ACKs, and graceful signal shutdown.
|
||||
|
||||
Make inputs/outputs visible in text/JSON slog events with sanitized bounded payload previews and MQTT properties. Distinguish publication attempt, transport acknowledgement, application acceptance and completion; expose counters and own-request latency. Do not introduce a database, offline outbox, HTTP viewer or dangerous commands. Run meaningful unit/race and real-broker happy-path/correlation/timeout/dedup/shutdown tests. Update quick start and contracts, report verified environment and limitations, then wait.
|
||||
|
||||
ESP32 reference: read docs/PLAN.md section 6.1 for the supplied Esp32MqttConfig. It is connection configuration, not a payload schema or a blocker for Go implementation. Preserve the proposed Go test contracts. Device-specific MQTT 5 selection depends on the actual C library/build/init; inspect TLS/CA handling, role-specific credentials and single publish_topic limitations during device integration. Do not modify existing C projects. Separate MQTT 3.1.1 payload interoperability from native MQTT 5 feature evidence.
|
||||
|
||||
Mandatory completion gate: after implementing this sprint, run its listed automated and manual checks plus relevant regression checks, inspect the final changes for correctness/scope/secrets/compatibility, fix failures and rerun affected checks. Report exact checks, real outcomes, environment, and every acceptance criterion as PASS/FAIL/BLOCKED. Missing required checks are BLOCKED, never a pass. Do not mark the sprint complete with unresolved failures/blockers or bypass verification when continuing to another sprint.
|
||||
```
|
||||
|
||||
## Completion evidence
|
||||
|
||||
Implemented only in the isolated test project; existing projects and server configuration were not changed. Added pinned Paho v0.23.0/autopaho, shared bounded transport, observer and safe command CLI, simulator, event history/counters and [RUNNING.md](../RUNNING.md). Plaintext requires explicit opt-in; TLS verifies CA and hostname. No browser, database, offline application outbox or physical-device implementation was added.
|
||||
|
||||
| Acceptance criterion | Status | Evidence |
|
||||
| --- | --- | --- |
|
||||
| Observer telemetry/status and correlated ping/version | PASS | Remote race tests on TLS 8884 and plaintext 1884; real CLI observer/simulator/command check |
|
||||
| Safe input/output logs, properties, lifecycle and counters | PASS | CLI JSON logs contain previews, native correlation and publish/transport/accepted/completed stages; copied credential passwords absent; bounded history tests |
|
||||
| Wrong topic/device/correlation, late/out-of-order ACK handling | PASS | Unit tests reject wrong identity/topic/correlation/retention/response/result/deadline; completion before acceptance remains terminal; remote wrong-correlation and late-ACK tests |
|
||||
| Repeated request ID and conflicting reuse | PASS | Unit and remote tests replay cached completion with one execution; changed command with same ID is rejected |
|
||||
| Timeout, disconnect unknown outcome and bounded shutdown | PASS | Remote deadline/late-ACK tests; abrupt relay connection loss returns unknown; CLI SIGTERM and simulator cancellation complete within configured 5-second bound |
|
||||
| Session/queue/dedup/restart settings documented | PASS | RUNNING.md records limits, overflow, in-memory state and no automatic application retry; real reconnect tests cover Session Present=true and false with subscription restoration |
|
||||
|
||||
Checks run from the new project with Go 1.27.1 and temporary module/build caches:
|
||||
|
||||
- `gofmt -l cmd internal`: no unformatted files.
|
||||
- `go vet ./...`: PASS.
|
||||
- `go test -race ./...`: PASS; default remote tests explicitly skip without the opt-in variable, separately verified below.
|
||||
- `go mod verify`: all modules verified.
|
||||
- `go build -buildvcs=false` for both CLI binaries: PASS.
|
||||
- `MQTT5_INTEGRATION_CREDENTIAL_DIR=/home/ryan/mqtt-test-credentials go test -race -v -count=1 -timeout=90s ./internal/integration`: PASS against 109.122.252.129. Covers TLS/plaintext happy paths, native properties, dedup/conflict, deadline/late ACK, local disconnect and actual TLS connection loss/reconnect with retained and lost broker sessions. The relay touches only its own loopback client connection; no broker restart/configuration change.
|
||||
- Real built CLI processes: observer/simulator/command ping; JSON previews/native properties/lifecycle; no credential output; successful bounded SIGTERM shutdown: PASS. Temporary check script/output lived under /tmp; the reproducible manual commands are in RUNNING.md.
|
||||
- Structural source review with graphify AST extraction: 20 Go files, 204 nodes and 660 extracted relationships; source and concurrency review completed without adding graph artifacts to the project.
|
||||
|
||||
Verified environment: Go MQTT 5 backend and local simulator against the dedicated remote Mosquitto 2.1.2 broker, both TLS and plaintext. Larger fault/compatibility matrix, crash durability, durable client/dedup recovery, browser viewing and physical ESP32 interoperability remain unverified or scheduled for later sprints. All Sprint 3 acceptance criteria passed; Sprint 4 requires authorization.
|
||||
@@ -0,0 +1,81 @@
|
||||
# Sprint 4 — Small Go-served browser viewer
|
||||
|
||||
Status: COMPLETE for local implementation and verification; ESP32 hardware evidence remains outstanding. Read [the plan](../PLAN.md), [sprint index and shared rules](../SPRINTS.md), and [README](../../README.md).
|
||||
|
||||
|
||||
### Goal and scope
|
||||
|
||||
Inspect the same safe message stream through a browser without adding another service or frontend toolchain.
|
||||
|
||||
### Deliverables and likely files
|
||||
|
||||
- `internal/web/` server, embedded HTML/CSS/vanilla JavaScript and tests.
|
||||
- `internal/events/` bounded snapshot/SSE fan-out if not already present.
|
||||
- Viewer configuration and README browser instructions.
|
||||
|
||||
### Tasks
|
||||
|
||||
1. Add opt-in viewer on `127.0.0.1:8080` in observation mode; default disabled.
|
||||
2. Provide read-only snapshot and SSE endpoints using the sanitized event model. Add event IDs, bounded replay, Last-Event-ID handling, gap/reset indicators, heartbeat and cleanup.
|
||||
3. Show connection/readiness, counters, filterable direction/topic/device/request timeline, bounded payload/property details and command-stage observations.
|
||||
4. Show compatibility rows with explicit not_tested/pass/fail/unsupported/blocked evidence. Do not infer native MQTT 5 support from a message's mere arrival.
|
||||
5. Render payloads as text, apply restrictive CSP/Host/Origin handling, use no CORS wildcard/CDN, and enforce viewer/queue limits.
|
||||
6. Ensure slow/disconnected viewers cannot block MQTT or grow memory; integrate bounded HTTP shutdown. No HTTP command-publishing endpoint.
|
||||
|
||||
### Dependencies and required inputs
|
||||
|
||||
Sprint 3 running locally; browser availability. No design assets, JavaScript package installation, server access or hardware required.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- Opening the loopback page shows live simulator input/output and state changes matching the logs.
|
||||
- Filters, pause/resume, reconnect, expired replay buffer and empty/disconnected states behave clearly.
|
||||
- Hostile payload strings render literally; secrets remain redacted; large previews show truncation.
|
||||
- Multiple/slow viewers obey bounds and cannot affect command completion.
|
||||
- Compatibility rows distinguish evidence from unknown status; observed ACK timing is not mislabeled as sender-owned latency.
|
||||
|
||||
### Automated checks
|
||||
|
||||
Go HTTP tests for snapshot/SSE/replay/gap, redaction, origin/host validation, cleanup and shutdown; race detector for fan-out. Use browser automation if available; no heavyweight frontend test stack solely for this page.
|
||||
|
||||
### Manual verification
|
||||
|
||||
Open the viewer while simulator traffic flows; run ping from CLI; inspect payload/properties and lifecycle. Disconnect/reopen the page, exercise filters and pause, send a harmless HTML-like test string, and verify literal rendering.
|
||||
|
||||
### Completion evidence
|
||||
|
||||
- `gofmt -w internal/web/server.go internal/web/server_test.go internal/events/store.go internal/events/store_test.go internal/cli/cli.go internal/config/config.go`: PASS.
|
||||
- `node --check internal/web/assets/app.js`: PASS after fixing the page script parse error discovered during browser review.
|
||||
- `go test ./internal/web ./internal/events`: PASS, including snapshot escaping, host/origin policy, SSE replay gaps, slow queue behavior and viewer cleanup/limit. Tests use handler concurrency because this sandbox denies test-process localhost listeners.
|
||||
- Regression checks: `go test ./...`, `go test -race ./...`, `go vet ./...`, `go mod verify`, and `gofmt -l cmd internal`: PASS.
|
||||
- Local browser verification against the deployed isolated broker: PASS. Observer connected with MQTT 5; simulator retained status and periodic telemetry appeared; the compatibility panel showed backend MQTT 5 passed, JSON contract passed, and native ESP32 round trip not tested. A harmless ping displayed command payload, response topic, correlation data, accepted/completed ACKs, and telemetry. The command CLI reported `pong:true`. Topic filter, pause/resume and clear filters worked.
|
||||
- One initial browser pass exposed a JavaScript parse error; it was fixed and the page was reloaded successfully. The simulator was stopped after verification.
|
||||
- Security boundary: listener binds only to validated `127.0.0.1`; snapshots/SSE are read-only, browser strings render through `textContent`, with same-host/origin checks, CSP and bounded event/viewer queues. There is no HTTP publish route.
|
||||
- Acceptance outcomes: live simulator input/state/counters PASS; filters/pause/resume and browser reload/reconnect PASS; bounded replay/gap and viewer limits PASS by automated tests; hostile string safety PASS by JSON escaping and text-only-sink regression test (an arbitrary HTML-like MQTT message was not sent because the contract rejects free-form fields and the viewer intentionally has no publish endpoint); command stages/observer latency labeling and compatibility evidence PASS.
|
||||
- Not claimed: ESP32 hardware/MQTT-version compatibility, native MQTT 5 property support on the ESP32, or long-running browser stress testing. Continue in Sprint 5/7 as planned.
|
||||
|
||||
### Risks and unresolved decisions
|
||||
|
||||
Long-lived SSE connections and event gaps; handle them explicitly. Public access/authentication and browser command submission remain outside scope.
|
||||
|
||||
### Mandatory end-of-sprint testing and completion gate
|
||||
|
||||
After implementing this sprint, run its automated checks and manual verification above. Inspect the final changes for correctness, scope, secret exposure, and compatibility with completed sprints. Fix failures and rerun affected checks before declaring completion.
|
||||
|
||||
Record the exact commands/checks, actual outcomes, verification environment, and each acceptance criterion as PASS, FAIL, or BLOCKED. Include applicable regression checks for previously completed behavior. A skipped/unavailable required check is BLOCKED, never PASS; do not claim full sprint completion while required checks fail or remain blocked. Do not run unrelated or production-project tests.
|
||||
|
||||
Report the results and remaining limitations to the user. Wait for authorization before the next sprint unless continued execution was explicitly authorized. Even with permission to continue, do not bypass this verification gate.
|
||||
|
||||
### Copy-ready implementation prompt
|
||||
|
||||
```text
|
||||
Implement Sprint 4 only in /home/ryan/Media/Code/Code/robot-market/test-mqtt-5 after authorization. Read applicable AGENTS.md, /home/ryan/.codex/RTK.md, README.md, docs/PLAN.md and docs/sprints/04-browser-viewer.md. Prefix shell commands with rtk. Existing projects are read-only; do not deploy remotely.
|
||||
|
||||
Add a minimal optional browser viewer to the working Go backend using net/http, embed, HTML/CSS and vanilla JavaScript. No Node/npm, CDN, framework or separate service. Bind only 127.0.0.1:8080, default disabled. Display the same sanitized input/output events as logs, including topic/direction/device/request filters, bounded payload and MQTT property details, connections/readiness, counters and observed command stages. Provide read-only snapshot plus SSE with event IDs, bounded replay, explicit gaps, viewer limits and slow-client cleanup. Display compatibility statuses and evidence without assuming native support from ordinary traffic. Keep locally measured own-request latency distinct from observer timing.
|
||||
|
||||
Use textContent for untrusted payloads, restrictive CSP and same-origin/Host checks. No HTTP publishing endpoint or public exposure. Verify real simulator traffic in the browser and meaningful Go HTTP/race tests for event replay, cleanup, redaction and shutdown. Update README, report actual browser verification and limitations, then wait.
|
||||
|
||||
ESP32 reference: read docs/PLAN.md section 6.1 for the supplied Esp32MqttConfig. It is connection configuration, not a payload schema or a blocker for Go implementation. Preserve the proposed Go test contracts. Device-specific MQTT 5 selection depends on the actual C library/build/init; inspect TLS/CA handling, role-specific credentials and single publish_topic limitations during device integration. Do not modify existing C projects. Separate MQTT 3.1.1 payload interoperability from native MQTT 5 feature evidence.
|
||||
|
||||
Mandatory completion gate: after implementing this sprint, run its listed automated and manual checks plus relevant regression checks, inspect the final changes for correctness/scope/secrets/compatibility, fix failures and rerun affected checks. Report exact checks, real outcomes, environment, and every acceptance criterion as PASS/FAIL/BLOCKED. Missing required checks are BLOCKED, never a pass. Do not mark the sprint complete with unresolved failures/blockers or bypass verification when continuing to another sprint.
|
||||
```
|
||||
@@ -0,0 +1,81 @@
|
||||
# Sprint 5 — Reliability scenarios and compatibility evidence
|
||||
|
||||
Status: implemented, with live ACK and reliability checks PASS; full completion BLOCKED by the server container-exec failure preventing the broker restart verifier. See [verification evidence](../VERIFICATION.md). Read [the plan](../PLAN.md), [sprint index and shared rules](../SPRINTS.md), and [README](../../README.md).
|
||||
|
||||
|
||||
### Goal and scope
|
||||
|
||||
Turn the working local environment into a repeatable test harness with explicit, evidence-backed feature results.
|
||||
|
||||
### Deliverables and likely files
|
||||
|
||||
- `tests/integration/` or equivalent opt-in Go integration suite; minimal scenario helpers.
|
||||
- Simulator controls for delayed/missing ACKs and controlled harmless duplicate tests.
|
||||
- `docs/VERIFICATION.md` scenario checklist and known limitations; event/result output consumable by logs and current-process viewer.
|
||||
|
||||
### Tasks
|
||||
|
||||
1. Implement explicit scenario result records with feature, environment, endpoint/version, status, observation and time; unknown remains not_tested.
|
||||
2. Exercise broker graceful restart, network interruption, session resume/loss/expiry and subscription restoration.
|
||||
3. Test queued command expiry separately from device-side expiry after delivery; synchronize clocks and use bounded test deadlines.
|
||||
4. Test duplicate requests, conflicting IDs, wrong/late/reordered/missing ACKs, invalid payloads and overloaded buffers.
|
||||
5. Verify Will after abrupt client stop versus graceful offline status, and stale-status handling.
|
||||
6. Test auth/TLS failures and unrestricted cross-device topic access (the user explicitly disabled ACLs for this test broker). Test and native Response Topic/Correlation Data echo. Attempt actual QoS 1 retransmission with controlled interruption; label unreproduced behavior honestly.
|
||||
7. Surface structured scenario verdicts in logs and viewer without adding persistent accounting. Document why repeated application publication is different from wire-level DUP retransmission.
|
||||
|
||||
### Added scope: backend receipt ACKs (2026-10-07)
|
||||
|
||||
Implement device publication → backend receipt → device application ACK, separate from broker PUBACK and device replies to commands. Raw observation continues to accept arbitrary payloads without schema, timestamp or device-ID checks.
|
||||
|
||||
- Enable with `receipts.enabled=true` in raw observe mode; Docker enables it.
|
||||
- Subscribe on the device to `<namespace>/<topic-device>/acks` before publishing. Route by the incoming topic, not JSON or username. A valid in-namespace MQTT 5 Response Topic overrides the default, with exact Correlation Data echoed.
|
||||
- Reply at QoS 1, retain false: `type=receipt_ack`, `ok=true`, original `source_topic`, observed time, payload byte length and SHA-256; echo optional message_id/request_id without requiring them.
|
||||
- Never reply to commands, ACK/receipt topics, receipt_ack payloads or retained replays. Repeated live publications may each get a receipt ACK; receipt confirmation does not execute a device action.
|
||||
- Keep reply queue bounded and log receipt sent/failed/dropped. Broker acknowledgement of the reply does not prove the device consumed it. Do not claim persistence or exactly-once handling.
|
||||
- Test raw text, empty/invalid JSON, optional IDs, mismatched JSON identity, unchanged timestamp placeholder, native reply routing/correlation, duplicate delivery, reply failure, queue overload, cancellation and ACK-loop prevention. Run live independent subscriber/publisher checks over MQTT 3.1.1 and MQTT 5 with both plaintext/TLS, then inspect UI inbound/outbound events.
|
||||
|
||||
### Dependencies and required inputs
|
||||
|
||||
Sprints 1–4; local broker control and a safe way to interrupt only test connections/containers. Do not change host-wide firewall or unrelated networking.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- Scenarios are bounded, repeatable, isolated, and leave no unrelated processes/network changes.
|
||||
- Resume and session-loss paths both recover subscriptions and readiness correctly.
|
||||
- Expired commands are not executed; repeated valid IDs execute once within the documented in-memory retention window.
|
||||
- Every compatibility verdict contains evidence/environment; simulator results are not marked physical-device results.
|
||||
- Logs/browser explain failures, gaps, timeouts, late responses and limits without secrets.
|
||||
|
||||
### Automated checks
|
||||
|
||||
Unit/race suite and opt-in real-broker scenarios with explicit pass/fail/skip outcomes and cleanup. Avoid timing-only assertions where an observable state can synchronize the test.
|
||||
|
||||
### Manual verification
|
||||
|
||||
Follow the checklist with the viewer open; observe reconnect, expiry, Will and late ACK events. Review the resulting feature table and distinguish unsupported, blocked and failed.
|
||||
|
||||
### Risks and unresolved decisions
|
||||
|
||||
QoS 1 wire retransmission may be nondeterministic; report not reproduced rather than substituting an application duplicate. Broker graceful persistence is not crash-durability proof.
|
||||
|
||||
### Mandatory end-of-sprint testing and completion gate
|
||||
|
||||
After implementing this sprint, run its automated checks and manual verification above. Inspect the final changes for correctness, scope, secret exposure, and compatibility with completed sprints. Fix failures and rerun affected checks before declaring completion.
|
||||
|
||||
Record the exact commands/checks, actual outcomes, verification environment, and each acceptance criterion as PASS, FAIL, or BLOCKED. Include applicable regression checks for previously completed behavior. A skipped/unavailable required check is BLOCKED, never PASS; do not claim full sprint completion while required checks fail or remain blocked. Do not run unrelated or production-project tests.
|
||||
|
||||
Report the results and remaining limitations to the user. Wait for authorization before the next sprint unless continued execution was explicitly authorized. Even with permission to continue, do not bypass this verification gate.
|
||||
|
||||
### Copy-ready implementation prompt
|
||||
|
||||
```text
|
||||
Implement Sprint 5 only in /home/ryan/Media/Code/Code/robot-market/test-mqtt-5 after authorization. Read applicable AGENTS.md, /home/ryan/.codex/RTK.md, README.md, docs/PLAN.md and docs/sprints/05-reliability.md. Use rtk. Never alter production brokers, siblings, or host-wide networking.
|
||||
|
||||
Build a focused opt-in real-broker reliability/compatibility harness around the existing Go backend, simulator and local viewer. Cover graceful broker restart, isolated network interruption, session resume versus loss/expiry, subscription restoration, broker-queued Message Expiry, device application deadlines, duplicates/conflicting request IDs, wrong/late/reordered/missing ACKs, Will versus graceful offline state, validation/overload and TLS/auth failures and unrestricted authenticated topic access. Prove native Response Topic/Correlation Data behavior with actual properties. Attempt controlled QoS 1 retransmission separately from deterministic repeated application requests and report unreproduced cases honestly.
|
||||
|
||||
Emit explicit feature/environment/evidence/status results to logs and the existing viewer, leaving untested features not_tested. Keep all state and test operations bounded, provide cleanup, and avoid a database or monitoring stack. Run meaningful unit/race and isolated integration checks. Update docs/VERIFICATION.md and known limitations; distinguish local simulator, remote and hardware evidence. Report results and wait.
|
||||
|
||||
ESP32 reference: read docs/PLAN.md section 6.1 for the supplied Esp32MqttConfig. It is connection configuration, not a payload schema or a blocker for Go implementation. Preserve the proposed Go test contracts. Device-specific MQTT 5 selection depends on the actual C library/build/init; inspect TLS/CA handling, role-specific credentials and single publish_topic limitations during device integration. Do not modify existing C projects. Separate MQTT 3.1.1 payload interoperability from native MQTT 5 feature evidence.
|
||||
|
||||
Mandatory completion gate: after implementing this sprint, run its listed automated and manual checks plus relevant regression checks, inspect the final changes for correctness/scope/secrets/compatibility, fix failures and rerun affected checks. Report exact checks, real outcomes, environment, and every acceptance criterion as PASS/FAIL/BLOCKED. Missing required checks are BLOCKED, never a pass. Do not mark the sprint complete with unresolved failures/blockers or bypass verification when continuing to another sprint.
|
||||
```
|
||||
@@ -0,0 +1,68 @@
|
||||
# Sprint 6 — Remote test-broker deployment and connectivity
|
||||
|
||||
Status: proposed; not implemented. Read [the plan](../PLAN.md), [sprint index and shared rules](../SPRINTS.md), and [README](../../README.md).
|
||||
|
||||
|
||||
### Goal and scope
|
||||
|
||||
Deploy the locally validated broker as a separate remote test instance and verify outbound-only local clients against it.
|
||||
|
||||
### Deliverables and likely files
|
||||
|
||||
- Environment-specific deployment examples with placeholders and final pinned digest.
|
||||
- `docs/DEPLOYMENT.md` complete remote certificate/firewall/operations runbook.
|
||||
- Remote entries in `docs/VERIFICATION.md`; no secrets committed.
|
||||
|
||||
### Tasks
|
||||
|
||||
1. Obtain OS/CPU, Docker/Compose support, available resources/port, DNS/SANs, access method and certificate renewal constraints without requesting secret values in chat.
|
||||
2. Inspect current listeners/services read-only; choose an isolated instance/port and validate artifacts locally before remote mutation.
|
||||
3. Prepare test-only credentials/ACLs, certificate provisioning/renewal, data/config mounts, health check, ownership, bounded logs and firewall requirements.
|
||||
4. Present exact remote operations and obtain any required authorization. Then deploy only the authorized test instance.
|
||||
5. Verify hostname/certificate/auth/ACL behavior from the development computer, simulator exchange, restart/session recovery and health.
|
||||
6. Document backup/update/rollback with compatible persistence snapshot and renewal verification. Browser viewer remains local.
|
||||
|
||||
### Dependencies and required inputs
|
||||
|
||||
Sprint 5 local verification; remote details/access and explicit authorization for remote changes. If unavailable, finish locally reviewable artifacts and report remote tests blocked.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- Separate remote TLS instance works without changing production listeners/configuration or requiring inbound development-computer access.
|
||||
- Local backend/simulator exchange messages and show them in logs/viewer through outbound connections.
|
||||
- Remote security and restart checks have actual evidence; runbook is complete.
|
||||
- If access is unavailable, the sprint is explicitly incomplete for deployment, not marked verified.
|
||||
|
||||
### Automated checks
|
||||
|
||||
Local artifact validation before deployment; remote health and focused MQTT integration scenarios after authorization. Repeat checks only where remote environment changes behavior.
|
||||
|
||||
### Manual verification
|
||||
|
||||
Inspect remote listening ports, certificate chain/SAN, log bounds, access controls and persistence. Walk through documented update/rollback steps using the test instance where authorized.
|
||||
|
||||
### Risks and unresolved decisions
|
||||
|
||||
Unknown remote environment, existing port conflicts, certificate renewal method and network restrictions. Native deployment is an explicit alternative if Compose is unsuitable.
|
||||
|
||||
### Mandatory end-of-sprint testing and completion gate
|
||||
|
||||
After implementing this sprint, run its automated checks and manual verification above. Inspect the final changes for correctness, scope, secret exposure, and compatibility with completed sprints. Fix failures and rerun affected checks before declaring completion.
|
||||
|
||||
Record the exact commands/checks, actual outcomes, verification environment, and each acceptance criterion as PASS, FAIL, or BLOCKED. Include applicable regression checks for previously completed behavior. A skipped/unavailable required check is BLOCKED, never PASS; do not claim full sprint completion while required checks fail or remain blocked. Do not run unrelated or production-project tests.
|
||||
|
||||
Report the results and remaining limitations to the user. Wait for authorization before the next sprint unless continued execution was explicitly authorized. Even with permission to continue, do not bypass this verification gate.
|
||||
|
||||
### Copy-ready implementation prompt
|
||||
|
||||
```text
|
||||
Implement Sprint 6 only in /home/ryan/Media/Code/Code/robot-market/test-mqtt-5 after authorization. Read applicable AGENTS.md, /home/ryan/.codex/RTK.md, README.md, docs/PLAN.md and docs/sprints/06-remote-broker.md. Use rtk. Existing projects, production listeners and credentials must remain untouched.
|
||||
|
||||
First obtain/inspect remote OS, CPU, Docker/Compose support, free test port, DNS/certificate constraints and access method without exposing secrets. Prepare and locally validate the dedicated TLS Mosquitto deployment from prior sprints, pinned by compatible digest, with separate test credentials, per-device ACLs, persistence, restrictive secret permissions, bounded logs and authenticated health check. Complete certificate issuance/renewal, firewall, backup/update/rollback procedures. Present concrete remote operations before requesting any still-required authorization. Do not treat documentation authorization as deployment authorization.
|
||||
|
||||
Only after required authorization, deploy the isolated remote test instance and verify outbound-only local backend/simulator connectivity, hostname validation, bad credentials/ACL behavior, restart and health. Keep the browser viewer on local loopback. If access/details are missing, preserve complete local artifacts and report remote work blocked/unverified. Update deployment/verification docs, report evidence by environment and wait.
|
||||
|
||||
ESP32 reference: read docs/PLAN.md section 6.1 for the supplied Esp32MqttConfig. It is connection configuration, not a payload schema or a blocker for Go implementation. Preserve the proposed Go test contracts. Device-specific MQTT 5 selection depends on the actual C library/build/init; inspect TLS/CA handling, role-specific credentials and single publish_topic limitations during device integration. Do not modify existing C projects. Separate MQTT 3.1.1 payload interoperability from native MQTT 5 feature evidence.
|
||||
|
||||
Mandatory completion gate: after implementing this sprint, run its listed automated and manual checks plus relevant regression checks, inspect the final changes for correctness/scope/secrets/compatibility, fix failures and rerun affected checks. Report exact checks, real outcomes, environment, and every acceptance criterion as PASS/FAIL/BLOCKED. Missing required checks are BLOCKED, never a pass. Do not mark the sprint complete with unresolved failures/blockers or bypass verification when continuing to another sprint.
|
||||
```
|
||||
@@ -0,0 +1,71 @@
|
||||
# Sprint 7 — Conditional real-device compatibility
|
||||
|
||||
Status: proposed; not implemented. Read [the plan](../PLAN.md), [sprint index and shared rules](../SPRINTS.md), and [README](../../README.md).
|
||||
|
||||
|
||||
### Goal and scope
|
||||
|
||||
Determine and verify actual device behavior without modifying existing device projects or operating real machinery.
|
||||
|
||||
### Deliverables and likely files
|
||||
|
||||
- `docs/DEVICE-SETUP.md` with verified framework/client/protocol and test configuration.
|
||||
- Device-specific compatibility results in `docs/VERIFICATION.md`.
|
||||
- Only if separately approved: a small independent test firmware/application under `device/` inside this project.
|
||||
|
||||
### Tasks
|
||||
|
||||
- Inspect how Esp32MqttConfig is consumed: host versus URI, ca_cert PEM versus path, TLS hostname checks, selected protocol and library version, and whether publish calls can target telemetry/status/ACK topics separately. Propose only necessary device-side settings; this struct is not a payload contract.
|
||||
|
||||
1. Inspect supplied Linux device and/or ESP32 source read-only; identify framework, exact client version, configured MQTT protocol, TLS/session handling, identities, payloads and command paths.
|
||||
2. Determine direct ESP32-to-broker versus gateway architecture and select one initial endpoint with the user.
|
||||
3. Map supported existing behavior to the test contract. Identify required source changes explicitly; do not modify existing projects or silently add a compatibility bridge.
|
||||
4. Use a dedicated safe test instance/board. If required, propose an isolated test application in this project and obtain authorization before flashing hardware or writing that new device implementation.
|
||||
5. Configure test credentials/CA/clock/topic identity; verify actual protocol and native-property evidence, telemetry/status, harmless command lifecycle, reconnect, expiry and duplicate handling where supported.
|
||||
6. Update comparison results and limitations. Leave unavailable features unsupported/not_tested/blocked with reasons.
|
||||
|
||||
### Dependencies and required inputs
|
||||
|
||||
Sprint 5 minimum; Sprint 6 if using the remote broker. Actual device repositories, board model, hardware/access, framework/toolchain and explicit authorization for flashing/test-device changes. Hardware unavailable means this sprint remains blocked; prior simulator work remains useful.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- “Device backend” is resolved using actual source and deployment evidence.
|
||||
- Exact tested SDK/client/protocol and environment are recorded.
|
||||
- Only harmless commands run; production/device repositories remain unchanged.
|
||||
- Native MQTT 5 features are marked passed only with actual end-to-end evidence. JSON-only interoperability is labeled separately.
|
||||
- No simulator result is substituted for hardware verification.
|
||||
|
||||
### Automated checks
|
||||
|
||||
Approved isolated device builds/contract checks and backend integration tests appropriate to the identified stack. Do not install arbitrary SDKs before choosing the actual target.
|
||||
|
||||
### Manual verification
|
||||
|
||||
Observe device traffic in backend logs/browser; issue ping/get_version; exercise approved device reconnect/restart and record actual outcomes. Inspect test-broker/client evidence of protocol selection.
|
||||
|
||||
### Risks and unresolved decisions
|
||||
|
||||
Hardware availability, flashing permissions, clock/certificate resources, SDK support and unknown device contract. Existing firmware may require changes outside authorized scope; stop at a concrete change proposal rather than modifying it.
|
||||
|
||||
### Mandatory end-of-sprint testing and completion gate
|
||||
|
||||
After implementing this sprint, run its automated checks and manual verification above. Inspect the final changes for correctness, scope, secret exposure, and compatibility with completed sprints. Fix failures and rerun affected checks before declaring completion.
|
||||
|
||||
Record the exact commands/checks, actual outcomes, verification environment, and each acceptance criterion as PASS, FAIL, or BLOCKED. Include applicable regression checks for previously completed behavior. A skipped/unavailable required check is BLOCKED, never PASS; do not claim full sprint completion while required checks fail or remain blocked. Do not run unrelated or production-project tests.
|
||||
|
||||
Report the results and remaining limitations to the user. Wait for authorization before the next sprint unless continued execution was explicitly authorized. Even with permission to continue, do not bypass this verification gate.
|
||||
|
||||
### Copy-ready implementation prompt
|
||||
|
||||
```text
|
||||
Perform Sprint 7 only after explicit authorization, using /home/ryan/Media/Code/Code/robot-market/test-mqtt-5 as the only writable project. Read applicable AGENTS.md, /home/ryan/.codex/RTK.md, README.md, docs/PLAN.md and docs/sprints/07-device-compatibility.md. Prefix shell commands with rtk. Existing device/server projects are read-only. No vending, motor, payment, credentials-change or OTA commands.
|
||||
|
||||
Obtain the actual Linux device/ESP32 repositories and board/access details. Inspect framework, exact MQTT client/version, configured protocol, TLS/session/topic/identity/correlation behavior. Resolve whether the target is direct ESP32 MQTT or a gateway and identify one initial test endpoint. Map compatibility to the isolated test contract; do not infer MQTT 5 support from broker capability or message delivery. If device changes are needed, present the concrete changes and prefer a separately authorized independent test application inside this project. Do not modify existing firmware or flash hardware without authorization.
|
||||
|
||||
On authorized dedicated test hardware, verify TLS/time, telemetry/status, ping/get_version ACK stages, native Response Topic/Correlation Data, reconnect/session/expiry/dedup behavior where supported. Inspect real connection/property evidence. Document exact versions, setup, results and limits in docs/DEVICE-SETUP.md and docs/VERIFICATION.md. If hardware/source/access is missing, report blocked/not_tested rather than substituting simulator evidence. Report acceptance status and stop.
|
||||
|
||||
ESP32 reference: read docs/PLAN.md section 6.1 for the supplied Esp32MqttConfig. It is connection configuration, not a payload schema or a blocker for Go implementation. Preserve the proposed Go test contracts. Device-specific MQTT 5 selection depends on the actual C library/build/init; inspect TLS/CA handling, role-specific credentials and single publish_topic limitations during device integration. Do not modify existing C projects. Separate MQTT 3.1.1 payload interoperability from native MQTT 5 feature evidence.
|
||||
|
||||
Mandatory completion gate: after implementing this sprint, run its listed automated and manual checks plus relevant regression checks, inspect the final changes for correctness/scope/secrets/compatibility, fix failures and rerun affected checks. Report exact checks, real outcomes, environment, and every acceptance criterion as PASS/FAIL/BLOCKED. Missing required checks are BLOCKED, never a pass. Do not mark the sprint complete with unresolved failures/blockers or bypass verification when continuing to another sprint.
|
||||
```
|
||||
Reference in New Issue
Block a user