mqtt-5-test/docs/VERIFICATION.md

# Sprint 5 verification

Checked 2026-10-07. Scope: isolated `test-mqtt-5` only. Go race tests on local Linux; Docker Go observer at http://127.0.0.1:8080; remote Mosquitto test broker 109.122.252.129, TLS 8884 and plaintext 1884. Independent Python socket clients use the ESP32 test account; Go tests use private backend/simulated-device handoffs. These checks do not prove physical ESP32 behavior.

Status: implemented with full sprint completion **BLOCKED** by broker restart verification. Receipt ACK functionality is implemented and verified live.

## Repeatable checks

Run from the test project. `rtk` is a local command convention; use ordinary Linux commands on the server.

```sh
rtk env GOCACHE=/tmp/mqtt5-go-cache go test -race ./...
rtk env GOCACHE=/tmp/mqtt5-go-cache go vet ./...
rtk env GOCACHE=/tmp/mqtt5-go-cache MQTT5_INTEGRATION_CREDENTIAL_DIR=/home/ryan/mqtt-test-credentials go test -race -v ./internal/integration
rtk docker compose up -d --build observer
rtk python3 scripts/check_receipts.py --credentials esp32-test-kit/connection.txt --ca esp32-test-kit/ca.crt
rtk docker compose ps -a
```

The normal unit suite skips remote tests without the opt-in credential directory; that skip is not remote evidence. The separately executed opt-in suite passed all remote tests. Tests create temporary simulated clients and clean them up; the Docker simulator stays stopped.

## Results and limits

| Feature | Outcome | Evidence and limit |
| --- | --- | --- |
| Unit/race regression and vet | PASS | All packages pass; includes ACK routing, arbitrary/empty payloads, binary correlation copying, duplicate receipt delivery, bounded queue overflow, publish failure, worker shutdown, correlation ordering and viewer tests. |
| Device receipt ACKs | PASS | Independent Python subscribers/publishers: MQTT 3.1.1 and 5 × plaintext and TLS; text, arbitrary JSON, empty payloads. Byte count and SHA-256 match original bytes; optional message ID echoed. Incoming message and outgoing receipt confirmed through viewer snapshot. |
| Native receipt routing | PASS | Actual MQTT 5 Response Topic and binary Correlation Data echoed exactly. |
| Loop prevention | PASS | ACK/command/receipt topics and receipt payloads suppressed; retained replay suppressed. Unit tests and bounded live no-extra-reply check. |
| Session resume/loss/expiry | PASS | Real TLS connection interrupted through loopback relay; session-present flag checked; subscriptions restored and fresh commands complete. Pending execution remains unknown after connection loss. |
| Broker-queued expiry | PASS | Persistent subscriber offline; Message Expiry=1 second with still-valid 15-second application deadline; expired packet absent, fresh control delivered after session resume. |
| Device application deadlines | PASS | Existing simulator unit tests prevent expired command execution; remote missing/late/wrong-correlation checks produce timeout/late/unknown rather than success. |
| Harmless commands and duplicates | PASS | Native correlated ping/version over TLS/plaintext; cached application duplicate executes once; conflicting ID rejected. This applies to in-memory simulator command execution, not ESP32 persistence. |
| Wire QoS 1 retransmission | PASS | Independent client sends QoS 1, deliberately does not read PUBACK, drops socket, resumes persistent session, resends same packet ID/payload with DUP=1. Subscriber gets two receipts; this is controlled wire retransmission, not proof of automatic hardware reconnect behavior. |
| Abrupt Will | PASS | Real abrupt TLS loss emits nonretained offline Will; creation timestamp retained, no invented disconnect timestamp. |
| Graceful offline/status and stale display | PASS / limited | Existing simulator graceful status behavior and unit tests; raw viewer labels observed time and does not derive current health from retained payloads. No general device liveness algorithm is claimed. |
| Scenario results | PASS | Opt-in tests publish feature/environment/endpoint/version/status/observation/time; observer logs and viewer expose reported evidence separately from untested ESP32 compatibility. Results are bounded and transient. |
| Authentication/TLS negatives | PASS | Independent external checks reject anonymous CONNECT, invalid password, untrusted CA and wrong hostname; successful TLS/plain sessions provide positive controls. |
| Unrestricted test topic access | PASS | ESP32 account publishes generated device topics and subscribes to their ACKs. ACL enforcement intentionally removed at user's request. |
| Graceful broker restart and persistence | BLOCKED | Server `docker compose exec -T broker true` fails with `OCI runtime exec failed ... procReady not received`. Server `python3 scripts/verify.py --insecure --restart` fails on its initial container health exchange, before restart. Existing broker accepts external MQTT traffic. Resolve server Docker runtime/container-exec issue, then rerun verifier; do not treat prior Sprint 2 evidence as a fresh Sprint 5 pass. |
| Physical ESP32 receipt consumption | Not tested | Developer must subscribe before publish, receive receipt_ack and confirm in firmware logs. User-reported inbound hardware messages are not independent proof of return-path consumption. |

Final browser refresh verified 21 receipt ACKs, zero reply failures/drops, and the reported Will scenario marked passed while ESP32 native compatibility remained not_tested. Final Docker status: observer running, simulator exited. The regenerated ESP32 ZIP includes the updated wildcard subscription and receipt instructions.

## Acceptance gate

- Bounded isolated scenarios and cleanup: PASS for executed scenarios.
- Resume/loss subscription recovery: PASS.
- Expired queued commands not delivered; simulator repeated valid ID executes once: PASS, bounded in-memory behavior only.
- Evidence/environment attached and no simulator results presented as hardware: PASS.
- Logs/viewer expose receipt failures, queue drops, timeout/unknown/late events and evidence limits: PASS.
- Required graceful broker restart/persistence check: BLOCKED. Sprint 5 remains unclosed until verified.

## Manual device check

1. Connect using the ESP32 handoff. Subscribe to `robot-market/test/v1/esp32-lab-001/acks` (or the handoff wildcard) and wait for subscription success.
2. Publish any nonretained payload to `robot-market/test/v1/esp32-lab-001/telemetry`.
3. Confirm inbound payload, outbound `receipt_ack_sent`, and ACK counters in the browser. Confirm the device receives JSON with `type=receipt_ack`, `ok=true`, and the original source topic.
4. A receipt means the backend observed bytes. Broker PUBACK only means broker transport acceptance; neither proves device command execution or durable storage. Publishing on /commands or /acks does not create a receipt ACK.
5. Repeat with MQTT 3.1.1 and MQTT 5 if the firmware library supports protocol selection; record the actual library/build and CONNACK behavior. For MQTT 5, test native response/correlation properties explicitly.