updated and tested till end of lesson 4

lessons/07-command-response.md

@@ -1,10 +1,36 @@
-# Section 7 — Build reliable command and response behavior
+# 🔁 Section 7 — Build reliable command and response behavior
 
-Prerequisite: Sections 5–6 validate messages and bound concurrent processing.
+## 🎬 Start with the system story
+
+> 💡 **Story question:** The broker acknowledged a backend's command publication. How can the backend match a later device response to the right request—and what should it report if no response arrives?
+
+The MQTT acknowledgement covers delivery on one hop. The device's application response is a separate publication. The backend therefore needs application-level command identity, correlation, a pending state registered before publication, and a timeout policy that does not claim the device definitely did nothing.
+
+```mermaid
+sequenceDiagram
+    participant H as HTTP/backend caller
+    participant G as Go command service
+    participant B as Broker
+    participant D as Device
+    H->>G: Request command
+    G->>G: Create command ID and pending waiter
+    G->>B: PUBLISH command with correlation data
+    B-->>G: PUBACK: broker hop accepted
+    B->>D: PUBLISH command
+    D->>D: Validate, deduplicate, act
+    D->>B: PUBLISH correlated application result
+    B->>G: PUBLISH result
+    G->>G: Match identity/correlation; update state
+    G-->>H: Accepted / completed / rejected / unknown
+```
+
+**Before each implementation step, ask:** Which event changes command state: broker acknowledgement, device acceptance, device completion, or timeout? Keep those events separate. Follow [Lesson 0](00-system-lifecycle.md) for the full client and broker lifecycle.
+
+Prerequisite: Sections 5–6 validate messages and bound concurrent processing. The backend and simulator continue to read their shared disposable password from `configs/lab.toml`; select each broker username with `-username` when starting the process.
 
 Run all commands from `/home/ryan/Media/Code/Code/robot-market/mqtt-5-learning`. Create files at the project-relative paths shown below, not under `lessons/`. Continue the same application and pinned Paho v0.23.0 dependency. These are guided implementation exercises; attempt each step before requesting a full solution.
 
-## MQTT request/response is an application conversation
+## ↩️ MQTT request/response is an application conversation
 
 Response Topic tells the responder where a result should be published. Correlation Data lets the requester recognize which pending request it belongs to. The broker forwards these properties; it does not run the command, create a result, or enforce your timeout.
 
@@ -22,11 +48,13 @@ request := &paho.Publish{
         CorrelationData: append([]byte(nil), correlation...),
     },
 }
+
+// Send this packet only after the pending request has been registered.
 ```
 
 Pass `request` to the bounded publication operation after registering pending state. The simulator validates `ResponseTopic`, publishes the result to it, and copies Correlation Data unchanged into the result's properties. Correlation bytes are opaque; they need not be text. Do not trust an arbitrary supplied topic as a response destination.
 
-### Trace a fast response and a lost response
+### ↩️ Trace a fast response and a lost response
 
 1. Backend establishes its response subscription and checks SUBACK.
 2. Backend stores the expected device, command ID, correlation, and deadline.
@@ -42,7 +70,7 @@ Registering after step 3 loses the race with a fast response. Holding the comman
 | Correlation Data | Request/response mechanism | Matches a response; does not authenticate it or prevent repeated effects |
 | Command ID | Application, stable across retries | Keys the dedup record and business status |
 
-### Make duplicate behavior observable
+### ♻️ Make duplicate behavior observable
 
 Use a synthetic effect counter. Send command C twice with the same content, then C with changed content. Expect one effect, a replayed recorded result, then a conflict. Also send the same C in two separate publish calls: application duplicates need not carry MQTT DUP. The DUP flag describes protocol retransmission, so deduplicating only packets with DUP set is wrong.
 
@@ -55,31 +83,49 @@ Only the transport acknowledgement is established. The device may be offline, ma
 
 </details>
 
-## 1. Preserve MQTT 5 request metadata
+## 💾 1. Preserve MQTT 5 request metadata
+
+> 💡 **Story question:** How can a responder know where to send a reply and the requester know which command the reply belongs to?
+
+Use the MQTT 5 Response Topic and Correlation Data as transport metadata, while retaining a stable application command ID and checking that the response destination is authorized.
 
 Create `internal/commands/service.go`, `store.go`, and `service_test.go`. Edit `internal/mqtt/client.go` to carry Response Topic and a copied Correlation Data slice in received messages. Add an outbound request form that accepts these properties. Retain the existing bounded publish timeout and reason-code checks.
 
 Read `references/paho.golang-v0.23.0/paho/cp_publish.go` for `paho.PublishProperties`. Set `ResponseTopic` to `lab/v1/devices/device-001/responses` and `CorrelationData` to opaque bytes associated with the pending command. Establish the backend response subscription before sending. Register pending state before publishing so an immediate response cannot beat registration.
 
-## 2. Model evidence explicitly
+## 🧩 2. Model evidence explicitly
+
+> 💡 **Story question:** What should the backend show after a PUBACK but before any device response arrives?
+
+Represent broker delivery evidence separately from device acceptance, completion, rejection, and timeout/unknown outcome.
 
 Store command ID, expected device, correlation bytes, creation/deadline times, business status, and independent transport acknowledgement evidence. Start with created → pending delivery → accepted → completed or rejected. A ping simulator can send accepted followed by completed. Response timeout means unknown outcome; it does not establish that no effect happened.
 
 Protect concurrent access and define allowed transitions. A completed or rejected command must not regress on a duplicate accepted response. Compare command identity, topic/body device identity, and correlation before applying a response. Keep timed-out records for a bounded retention period even after removing their waiters, so status can explain the uncertainty.
 
-## 3. Make the simulator idempotent
+## ♻️ 3. Make the simulator idempotent
+
+> 💡 **Story question:** If QoS 1 or an application retry delivers the same command again, how does the device avoid repeating a non-idempotent effect?
+
+Record results by stable command ID and replay the prior result for duplicates instead of applying the simulated effect twice.
 
 Create `internal/simulator/dedup.go` and its tests. Index effects by stable command ID and retain a fingerprint of the command. For an identical duplicate, replay the recorded result without repeating the effect. Reject reuse of the ID with different content. Bound cache lifetime and size; state clearly that restarting the simulator loses this in-memory protection.
 
 Allow a Response Topic only when it exactly matches the authorized response topic for this simulator. Correlation is matching data, not authorization. Continue rejecting retained one-shot commands.
 
-## 4. Exercise timing and failures
+## 🧪 4. Exercise timing and failures
+
+> 💡 **Story question:** If a response is late or missing, what can the backend safely conclude, and what should happen to a later result?
+
+Exercise timeouts and duplicates while keeping uncertainty visible; a timeout is not proof that the device did nothing.
 
 Replace the backend's unconditional ticker with an explicit finite command exercise or service calls. Send 20 concurrent synthetic pings with separate IDs. Introduce delayed, duplicate, wrong-device, wrong-correlation, and out-of-order responses. Check every command receives only its own evidence. Retry a timed-out request with the same stable ID; never generate a new ID merely because publication timed out.
 
 ```sh
 mkdir -p internal/commands internal/simulator
+
 gofmt -w internal cmd
+
 go test -race ./...
 ```
 
@@ -91,4 +137,8 @@ Test responses arriving before publish returns, publish failure with uncertain o
 
 **Done when:** Concurrent requests match correctly, identical duplicates cause one effect, unexpected destinations are rejected, and late responses follow a documented policy. Explain why PUBACK and Correlation Data cannot prove business completion.
 
+## ✅ Result — the story so far
+
+The backend now treats a ping as a tracked command: it records the command ID, expected device, correlation data, and deadline before publishing. The simulator checks the command and authorized response destination, applies the simulated effect once for that stable ID, and publishes a result. The backend matches that result to the pending command and updates its status. A broker PUBACK still means only that the broker acknowledged its hop; a missing response leaves the outcome unknown, not necessarily unexecuted.
+
 Previous: [06-concurrency-cancellation.md](06-concurrency-cancellation.md). Next: [08-http-commands.md](08-http-commands.md).