feat: expand MQTT contract and reliability lessons

lessons/00-system-lifecycle.md

@@ -34,7 +34,9 @@ The broker has its own setup and runtime. Each client has its own configuration
 
 ## 🖥️ 1. Set up the broker
 
-> 💡 **Story question:** Before a client arrives, how does the broker know where to listen, which credentials to accept, and which topics each identity may use?
+**❓ Question:** **Before a client arrives, how does the broker know where to listen, which credentials to accept, and which topics each identity may use?**
+
+**💬 Answer:** Broker configuration supplies the listener, authentication source, and topic permissions. The clients present credentials; they do not create their own broker accounts.
 
 Those decisions belong to broker configuration or to an authentication/authorization service configured by the broker. In this lab, Mosquitto is configured with a listener, a password file, and an ACL file. The `backend` and `simulator` accounts are created before the clients connect.
 
@@ -58,7 +60,9 @@ For this learning environment, the broker is isolated and uses synthetic lab dat
 
 ## ⚙️ 2. Configure the backend and device
 
-> 💡 **Story question:** If both programs are MQTT clients, what distinguishes the backend's connection from the device's connection?
+**❓ Question:** **If both programs are MQTT clients, what distinguishes the backend's connection from the device's connection?**
+
+**💬 Answer:** Each process has its own Client ID, credentials, configuration, and subscriptions. In this lab, the username selects broker ACL rules; a Client ID identifies an MQTT connection/session and is not a password.
 
 Each process has its own configuration and client identity. Both need a broker URL and credentials accepted by that broker. They also need distinct Client IDs while running at the same time. Their usernames determine which broker ACL rules apply in this lab; Client ID identifies the MQTT connection/session and is not a password.
 
@@ -82,7 +86,9 @@ Connection settings are distinct from message choices. Keep Alive, session behav
 
 ## 🔌 3. Connect both clients
 
-> 💡 **Story question:** Once the Go process has configuration, what must it prove before it can claim to be connected?
+**❓ Question:** **Once the Go process has configuration, what must it prove before it can claim to be connected?**
+
+**💬 Answer:** It needs an accepted CONNACK. Creating a connection manager is not enough, and a successful connection does not prove that subscriptions or business operations succeeded.
 
 The program parses the broker URL, creates a Paho/AutoPaho configuration, and starts the connection manager. AutoPaho manages the connection lifecycle and reconnect attempts; Paho implements MQTT protocol exchanges. The manager being created is not yet proof of a successful connection. The broker must accept MQTT `CONNECT` with `CONNACK`, and the application can wait for that state with `AwaitConnection`.
 
@@ -107,7 +113,9 @@ The backend and device go through this independently. A successful `CONNACK` pro
 
 ## 📬 4. Register subscriptions
 
-> 💡 **Story question:** The broker accepted the connection. How does it learn which messages this client wants to receive?
+**❓ Question:** **The broker accepted the connection. How does it learn which messages this client wants to receive?**
+
+**💬 Answer:** The client sends SUBSCRIBE topic filters, and the broker reports each filter's result in SUBACK. The local Paho handler is a separate step that processes publications the broker delivers.
 
 The client sends `SUBSCRIBE` requests containing topic filters. The broker checks each request against its rules and returns `SUBACK` results. In the Go application, a local callback is also configured so Paho knows what to do when a matching publication arrives. The broker-side subscription and the Go-side callback are two separate parts of receiving a message.
 
@@ -130,7 +138,9 @@ The backend commonly listens for telemetry and responses. The device listens for
 
 ## 📨 5. Publish and route messages
 
-> 💡 **Story question:** Once subscriptions exist, how does a command travel from backend code to the device application?
+**❓ Question:** **Once subscriptions exist, how does a command travel from backend code to the device application?**
+
+**💬 Answer:** The backend publishes bytes to a topic, the broker routes them to matching subscriptions, and the device's MQTT client passes the publication to application code. QoS acknowledgements cover a delivery hop; the device's application response carries business evidence.
 
 The backend serializes an application value into bytes, publishes those bytes to a concrete topic, and the broker routes the publication to matching subscribed clients. Paho's receive callback runs inside the client process; the application can copy and enqueue the message for its main loop to handle.
 
@@ -157,7 +167,9 @@ There are separate MQTT delivery hops. A `PUBACK` on the backend's publication c
 
 ## 🔄 6. Recover and stop
 
-> 💡 **Story question:** What should happen if the broker or network disappears while the backend process remains alive?
+**❓ Question:** **What should happen if the broker or network disappears while the backend process remains alive?**
+
+**💬 Answer:** The client can reconnect, but the application must ensure needed subscriptions are present. Offline delivery depends on session settings, QoS, expiry, and broker state, so reconnect alone does not guarantee that every message is recovered.
 
 AutoPaho can reconnect according to its lifecycle policy. After a connection comes back, the application must ensure required subscriptions are present—either because the broker resumed a session or because the client re-subscribed. In this lab, Lesson 4's helper explicitly subscribes again on connection-up. Whether offline publications arrive depends on MQTT session settings, QoS, broker state, and expiry; reconnect alone does not promise recovery of every message.
 
@@ -185,35 +197,50 @@ stateDiagram-v2
 
 | Lifecycle area | Lessons | Evidence to understand |
 |---|---|---|
-| Broker setup, credentials, and topic permissions | [1](01-lab-setup.md), [11](11-security-boundaries.md) | Which process owns each setting; authentication versus authorization |
+| Broker setup, credentials, and topic permissions | [1](01-lab-setup.md), [11](11-security-boundaries.md), [11B](11b-dynamic-security.md, optional) | Mosquitto owns broker authentication and topic ACL enforcement; optional live per-device provisioning |
 | Go client configuration | [2](02-go-connection.md), [3](03-typed-configuration.md) | Effective settings, identity, validation, and precedence |
 | CONNECT, CONNACK, reconnect, and shutdown | [2](02-go-connection.md), [4](04-publish-subscribe-simulator.md), [9](09-sessions-recovery.md) | What connection acceptance proves and what must be restored |
-| Subscriptions and broker routing | [4](04-publish-subscribe-simulator.md), [9](09-sessions-recovery.md), [16](16-mqtt5-controls.md) | SUBACK results, matching filters, session restoration, and options |
-| Publish, delivery, and application response | [4](04-publish-subscribe-simulator.md), [5](05-contracts-validation.md) | Per-hop QoS acknowledgements versus validated business evidence |
+| Topic names, filters, wildcards, and routing | [1](01-lab-setup.md), [4](04-publish-subscribe-simulator.md), [16](16-mqtt5-controls.md) | Concrete topic names, `+`/`#` filter matching, broker routing, and subscription options |
+| Subscriptions and broker routing | [4](04-publish-subscribe-simulator.md), [9](09-sessions-recovery.md), [16](16-mqtt5-controls.md) | SUBACK results, matching filters, session restoration, shared subscriptions, and options |
+| QoS 0/1/2, duplicates, and acknowledgement scope | [4](04-publish-subscribe-simulator.md), [6](06-concurrency-cancellation.md), [7](07-command-response.md), [16](16-mqtt5-controls.md) | Per-hop packet evidence, DUP/retries, and why QoS does not make business effects exactly once |
+| Topic and payload validation in Go | [5](05-contracts-validation.md) | `ParseTopic` checks topic shape and kind; `Decode` validates JSON, schema, route/body device identity, and message values before handling |
 | Callback work, queues, and overload | [6](06-concurrency-cancellation.md) | Admission, acknowledgement timing, bounded capacity, and shutdown |
 | Reliable command/response and HTTP | [7](07-command-response.md), [8](08-http-commands.md) | Correlation, idempotency, timeout, and status reporting |
-| Expiry and device presence | [10](10-expiry-presence-state.md) | Stale messages and the limits of retained state |
+| Sessions, retained state, expiry, and presence | [9](09-sessions-recovery.md), [10](10-expiry-presence-state.md) | Offline delivery prerequisites, retained desired state, message expiry, Will, and heartbeat freshness |
+| MQTT 5 flow controls and capabilities | [16](16-mqtt5-controls.md) | Receive Maximum, packet limits, reason codes, aliases, and capability boundaries |
 
-The table links each part of this lifecycle to the lessons that explain or exercise it. Sections 12–15 and 17–18 in [the course plan](../COURSE.md) extend the story into persistence, process boundaries, and operations.
+The table points to the lessons that explain and exercise each part of the lifecycle. Lessons 1–11 and 16 contain detailed instructions; Sections 12–15 and 17–18 in [the course plan](../COURSE.md) are currently guided checklists that extend the story into vending workflows, persistence, process boundaries, and operations.
+
+### What validates a topic?
+
+Lesson 5's `contracts.ParseTopic` and each message's `Validate` method are necessary backend checks. They reject malformed routes, unsupported message kinds, and a payload whose claimed `device_id` does not match the topic. The typed decoder then checks the payload contract before application handling.
+
+Those checks do not authorize the publisher. A client with valid credentials could still try to publish to another device's topic, so Mosquitto must enforce a least-privilege ACL for both publish and subscribe operations. Lesson 11 tests that broker boundary and the separate HTTP-user authorization boundary; optional Lesson 11B covers per-device ACL provisioning. Authentication identifies a broker client, ACLs authorize topic operations, and Lesson 5 validates message content. Use all three boundaries.
 
 ---
 
 ## 🧪 Understanding check
 
 1. What does a successful `CONNACK` prove, and what does it not prove?
-2. Why does the backend need both a broker subscription and a local receive handler?
-3. Which acknowledgement tells the backend that the device application completed a command?
-4. What is the difference between a client credential and a Client ID in this lab?
+2. How are a topic name, a subscription filter, and an ACL rule different?
+3. Why does the backend need both a broker subscription and a local receive handler?
+4. Which acknowledgement tells the backend that the device application completed a command?
+5. What is the difference between backend topic parsing/validation and broker ACL authorization?
+6. What is the difference between a client credential and a Client ID in this lab?
 
 > [!answer]- 📋 Answers
 >
 > **1.** It proves the broker accepted that MQTT connection. It does not prove a subscription was accepted, a device is online, or business work completed.
 >
-> **2.** The broker subscription controls routing; the local handler controls what Paho/the application does with a delivered PUBLISH.
+> **2.** A topic name labels one publication route. A filter can match multiple topic names when the broker routes to a subscription; `+` matches one level and `#` matches the remaining levels. An ACL rule permits or denies a client's publish or subscribe operation.
 >
-> **3.** None of the MQTT packet acknowledgements proves application completion. The device must publish an application-level response, and the backend must validate it.
+> **3.** The broker subscription controls routing; the local handler controls what Paho/the application does with a delivered PUBLISH.
 >
-> **4.** Credentials authenticate a client and select its ACL identity here. Client ID identifies the MQTT connection/session and can be configured as an ACL input, but it is not inherently authentication.
+> **4.** None of the MQTT packet acknowledgements proves application completion. The device must publish an application-level response, and the backend must validate it.
+>
+> **5.** Backend validation checks message shape and consistency after delivery. Broker ACLs decide whether that client may publish or subscribe to the topic at all. Payload validation does not replace authorization.
+>
+> **6.** Credentials authenticate a client and select its ACL identity here. Client ID identifies the MQTT connection/session and can be configured as an ACL input, but it is not inherently authentication.
 
 ---