updated and tested till end of lesson 4

lessons/05-contracts-validation.md

@@ -1,10 +1,28 @@
-# Section 5 — Design contracts and validate inbound messages
+# 📋 Section 5 — Design contracts and validate inbound messages
 
-Prerequisite: Section 4 exchanges heartbeats and ping/pong.
+## 🎬 Start with the system story
+
+> 💡 **Story question:** The broker can route a message because its topic matches. How does the Go backend decide that the topic and payload describe a valid message from the expected device?
+
+Routing is not validation or identity proof. A message arrives as a topic plus bytes; the application must parse the route, decode the payload, and check that fields and identity make sense before changing state. A device ID inside JSON is a claim, so broker authentication and topic authorization remain separate safeguards.
+
+```mermaid
+flowchart LR
+    B[Broker routes matching topic] --> P[Paho delivers topic + bytes]
+    P --> T[Parse topic and device scope]
+    T --> J[Decode versioned JSON]
+    J --> V[Validate identity, IDs, size, time]
+    V -->|valid| H[Business handler]
+    V -->|invalid| E[Safe rejection / metric]
+```
+
+**Before each implementation step, ask:** What can be trusted at this boundary, and what must be checked before the application acts? Follow [Lesson 0](00-system-lifecycle.md) for the complete message path.
+
+Prerequisite: Section 4 exchanges heartbeats and ping/pong. The clients share the disposable password in `configs/lab.toml`; select the simulator account with `-username simulator` when starting it.
 
 Run all commands from `/home/ryan/Media/Code/Code/robot-market/mqtt-5-learning`. Create files at the project-relative paths shown below, not under `lessons/`. Continue the same application and pinned Paho v0.23.0 dependency. These are guided implementation exercises; attempt each step before requesting a full solution.
 
-## Follow one message through the trust boundaries
+## 📨 Follow one message through the trust boundaries
 
 Suppose the backend receives topic `lab/v1/devices/device-001/sales` with a body claiming `device_id: device-002`. The broker may have correctly authorized the sender to publish to the topic. It has not established that the JSON is truthful. Reject the mismatch before selecting a device record or changing a balance.
 
@@ -17,24 +35,27 @@ Each step answers a different question. The size limit bounds decoding cost. Par
 
 The 16 KiB application limit applies to `Payload`, not the entire MQTT packet. Topic and properties add bytes; the library has already received the packet before this validator runs. Lesson 16 adds protocol packet-size limits to bound an earlier layer.
 
-### Map encoding metadata into Paho
+### 📨 Map encoding metadata into Paho
 
 This fragment belongs where you build a `paho.Publish`; `payload` is the JSON produced by your contract encoder:
 
 ```go
 utf8Payload := byte(1)
+
+// Describe the payload's encoding and media type for MQTT 5 consumers.
 properties := &paho.PublishProperties{
     ContentType:   "application/json",
     PayloadFormat: &utf8Payload,
 }
-_ = properties // attach as Publish.Properties in your implementation
+// Attach these properties to the PUBLISH packet built by your transport helper.
+_ = properties
 ```
 
 Payload Format Indicator 1 declares UTF-8 character data. It does not mean JSON, schema version 1, or trusted content. Content Type describes the media type. For example, `hello` is UTF-8 but fails your JSON object contract. Property pointers distinguish an omitted property from an explicitly supplied value; inspect nil before dereferencing incoming fields.
 
 Keep these distinctions in your transport type when needed: packet metadata belongs alongside payload bytes, while `schema_version` and `command_id` belong to your application contract. Do not make handlers depend on a mutable packet owned elsewhere merely to read metadata.
 
-### Predict, modify, explain
+### 🧪 Predict, modify, explain
 
 Send a valid heartbeat, then change only its body device ID, then change only its schema version. Predict which validation stage rejects each. Finally publish valid JSON with an extra trailing object: one successful decoder call is insufficient unless you also require EOF.
 
@@ -45,7 +66,11 @@ MQTT receipt and application validity are separate. With the early automatic-ack
 
 </details>
 
-## 1. Create the contract package
+## 📋 1. Create the contract package
+
+> 💡 **Story question:** What shared vocabulary lets the backend and device agree on the shape and meaning of each message?
+
+The contract package defines those wire-level names and fields so neither client silently invents a different JSON shape.
 
 ```sh
 mkdir -p internal/contracts
@@ -55,23 +80,37 @@ Create `internal/contracts/messages.go`, `topics.go`, `decode.go`, and `decode_t
 
 Define JSON structs for Heartbeat, Command, CommandResult, and SalesReport. Use `schema_version`, `message_id`, `device_id`, and `occurred_at` consistently. Command also needs `command_id` and `command`; CommandResult needs the same command ID and a status. A sales report needs a stable report ID and synthetic line items with positive quantities and integer minor-unit amounts. Keep command identity stable across retries; an MQTT packet identifier is not that identity.
 
-## 2. Parse topics before routing
+## 🏷️ 2. Parse topics before routing
+
+> 💡 **Story question:** A message arrives with both a topic and a payload. Why decide which device and message kind the topic represents before running a handler?
+
+Parsing the topic establishes the intended route from broker metadata; it also gives validation a value to compare against any device identity claimed in the payload.
 
 Implement a parser accepting exactly `lab/v1/devices/{deviceID}/{kind}`. Initially permit telemetry, commands, responses, and sales. Reject empty device IDs, extra segments, wildcards, and unknown kinds. Return a typed device ID and kind rather than making every handler split strings itself. Extend the lab ACL explicitly for sales only when you start publishing reports.
 
-## 3. Decode and validate
+## ✅ 3. Decode and validate
+
+> 💡 **Story question:** JSON decoding succeeded. What else must be true before this message is safe for application state?
+
+Structure is only one check. Bound the input, enforce the supported schema and required IDs, validate timestamps and values, and compare topic identity with the body claim.
 
 Apply a 16 KiB payload limit before decoding. Use `json.Decoder` with `DisallowUnknownFields`, then require EOF after the first JSON value. Validate schema version 1, required IDs, supported commands, topic/body device agreement, RFC3339 timestamps, and field lengths. Choose and document a future-clock tolerance, such as 30 seconds; inject the current time into validation to keep tests deterministic. Historical sales may legitimately be old, so do not apply heartbeat freshness rules to every report.
 
 Treat Content Type and Payload Format Indicator as descriptive metadata. Validate bytes even when they claim to contain JSON. A payload device ID is a claim; broker credentials and ACLs supply the separate authorization boundary.
 
-## 4. Integrate and verify
+## 🧪 4. Integrate and verify
+
+> 💡 **Story question:** Where should validation sit so malformed messages cannot trigger a response or change state?
+
+Run it between transport reception and business handling. Verify both accepted messages and rejection cases, and inspect broker ACLs separately when testing sender permissions.
 
 Replace Section 4's anonymous JSON structs in both programs with these contracts. Validate before updating state or sending a result. Log the error category and bounded identifiers, without dumping rejected payloads. Unsupported or malformed commands must have no simulated effect.
 
 ```sh
 gofmt -w internal/contracts cmd
+
 go test ./internal/contracts
+
 go test ./...
 ```
 
@@ -83,4 +122,8 @@ Test valid messages, malformed JSON, trailing JSON, unknown fields, missing IDs,
 
 **Done when:** Both programs still exchange ping/pong, every rejection has a predictable reason, and invalid sales or commands leave state unchanged. Explain why schema validation does not authenticate a device.
 
+## ✅ Result — the story so far
+
+The backend and simulator now have a shared vocabulary for heartbeats, commands, results, and sales reports. When a message arrives, the receiver checks its topic, size, JSON shape, schema version, identifiers, timestamp, and topic/body device match before changing application state. A broker-routed message can still be rejected by the application; successful MQTT delivery alone does not make its contents valid or trusted.
+
 Previous: [04-publish-subscribe-simulator.md](04-publish-subscribe-simulator.md). Next: [06-concurrency-cancellation.md](06-concurrency-cancellation.md).