updated and tested till end of lesson 4

lessons/01-lab-setup.md

@@ -1,11 +1,31 @@
-# Section 1 — Prepare the MQTT 5 lab
+# 🛠️ Section 1 — Prepare the MQTT 5 lab
 
 Read this first. You will create the files and run the commands yourself. All paths below are relative to the learning project root. The examples use the module path `example.com/mqtt5-learning`; if you change it, change the imports in later lessons too.
 
 **Version baseline:** Paho `v0.23.0`, verified from upstream release tags on 2026-10-07. It requires Go 1.24 or newer; Go 1.25 is compatible. The matching local reference is `references/paho.golang-v0.23.0/` (commit `1aa03969c5932464359b3e4ed752d9ac61abccc9`).
 
+## 🎬 Start with the system story
 
-## Working directory and file placement
+> 💡 **Story question:** Before we write a Go client, what must already exist so the backend and a simulated device can connect safely and exchange messages?
+
+The broker is a separately configured service. It needs a listener, an authentication source, topic permissions, and storage settings. The backend and device are separate clients with their own connection settings and credentials. This lesson prepares the local broker and verifies its behavior with MQTT 5 CLI clients before adding Go to the path.
+
+```mermaid
+flowchart LR
+    O[Prepare Compose and Mosquitto config] --> B[Mosquitto broker starts]
+    P[Create backend and simulator credentials] --> B
+    A[Define ACL topic permissions] --> B
+    G[Go backend comes in a later lesson] -. connects .-> B
+    D[Go simulator comes in a later lesson] -. connects .-> B
+    C[Mosquitto CLI verifies MQTT 5] --> B
+```
+
+The Compose file starts and isolates the broker; the Mosquitto configuration tells the broker how to listen and enforce authentication/ACLs. A Docker volume preserves broker-owned state. None of these files configures the Go clients—that happens in later lessons.
+
+Before each setup step, ask which process owns the setting and what observation will prove it took effect.
+
+
+## 📁 Working directory and file placement
 
 Run every shell command in this lesson from the learning project root:
 
@@ -15,7 +35,11 @@ cd /home/ryan/Media/Code/Code/robot-market/mqtt-5-learning
 
 Paths such as `cmd/backend/main.go` are relative to this directory. Markdown files under `lessons/` explain the work; create the application files at the paths shown, not inside `lessons/` or `references/`. Code blocks labeled Go, TOML, YAML, or conf are file contents; shell blocks are terminal commands. Create parent directories before saving files.
 
-## 1. Check the tools and create the module
+## 🛠️ 1. Check the tools and create the module
+
+> 💡 **Story question:** What must be available on this machine to build the Go client and start the isolated broker, and which Paho API speaks MQTT 5?
+
+Check the Go toolchain and both Docker components, then pin the MQTT 5 module in this learning project. The old `paho.mqtt.golang` client is a separate MQTT 3 API.
 
 ```sh
 go version
@@ -33,7 +57,11 @@ Run `go mod init` only once. `go get` pins the dependency in `go.mod`; commit `g
 
 The old backend's `github.com/eclipse/paho.mqtt.golang` is a different MQTT 3 client. MQTT 5 uses `github.com/eclipse/paho.golang/autopaho` and `/paho`. There is no `SetProtocolVersion(5)` needed in these MQTT 5 examples.
 
-## 2. Create the broker files
+## 🦟 2. Create the broker files
+
+> 💡 **Story question:** What configuration belongs to Mosquitto itself, and how will host applications reach the broker without exposing the lab listener to the network?
+
+Compose controls the container boundary and port mapping. Mosquitto's own files control its listener, credentials, ACL, and persistence. Keep those roles distinct as you create the files below.
 
 Create these directories:
 
@@ -79,7 +107,7 @@ volumes:
 
 This is a pinned Mosquitto 2 lab image, not a claim that it is the newest broker release. Host port 18883 maps to container port 1883. Containers use `broker:1883`; Go programs on your host use `127.0.0.1:18883`. Loopback binding keeps this listener off your LAN. Persistence belongs to this Compose project only.
 
-### Understand the Compose definition
+### 🐳 Understand the Compose definition
 
 `services` lists the containers in this application; `broker` is the service name you use in commands and the DNS name on this Compose network. `image` selects a prebuilt image; Docker downloads it if needed. There is no `build` section because this lesson does not build a broker image.
 
@@ -100,7 +128,7 @@ log_dest stdout
 connection_messages true
 ```
 
-### Understand each broker setting
+### 🦟 Understand each broker setting
 
 | Setting | Purpose and operational consequence |
 | --- | --- |
@@ -142,7 +170,11 @@ Add to `.gitignore` (preserve any existing rules):
 
 Git ignore patterns are file contents, not terminal commands. `/lab/secrets/` ignores the root-level directory and its contents; `/.env` ignores a root environment file. Ignoring a file does not untrack a secret already committed. Keep real secrets out of the initial commit as well as later changes.
 
-## 3. Create passwords interactively
+## 🔐 3. Create passwords interactively
+
+> 💡 **Story question:** How will the broker recognize the backend and simulator when they connect, and how can those identities receive different topic permissions?
+
+Create separate broker accounts and ACL rules before starting clients. The client supplies credentials; the broker validates them against its configured source. Authentication permits connection, while ACL rules govern topic operations.
 
 These commands create a lab password file without putting the password literal in your shell history. Run them from the project root:
 
@@ -155,7 +187,7 @@ docker run --rm --user 0 -v "$PWD/lab/secrets:/work" eclipse-mosquitto:2.0.22 ch
 
 `-c` creates/replaces the file. Use it only for the first username; using it again would erase the backend entry. The last two commands make the file readable by the broker account in this image. Container-root use is limited to setting up this mounted lab file.
 
-### Why four Docker commands?
+### 🔧 Why four Docker commands?
 
 There are two jobs here: generate password hashes for two users, then give the broker permission to read the resulting file. Docker adds a wrapper because we use the tool bundled with the pinned broker image instead of installing Mosquitto utilities on your computer. Each command runs a temporary utility container; none starts the broker.
 
@@ -178,7 +210,7 @@ The generated file stores password hashes, not the plaintext passwords you type.
 
 Linux checks numeric user/group IDs. `chown` inside this image picks the broker's IDs; your host may display a numeric owner or a different name for those same IDs. With mode `600`, your regular host user may be unable to read the file afterward. That is expected: the broker must read it, and subsequent password maintenance can use the same temporary root utility container. Do not make the file world-readable to work around this.
 
-### What is usual practice?
+### 💡 What is usual practice?
 
 For a small Mosquitto 2.0 installation, generating a password file with `mosquitto_passwd`, configuring `password_file`, and restricting file access is a conventional approach. The official [authentication guide](https://mosquitto.org/documentation/authentication-methods/) documents creating the first user with `-c` and adding or updating users without it. Docker is simply how this lesson obtains that utility.
 
@@ -189,7 +221,7 @@ For a small Mosquitto 2.0 installation, generating a password file with `mosquit
 
 The four explicit commands make the setup visible for learning. Once you understand them, wrapping them in a setup script is reasonable. We keep interactive prompts here so password literals do not appear in shell history or process arguments.
 
-### Verify password setup before starting
+### 🧪 Verify password setup before starting
 
 Do not start the broker until both password prompts and both permission commands have completed successfully. The two accounts must exist in the generated file; creating an empty file does not create usable credentials.
 
@@ -201,7 +233,7 @@ docker run --rm --user mosquitto -v "$PWD/lab/secrets:/work:ro" eclipse-mosquitt
 
 `--user mosquitto` tests access as the image's broker account. `sh -c` runs the quoted checks inside the utility container; `test -s` requires a nonempty file, `test -r` requires read access, and `&&` runs the second check only if the first succeeds. Success is quiet with exit status zero. Failure means you must fix the file/path/permissions before starting. This checks file access, not whether the typed client password is correct.
 
-### If permission changes do not stick on the host filesystem
+### 🔐 If permission changes do not stick on the host filesystem
 
 Verify final metadata with `stat -c '%a %u:%g %n' lab/secrets/passwords`. `stat` reports mode, numeric owner/group, and pathname without exposing hashes. Also use `findmnt -T lab/secrets/passwords -o TARGET,FSTYPE,OPTIONS` to identify the underlying mount. Some mounted filesystems synthesize permissions: `chmod`/`chown` can appear successful while the metadata remains unchanged. Do not conclude the file is protected from the commands' exit status alone.
 
@@ -227,7 +259,7 @@ volumes:
 
 For every subsequent add-user, rotation, or permission command, mount `mqtt5-learning_broker-secrets:/work` rather than `$PWD/lab/secrets:/work`: the broker now reads the volume copy. Editing the old host file does not update the live credential store. Include this external volume in backups; Compose `down -v` does not delete external volumes. The old host copy remains sensitive and ignored by Git; it has been preserved rather than automatically deleted.
 
-### Understand password-tool and entrypoint warnings
+### 🔐 Understand password-tool and entrypoint warnings
 
 During creation or update, `mosquitto_passwd` can warn about world-readable permissions or ownership. Do not assume these warnings prove the final file is correct: run the ownership/mode commands afterward and verify access. For this pinned Mosquitto 2.0.22 image, use `mosquitto:mosquitto` ownership and mode `600`. A password file needs no executable permission. Blindly following a warning suggesting root ownership with owner-only access can prevent the unprivileged broker from reading it. Review requirements again when upgrading rather than treating a future-version warning as this version's deployment instructions.
 
@@ -266,7 +298,11 @@ For this project, resource names commonly include `mqtt5-learning-broker-1`, `mq
 
 In `ps`, the local-only mapping should show `127.0.0.1:18883->1883/tcp`. A mapping showing `0.0.0.0` or `[::]` exposes that published port on all corresponding host interfaces. Change the Compose mapping to the loopback form and apply it with `up -d broker`; restarting alone does not change port bindings.
 
-## 4. Verify MQTT 5 without Go first
+## 🧪 4. Verify MQTT 5 without Go first
+
+> 💡 **Story question:** How can we prove that the broker is listening, speaks the intended MQTT version, accepts the right credentials, and enforces topic permissions before debugging Go code?
+
+Use Mosquitto CLI clients as a controlled first pair of MQTT clients. Their packet traces isolate broker reachability, protocol negotiation, authentication, authorization, and routing from application code.
 
 Use host-installed `mosquitto_sub` and `mosquitto_pub`, or run their equivalents through `docker compose exec broker` with port 1883 inside the container. Keep a subscription open in terminal A:
 
@@ -290,7 +326,7 @@ For the CLI only, `-P` exposes its argument to shell history/process inspection.
 
 Repeat with a wrong password. Also try publishing as `simulator` to `lab/v1/devices/device-002/telemetry`; this must not reach an authorized observer. Check the MQTT reason and broker logs rather than assuming every denied operation has the same client error.
 
-### Read the CLI exchange, not just the payload
+### 📘 Read the CLI exchange, not just the payload
 
 With `-d`, identify CONNECT/CONNACK first, then SUBSCRIBE/SUBACK in the subscriber terminal. Only publish after SUBACK. In the publisher terminal, find PUBLISH/PUBACK at QoS 1. In the subscriber terminal, find the separately forwarded PUBLISH and its acknowledgement. Debug formatting varies by client version; learn the packet names rather than memorizing one output line.
 
@@ -303,7 +339,7 @@ The broker needs an established matching subscription when this nonretained mess
 
 </details>
 
-## API preview: connect each setting to its purpose
+## ⚙️ API preview: connect each setting to its purpose
 
 ```go
 broker, err := url.Parse("mqtt://127.0.0.1:18883")
@@ -317,7 +353,7 @@ cfg := autopaho.ClientConfig{
 
 This is an illustrative fragment, not a standalone program. The URL chooses the transport endpoint, while Client ID selects the broker session identity. `KeepAlive` is an integer number of seconds, not a `time.Duration`. The embedded `ClientConfig` describes the underlying MQTT client; the outer configuration describes managed connections.
 
-## Exercises and questions
+## 🧪 Exercises and questions
 
 1. Explain why switching the Go library is more than changing one import.
 2. Predict whether the simulator may subscribe to another device's commands; verify it.
@@ -326,11 +362,15 @@ This is an illustrative fragment, not a standalone program. The URL chooses the
 
 **Completion:** CLI publish/subscribe works using MQTT 5, invalid credentials fail, and cross-device writes are denied. Do not use `down -v` unless you deliberately want to erase the lab's broker persistence.
 
-## Broker administration you should practice
+## ✅ Result — the story so far
+
+The broker is now the meeting point for the lab, listening only on this computer. It checks each client's username and password, then uses the ACL to decide which topic directions that identity may use. MQTT 5 CLI clients can prove the broker routes an allowed message and rejects an unauthorized one. The Go backend and simulator have not joined the story yet; this lesson prepares the place they will meet.
+
+## 🦟 Broker administration you should practice
 
 Read [Broker operations](broker-operations.md) for explained stop/start, restart versus recreate, password rotation, backups, restore, upgrades, and the steps required before remote deployment. Complete its local lifecycle and credential-rotation exercises before treating Section 1 as finished. Later Section 9 measures session recovery, and Section 11 in the course adds TLS and stronger access controls.
 
-## Common failures
+## 🔎 Common failures
 
 | Symptom | Check |
 | --- | --- |
@@ -345,7 +385,7 @@ Read [Broker operations](broker-operations.md) for explained stop/start, restart
 | Not authorized | Username, password, and exact permitted topic direction |
 | Pub/sub works but Go fails later | Go URL/credentials, client ID, then tagged API version |
 
-## Sources
+## 🔗 Sources
 
 - [Paho v0.23.0 release](https://github.com/eclipse-paho/paho.golang/releases/tag/v0.23.0)
 - [Mosquitto broker configuration](https://mosquitto.org/man/mosquitto-conf-5.html)