mqtt-5-learning/lessons/01-lab-setup.md

🛠️ Section 1 — Prepare the MQTT 5 lab

Read this first. You will create the files and run the commands yourself. All paths below are relative to the learning project root. The examples use the module path example.com/mqtt5-learning; if you change it, change the imports in later lessons too.

Version baseline: Paho v0.23.0, verified from upstream release tags on 2026-10-07. It requires Go 1.24 or newer; Go 1.25 is compatible. The matching local reference is references/paho.golang-v0.23.0/ (commit 1aa03969c5932464359b3e4ed752d9ac61abccc9).

🎬 Start with the system story

💡 Story question: Before we write a Go client, what must already exist so the backend and a simulated device can connect safely and exchange messages?

The broker is a separately configured service. It needs a listener, an authentication source, topic permissions, and storage settings. The backend and device are separate clients with their own connection settings and credentials. This lesson prepares the local broker and verifies its behavior with MQTT 5 CLI clients before adding Go to the path.

flowchart LR
    O[Prepare Compose and Mosquitto config] --> B[Mosquitto broker starts]
    P[Create backend and simulator credentials] --> B
    A[Define ACL topic permissions] --> B
    G[Go backend comes in a later lesson] -. connects .-> B
    D[Go simulator comes in a later lesson] -. connects .-> B
    C[Mosquitto CLI verifies MQTT 5] --> B

The Compose file starts and isolates the broker; the Mosquitto configuration tells the broker how to listen and enforce authentication/ACLs. A Docker volume preserves broker-owned state. None of these files configures the Go clients—that happens in later lessons.

Before each setup step, ask which process owns the setting and what observation will prove it took effect.

📁 Working directory and file placement

Run every shell command in this lesson from the learning project root:

cd /home/ryan/Media/Code/Code/robot-market/mqtt-5-learning

Paths such as cmd/backend/main.go are relative to this directory. Markdown files under lessons/ explain the work; create the application files at the paths shown, not inside lessons/ or references/. Code blocks labeled Go, TOML, YAML, or conf are file contents; shell blocks are terminal commands. Create parent directories before saving files.

🛠️ 1. Check the tools and create the module

💡 Story question: What must be available on this machine to build the Go client and start the isolated broker, and which Paho API speaks MQTT 5?

Check the Go toolchain and both Docker components, then pin the MQTT 5 module in this learning project. The old paho.mqtt.golang client is a separate MQTT 3 API.

go version
docker version
docker compose version
go mod init example.com/mqtt5-learning
go get github.com/eclipse/paho.golang@v0.23.0

go version reports the installed compiler. docker version reports both CLI and daemon versions: a missing server response means the CLI cannot reach the Docker engine. docker compose version checks the Compose plugin, which reads compose.yaml and manages the application's services. Having Docker installed does not prove the daemon is running.

go mod init example.com/mqtt5-learning creates go.mod, declaring this application's import-path prefix. It does not create the source directories. go get ...@v0.23.0 downloads a selected dependency and records module requirements and checksums in go.mod/go.sum; it may also update transitive requirements needed by that dependency.

Run go mod init only once. go get pins the dependency in go.mod; commit go.mod and go.sum when you initialize version control. Do not use a local replace directive to substitute the newer default-branch reference. A repository checkout is reading material, while a module requirement selects the compiled library.

The old backend's github.com/eclipse/paho.mqtt.golang is a different MQTT 3 client. MQTT 5 uses github.com/eclipse/paho.golang/autopaho and /paho. There is no SetProtocolVersion(5) needed in these MQTT 5 examples.

🦟 2. Create the broker files

💡 Story question: What configuration belongs to Mosquitto itself, and how will host applications reach the broker without exposing the lab listener to the network?

Compose controls the container boundary and port mapping. Mosquitto's own files control its listener, credentials, ACL, and persistence. Keep those roles distinct as you create the files below.

Create these directories:

mkdir -p lab/mosquitto
mkdir -p lab/secrets

mkdir creates directories; -p creates missing parents and tolerates an already-existing directory. These commands do not create configuration files.

Create compose.yaml directly in the project root, next to go.mod. Its full path is /home/ryan/Media/Code/Code/robot-market/mqtt-5-learning/compose.yaml. The ./lab/... volume paths below are relative to this Compose file:

mqtt-5-learning/
├── go.mod
├── compose.yaml
└── lab/
    ├── mosquitto/
    │   ├── mosquitto.conf
    │   └── acl
    └── secrets/
        └── passwords  # generated in step 3, not written by hand

Your existing compose.yml is also a supported Compose filename; keep it rather than creating a competing compose.yaml. Use one primary Compose file consistently. YAML nesting is controlled by spaces: the final volumes: declaration must align with services:, not with broker:. If nested under services, Compose treats volumes as another service and reports services.volumes additional properties 'broker-data' not allowed.

Paste this YAML into your primary Compose file:

services:
  broker:
    image: eclipse-mosquitto:2.0.22
    ports:
      - "127.0.0.1:18883:1883"
    volumes:
      - ./lab/mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
      - ./lab/mosquitto/acl:/mosquitto/config/acl:ro
      - ./lab/secrets:/mosquitto/secrets:ro
      - broker-data:/mosquitto/data
volumes:
  broker-data:

This is a pinned Mosquitto 2 lab image, not a claim that it is the newest broker release. Host port 18883 maps to container port 1883. Containers use broker:1883; Go programs on your host use 127.0.0.1:18883. Loopback binding keeps this listener off your LAN. Persistence belongs to this Compose project only.

🐳 Understand the Compose definition

services lists the containers in this application; broker is the service name you use in commands and the DNS name on this Compose network. image selects a prebuilt image; Docker downloads it if needed. There is no build section because this lesson does not build a broker image.

ports uses host-address:host-port:container-port. 127.0.0.1:18883:1883 forwards connections from this computer's loopback port 18883 to the container's listener at 1883. A host client cannot use container port 1883 unless that port is separately published. broker:1883 works for containers on the same Compose network, not arbitrary host programs or remote devices.

Each volumes entry is source:container-destination[:mode]. Sources beginning with ./ are host files/directories (bind mounts); :ro makes them read-only to this container. Create the configuration files before starting Docker: missing file paths can result in directories being mounted where files were expected. broker-data is a Docker-managed named volume, declared at the bottom, for writable broker persistence. It is separate from the image and survives ordinary container replacement.

Create lab/mosquitto/mosquitto.conf:

listener 1883
allow_anonymous false
password_file /mosquitto/secrets/passwords
acl_file /mosquitto/config/acl
persistence true
persistence_location /mosquitto/data/
log_dest stdout
connection_messages true

🦟 Understand each broker setting

Setting	Purpose and operational consequence
listener 1883	Opens the MQTT listener inside the container; host exposure is controlled separately by Compose
allow_anonymous false	Requires authentication; it does not grant topic permissions
password_file ...	Reads the generated username/password hashes at this container path
acl_file ...	Reads topic permissions; a correct password alone does not authorize every topic
persistence true	Enables saving broker state, including eligible retained/session information; it does not persist Go application state
persistence_location ...	Places saved broker data in the named volume rather than the disposable container filesystem
log_dest stdout	Sends broker logs to Docker's log stream for compose logs
connection_messages true	Logs client connection/disconnection events for troubleshooting

Persistence is not a database transaction for your application. Recently changed broker state may not survive an abrupt crash until it has been saved. Session settings and message QoS still decide which offline data is eligible to survive.

Create lab/mosquitto/acl:

user backend
topic write lab/v1/devices/+/commands
topic read lab/v1/devices/+/telemetry
topic read lab/v1/devices/+/responses

user simulator
topic read lab/v1/devices/device-001/commands
topic write lab/v1/devices/device-001/telemetry
topic write lab/v1/devices/device-001/responses

user starts rules for that authenticated username. topic write permits publishing; topic read permits receiving matching topics. + matches exactly one topic level, so the backend rules cover one device-ID segment. The simulator's rules use an exact device ID. Access outside the allowed patterns is denied by this simple ACL configuration. The text file has no extension because Mosquitto reads the path you configure, regardless of its filename suffix.

The simulator can act only as device-001. The backend can operate all lab devices. ACL checks authorize topics; they do not validate JSON or prove that a payload's identity is truthful. The CLI checks below must use a direction permitted for that username.

Add to .gitignore (preserve any existing rules):

/lab/secrets/
/.env

Git ignore patterns are file contents, not terminal commands. /lab/secrets/ ignores the root-level directory and its contents; /.env ignores a root environment file. Ignoring a file does not untrack a secret already committed. Keep real secrets out of the initial commit as well as later changes.

🔐 3. Create passwords interactively

💡 Story question: How will the broker recognize the backend and simulator when they connect, and how can those identities receive different topic permissions?

Create separate broker accounts and ACL rules before starting clients. The client supplies credentials; the broker validates them against its configured source. Authentication permits connection, while ACL rules govern topic operations.

These commands create a lab password file without putting the password literal in your shell history. Run them from the project root:

docker run --rm -it --user 0 -v "$PWD/lab/secrets:/work" eclipse-mosquitto:2.0.22 mosquitto_passwd -c /work/passwords backend
docker run --rm -it --user 0 -v "$PWD/lab/secrets:/work" eclipse-mosquitto:2.0.22 mosquitto_passwd /work/passwords simulator
docker run --rm --user 0 -v "$PWD/lab/secrets:/work" eclipse-mosquitto:2.0.22 chown mosquitto:mosquitto /work/passwords
docker run --rm --user 0 -v "$PWD/lab/secrets:/work" eclipse-mosquitto:2.0.22 chmod 600 /work/passwords

-c creates/replaces the file. Use it only for the first username; using it again would erase the backend entry. The last two commands make the file readable by the broker account in this image. Container-root use is limited to setting up this mounted lab file.

🔧 Why four Docker commands?

There are two jobs here: generate password hashes for two users, then give the broker permission to read the resulting file. Docker adds a wrapper because we use the tool bundled with the pinned broker image instead of installing Mosquitto utilities on your computer. Each command runs a temporary utility container; none starts the broker.

Part of the command	What it does
docker run	Creates and runs a new container for this one operation
--rm	Removes that container when its command exits; the mounted host files remain
-it	Keeps standard input open and supplies a terminal for the password prompt
--user 0	Runs this setup operation as container root, permitting file creation and ownership changes on the mount
-v "$PWD/lab/secrets:/work"	Makes the host project's lab/secrets directory available at /work inside the container; this mount is writable
eclipse-mosquitto:2.0.22	Selects the same pinned image used by the broker
mosquitto_passwd -c /work/passwords backend	Creates the password file and prompts for the backend user's password
mosquitto_passwd /work/passwords simulator	Adds the simulator user to that same file, without replacing the backend entry
chown mosquitto:mosquitto /work/passwords	Assigns file ownership to the broker account as defined inside this image
chmod 600 /work/passwords	Gives the owner read/write access and removes group/other access

$PWD is your current terminal directory, so run these from the project root. /work/passwords in the setup container and lab/secrets/passwords on your host are the same file. Later, Compose mounts that directory at /mosquitto/secrets with :ro, and mosquitto.conf reads /mosquitto/secrets/passwords. The setup mount is writable; the running broker mount is read-only.

The generated file stores password hashes, not the plaintext passwords you type. Your backend and simulator still need their original passwords to connect; a client cannot use the stored hash as its login password. Keep the generated file out of Git. The ACL file separately determines which topics each authenticated username may use.

Linux checks numeric user/group IDs. chown inside this image picks the broker's IDs; your host may display a numeric owner or a different name for those same IDs. With mode 600, your regular host user may be unable to read the file afterward. That is expected: the broker must read it, and subsequent password maintenance can use the same temporary root utility container. Do not make the file world-readable to work around this.

💡 What is usual practice?

For a small Mosquitto 2.0 installation, generating a password file with mosquitto_passwd, configuring password_file, and restricting file access is a conventional approach. The official authentication guide documents creating the first user with -c and adding or updating users without it. Docker is simply how this lesson obtains that utility.

• Local Docker lab: Generate the file once with a temporary container, mount it into the broker, and repeat the update command only when adding users or rotating passwords. You do not rerun all four commands on each broker start. After modifying the file, reapply the ownership and mode if the utility replaced it, then reload authentication or restart only the lab broker.
• Host-installed Mosquitto: Run the installed mosquitto_passwd utility directly, then set permissions for the broker's service account. The core operations are the same; there is no container wrapper.
• Repeated deployment: Put the provisioning steps in a small reviewed setup script or dedicated setup job. Supply credentials through the deployment's secret-management mechanism and mount the resulting authentication file read-only. A secret mount still needs the correct broker-readable ownership and permissions. Avoid embedding passwords in image layers, committed Compose files, or command arguments.
• Larger device fleets: Evaluate managed authentication and authorization, such as Mosquitto's Dynamic Security plugin, rather than maintaining a large static file manually. This is a deployment choice, not something needed for this two-user exercise.

The four explicit commands make the setup visible for learning. Once you understand them, wrapping them in a setup script is reasonable. We keep interactive prompts here so password literals do not appear in shell history or process arguments.

🧪 Verify password setup before starting

Do not start the broker until both password prompts and both permission commands have completed successfully. The two accounts must exist in the generated file; creating an empty file does not create usable credentials.

Verify existence and access without printing hashes:

docker run --rm --user mosquitto -v "$PWD/lab/secrets:/work:ro" eclipse-mosquitto:2.0.22 sh -c 'test -s /work/passwords && test -r /work/passwords'

--user mosquitto tests access as the image's broker account. sh -c runs the quoted checks inside the utility container; test -s requires a nonempty file, test -r requires read access, and && runs the second check only if the first succeeds. Success is quiet with exit status zero. Failure means you must fix the file/path/permissions before starting. This checks file access, not whether the typed client password is correct.

🔐 If permission changes do not stick on the host filesystem

Verify final metadata with stat -c '%a %u:%g %n' lab/secrets/passwords. stat reports mode, numeric owner/group, and pathname without exposing hashes. Also use findmnt -T lab/secrets/passwords -o TARGET,FSTYPE,OPTIONS to identify the underlying mount. Some mounted filesystems synthesize permissions: chmod/chown can appear successful while the metadata remains unchanged. Do not conclude the file is protected from the commands' exit status alone.

This workspace exhibited that behavior on its fuseblk mount: the password file remained 755 and 1000:1000. The current compose.yml therefore uses a separately provisioned Docker volume for credentials. To reproduce that fallback after generating the users:

docker run --rm --user 0 -v "$PWD/lab/secrets:/source:ro" -v mqtt5-learning_broker-secrets:/target eclipse-mosquitto:2.0.22 sh -c 'cp /source/passwords /target/passwords && chown mosquitto:mosquitto /target/passwords && chmod 600 /target/passwords && stat -c "%a %u:%g" /target/passwords'

The host source is read-only. The new Docker-managed target is writable; cp copies the existing file and the following commands apply permissions there. This command replaces the target password file, so use it only for initial provisioning or an intentional replacement. The verified result in this image is 600 1883:1883; always inspect the actual result for your environment. A named volume is not automatically secure or encrypted: its actual filesystem and file permissions still matter.

Replace the service's ./lab/secrets:/mosquitto/secrets:ro mount with broker-secrets:/mosquitto/secrets:ro. Declare this additional volume at the root, alongside broker-data:

volumes:
  broker-data:
  broker-secrets:
    external: true
    name: mqtt5-learning_broker-secrets

name specifies the already-provisioned Docker volume's exact name. external: true tells Compose to use it rather than create/delete it. The preceding utility run creates it if missing. Apply the changed mount with up -d broker and inspect recent logs. Verify access with the earlier read-check command but replace its host mount with -v mqtt5-learning_broker-secrets:/work:ro.

For every subsequent add-user, rotation, or permission command, mount mqtt5-learning_broker-secrets:/work rather than $PWD/lab/secrets:/work: the broker now reads the volume copy. Editing the old host file does not update the live credential store. Include this external volume in backups; Compose down -v does not delete external volumes. The old host copy remains sensitive and ignored by Git; it has been preserved rather than automatically deleted.

🔐 Understand password-tool and entrypoint warnings

During creation or update, mosquitto_passwd can warn about world-readable permissions or ownership. Do not assume these warnings prove the final file is correct: run the ownership/mode commands afterward and verify access. For this pinned Mosquitto 2.0.22 image, use mosquitto:mosquitto ownership and mode 600. A password file needs no executable permission. Blindly following a warning suggesting root ownership with owner-only access can prevent the unprivileged broker from reading it. Review requirements again when upgrading rather than treating a future-version warning as this version's deployment instructions.

The image entrypoint can also print chown: ... Read-only file system for the intentional :ro mounts. Those lines alone are not a broker authentication failure. Keep configuration and secrets read-only; identify the subsequent Mosquitto messages. Error opening password file is fatal and must be fixed. A directory can exist while the required passwords file inside it is missing.

Start the broker for the first time:

docker compose -p mqtt5-learning config
docker compose -p mqtt5-learning up -d
docker compose -p mqtt5-learning ps
docker compose -p mqtt5-learning logs --tail=50 broker

Command part	What happens and what to check
-p mqtt5-learning	-p means project name in Compose, not network port. Selects the Compose project name, keeping its containers/network/volume names separate from other projects; use this same name for every lifecycle command
config	Parses and resolves Compose configuration; catches YAML/interpolation errors, but does not test broker authentication or every mounted file's contents
up -d	Creates/updates the required resources and starts containers in the background; -d means detached
ps	Shows container state and published ports; a running container is not an MQTT readiness check
logs --tail=50 broker	Shows the last 50 log lines for this service without opening a shell inside it

If a container was already created or ran before you fixed its password file, explicitly retry startup:

docker compose -p mqtt5-learning restart broker
docker compose -p mqtt5-learning logs --since=1m broker
docker compose -p mqtt5-learning ps

restart stops/starts the existing service even if it has stopped after an error. up -d is not a guaranteed restart of an unchanged running container. --since=1m restricts logs to the last minute so earlier failures do not get mistaken for a fresh failure. If there is no existing container, use the initial up -d sequence instead. If you changed Compose ports or mounts, use up -d to reconcile them; if a replaced single-file mount still points at old content, use up -d --force-recreate broker.

Expect a listening broker and no new password/ACL file errors. A running container alone does not prove authentication works.

For this project, resource names commonly include mqtt5-learning-broker-1, mqtt5-learning_default, and mqtt5-learning_broker-data. Keep -p mqtt5-learning on every lifecycle command so they refer to the same project. In mosquitto_pub/sub, -p instead means port; in mkdir -p, it means create parent directories. Flags belong to their commands, not to a universal vocabulary.

In ps, the local-only mapping should show 127.0.0.1:18883->1883/tcp. A mapping showing 0.0.0.0 or [::] exposes that published port on all corresponding host interfaces. Change the Compose mapping to the loopback form and apply it with up -d broker; restarting alone does not change port bindings.

🧪 4. Verify MQTT 5 without Go first

💡 Story question: How can we prove that the broker is listening, speaks the intended MQTT version, accepts the right credentials, and enforces topic permissions before debugging Go code?

Use Mosquitto CLI clients as a controlled first pair of MQTT clients. Their packet traces isolate broker reachability, protocol negotiation, authentication, authorization, and routing from application code.

Use host-installed mosquitto_sub and mosquitto_pub, or run their equivalents through docker compose exec broker with port 1883 inside the container. Keep a subscription open in terminal A:

mosquitto_sub -h 127.0.0.1 -p 18883 -V mqttv5 -u backend -P '<backend-lab-password>' -t 'lab/v1/devices/device-001/telemetry' -q 1 -v -d

In terminal B:

mosquitto_pub -h 127.0.0.1 -p 18883 -V mqttv5 -u simulator -P '<simulator-lab-password>' -t 'lab/v1/devices/device-001/telemetry' -q 1 -m '{"online":true}' -d

-h chooses the broker host; -p here chooses the network port (unlike Compose's project-name flag). -u supplies a username, -P its password, -t the subscription filter or publication topic, and -m the message body. Single quotes preserve the JSON bytes instead of letting the shell interpret them. Start the subscriber first so it is ready before the nonretained publication. mosquitto_sub keeps running; Ctrl+C stops this subscriber. mosquitto_pub normally exits after completing its publication.

If using docker compose -p mqtt5-learning exec broker, that prefix executes the utility inside the already-running broker container. Use -h 127.0.0.1 -p 1883 there, because loopback refers to that container. The other MQTT flags remain the same. exec requires a running service; the earlier docker run setup commands do not.

The placeholders are not literal passwords. -V mqttv5 selects the protocol; -q 1 selects QoS; -d shows packet activity; -v prints topic plus payload. Expect the subscriber to print the telemetry JSON. This is a synthetic message, not a production heartbeat contract.

For the CLI only, -P exposes its argument to shell history/process inspection. Use temporary lab credentials and avoid pasting real credentials; consult your installed client's options for a protected configuration file if you need repeated CLI checks. Go examples read the password from the environment and never print it.

Repeat with a wrong password. Also try publishing as simulator to lab/v1/devices/device-002/telemetry; this must not reach an authorized observer. Check the MQTT reason and broker logs rather than assuming every denied operation has the same client error.

📘 Read the CLI exchange, not just the payload

With -d, identify CONNECT/CONNACK first, then SUBSCRIBE/SUBACK in the subscriber terminal. Only publish after SUBACK. In the publisher terminal, find PUBLISH/PUBACK at QoS 1. In the subscriber terminal, find the separately forwarded PUBLISH and its acknowledgement. Debug formatting varies by client version; learn the packet names rather than memorizing one output line.

Predict before running: stop the subscriber, publish again with the same command, then restart the subscriber. This nonretained message is not a history entry for a new subscriber. A successful publisher acknowledgement does not create a future subscription or a device response. Lesson 4 maps these exchanges to Go calls; Lesson 9 introduces the different case of an existing offline session.

Check your explanation: why start the subscriber first?

The broker needs an established matching subscription when this nonretained message arrives. Starting two terminals almost simultaneously leaves a race between SUBSCRIBE and PUBLISH. Seeing the subscriber process running is weaker evidence than seeing its successful SUBACK.

⚙️ API preview: connect each setting to its purpose

broker, err := url.Parse("mqtt://127.0.0.1:18883")
// Check err before using broker.
cfg := autopaho.ClientConfig{
    ServerUrls: []*url.URL{broker},
    KeepAlive: 20,
    ClientConfig: paho.ClientConfig{ClientID: "lab-backend"},
}

This is an illustrative fragment, not a standalone program. The URL chooses the transport endpoint, while Client ID selects the broker session identity. KeepAlive is an integer number of seconds, not a time.Duration. The embedded ClientConfig describes the underlying MQTT client; the outer configuration describes managed connections.

🧪 Exercises and questions

1. Explain why switching the Go library is more than changing one import.
2. Predict whether the simulator may subscribe to another device's commands; verify it.
3. Identify which packet proves a QoS 1 publication reached its immediate receiver, and why it does not prove application processing.
4. Stop and start only this lab with docker compose -p mqtt5-learning stop and docker compose -p mqtt5-learning start.

Completion: CLI publish/subscribe works using MQTT 5, invalid credentials fail, and cross-device writes are denied. Do not use down -v unless you deliberately want to erase the lab's broker persistence.

✅ Result — the story so far

The broker is now the meeting point for the lab, listening only on this computer. It checks each client's username and password, then uses the ACL to decide which topic directions that identity may use. MQTT 5 CLI clients can prove the broker routes an allowed message and rejects an unauthorized one. The Go backend and simulator have not joined the story yet; this lesson prepares the place they will meet.

🦟 Broker administration you should practice

Read Broker operations for explained stop/start, restart versus recreate, password rotation, backups, restore, upgrades, and the steps required before remote deployment. Complete its local lifecycle and credential-rotation exercises before treating Section 1 as finished. Later Section 9 measures session recovery, and Section 11 in the course adds TLS and stronger access controls.

🔎 Common failures

Symptom	Check
Port is already allocated	Choose another loopback host port and update all host clients
Connection refused	Container state, listener logs, and host/container port distinction
Password file missing	Complete password generation before startup; verify lab/secrets/passwords exists and is nonempty
Password file unreadable	Verify broker-account read access, ownership/mode, mounted path, and directory traversal permissions
Read-only chown warnings	Check subsequent broker logs; retain :ro mounts and fix real read failures separately
Old error still appears in logs	Use logs --since=1m immediately after retrying startup
services.volumes validation error	Align the named-volume declaration with root-level services:
Port shows 0.0.0.0 or [::]	Set 127.0.0.1:18883:1883 and apply with up -d broker
Not authorized	Username, password, and exact permitted topic direction
Pub/sub works but Go fails later	Go URL/credentials, client ID, then tagged API version

🔗 Sources

• Paho v0.23.0 release
• Mosquitto broker configuration
• Password-file tool
• Mosquitto publish tool

Next: Section 2.