Files

T

Ryan 2d639d78eb updated and tested till end of lesson 4

2026-10-11 15:00:58 +03:30

14 KiB

Raw Blame History

🔌 Section 2 — Understand a Go MQTT 5 connection

Prerequisite: Section 1's broker and backend credentials work. This section contains a complete small program. Type it in stages, compile after each meaningful change, and explain the code before running the failure exercises.

Version baseline: Paho v0.23.0, verified from upstream release tags on 2026-10-07. It requires Go 1.24 or newer; Go 1.25 is compatible. The matching local reference is references/paho.golang-v0.23.0/ (commit 1aa03969c5932464359b3e4ed752d9ac61abccc9).

🎬 Start with the system story

💡 Story question: The broker is running and has accepted CLI clients. How does the Go backend become another MQTT client, and what event proves the broker accepted its connection?

The Go process loads its connection values, gives AutoPaho a client configuration, and starts a connection manager. AutoPaho manages the connection lifecycle; the underlying Paho client performs MQTT 5 protocol exchanges. The manager can start before the network connection succeeds, so the application waits for a successful CONNACK through AwaitConnection.

sequenceDiagram
    participant App as Go backend application
    participant Auto as AutoPaho manager
    participant Paho as Paho MQTT 5 client
    participant Broker as Mosquitto broker
    App->>Auto: NewConnection(lifecycle context, config)
    Auto->>Paho: create and manage client connection
    Paho->>Broker: TCP connection, then CONNECT
    Broker->>Broker: authenticate client credentials
    Broker-->>Paho: CONNACK accepted or rejected
    Paho-->>Auto: connection state / callback
    App->>Auto: AwaitConnection(startup deadline)
    Auto-->>App: connected or startup error

The backend is an MQTT client, just like the device. Its application role does not make the broker handshake different. A successful CONNACK proves connection acceptance; subscriptions, device availability, and business completion are later evidence. See Lesson 0 for the complete lifecycle.

📁 Working directory and file placement

Run every shell command in this lesson from the learning project root:

cd /home/ryan/Media/Code/Code/robot-market/mqtt-5-learning

Paths such as cmd/backend/main.go are relative to this directory. Markdown files under lessons/ explain the work; create the application files at the paths shown, not inside lessons/ or references/. Code blocks labeled Go, TOML, YAML, or conf are file contents; shell blocks are terminal commands. Create parent directories before saving files.

🔌 1. Build the connection program

💡 Story question: Which settings and lifecycle controls must the Go process pass to the client library so a connection attempt can start and stop predictably?

The program builds one AutoPaho configuration from the broker URL, credentials, Client ID, connection options, and callbacks. The root context owns the manager lifetime; a separate startup context bounds how long the application waits for the first connection.

Create the directory first:

mkdir -p cmd/backend

Create cmd/backend/main.go and paste the complete program below into it. If the file already exists, replace its contents rather than appending a second program:

package main

import (
	"context"
	"fmt"
	"log"
	"net/url"
	"os"
	"os/signal"
	"syscall"
	"time"

	"github.com/eclipse/paho.golang/autopaho"
	"github.com/eclipse/paho.golang/paho"
)

func main() {
	if err := run(); err != nil {
		log.Print(err)
		os.Exit(1)
	}
}

func run() error {
	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
	defer stop()

	user := os.Getenv("TEST_MQTT5_MQTT_USERNAME")
	password := os.Getenv("TEST_MQTT5_MQTT_PASSWORD")
	if user == "" || password == "" {
		return fmt.Errorf("set the lab username and password environment variables")
	}
	broker, err := url.Parse("mqtt://127.0.0.1:18883")
	if err != nil {
		return fmt.Errorf("parse broker URL: %w", err)
	}

	cfg := autopaho.ClientConfig{
		ServerUrls:                    []*url.URL{broker},
		KeepAlive:                     20,
		CleanStartOnInitialConnection: true,
		SessionExpiryInterval:         0,
		ConnectUsername:               user,
		ConnectPassword:               []byte(password),
		OnConnectionUp: func(_ *autopaho.ConnectionManager, ack *paho.Connack) {
			log.Printf("connected: session_present=%t reason=0x%02x", ack.SessionPresent, ack.ReasonCode)
		},
		OnConnectionDown: func() bool {
			log.Print("connection down; AutoPaho will retry")
			return true
		},
		OnConnectError: func(err error) {
			// Keep errors free of credentials; never log cfg or broker userinfo.
			log.Printf("connect attempt failed: %v", err)
		},
		ClientConfig: paho.ClientConfig{
			ClientID:      "lab-backend",
			OnClientError: func(err error) { log.Printf("client error: %v", err) },
			OnServerDisconnect: func(packet *paho.Disconnect) {
				log.Printf("server disconnect: reason=0x%02x", packet.ReasonCode)
			},
		},
	}

	manager, err := autopaho.NewConnection(ctx, cfg)
	if err != nil {
		return fmt.Errorf("create connection manager: %w", err)
	}
	defer func() {
		shutdown, cancel := context.WithTimeout(context.Background(), 5*time.Second)
		defer cancel()
		if err := manager.Disconnect(shutdown); err != nil {
			log.Printf("shutdown did not finish within deadline: %v", err)
		}
	}()

	startup, cancel := context.WithTimeout(ctx, 10*time.Second)
	err = manager.AwaitConnection(startup)
	cancel()
	if err != nil {
		return fmt.Errorf("initial connection: %w", err)
	}
	log.Print("connection ready; press Ctrl+C to stop")
	<-ctx.Done()
	return nil
}

🔌 2. Understand the connection lifecycle

💡 Story question: What is happening between creating the manager and logging “connection ready,” and why are those moments not equivalent?

Trace the background attempt through transport connection, MQTT CONNECT, broker authentication, CONNACK, and AwaitConnection. Then trace what AutoPaho does when the connection drops.

Before reading the full program, predict what evidence you need at each stage:

Go process → AutoPaho retry loop → TCP connection → CONNECT → CONNACK
                                                        → subscriptions (Lesson 4)

AutoPaho owns reconnect attempts and creates the underlying Paho client for a connection. Paho implements MQTT packets and protocol exchanges. Your application owns readiness, authorization policy, command state, and business retries. A running goroutine or open TCP socket is too early to declare MQTT ready: the broker can still reject CONNECT.

Setting or call	MQTT meaning and reason for this choice
`ServerUrls`	Where to dial; the lab host endpoint is different from the Compose service endpoint
`ClientID`	Broker session identity; two simultaneous processes with this ID compete for one connection
`ConnectUsername`, `ConnectPassword`	CONNECT authentication inputs; independent of Client ID and payload identity
`KeepAlive: 20`	Maximum idle interval between client control packets, in seconds; Paho handles PINGREQ/PINGRESP when needed
`CleanStartOnInitialConnection: true`	Discard a previous broker session on the manager's first successful connection
`SessionExpiryInterval: 0`	Do not retain this broker session after the connection closes
`NewConnection`	Start connection management; a nil error does not establish a successful CONNACK
`AwaitConnection`	Wait for connection availability; it does not wait for SUBACK or a device heartbeat
`OnConnectionUp`	Observe a successful connection, including reconnection; inspect CONNACK evidence here
`OnConnectError`	Observe failed attempts, including refused connections or unsuccessful CONNACK
`OnConnectionDown`	Decide whether reconnect attempts should continue; returning false ends that retry lifecycle
`OnServerDisconnect`	Inspect a DISCONNECT actually sent by the broker; a severed cable cannot send this packet

Keep Alive checks the MQTT connection, not the health of the vending application. With an effective Keep Alive of 20 seconds, the broker's protocol inactivity limit is 30 seconds without a client control packet; an already-detected closed socket can be handled earlier. A CONNACK Server Keep Alive can override the requested value. Your three-second heartbeat in Lesson 4 carries application evidence and has a separate purpose.

There are three different contexts here. The signal context controls the program lifetime. startup limits the initial wait to ten seconds. shutdown is a fresh context because the signal context is already canceled during shutdown. The manager receives the long-lived context, not the startup deadline; passing the startup context to NewConnection would terminate your service when that deadline expires.

CleanStartOnInitialConnection: true and SessionExpiryInterval: 0 choose a disposable early-lab session. This is convenient for first experiments and intentionally offers no offline broker session after disconnect. Later lessons change these settings and test recovery. Do not copy them as a durable production policy.

Paho v0.23.0's Disconnect(ctx) cancels the manager and waits for manager shutdown with the provided deadline. Its Done() documentation notes that underlying MQTT cleanup may still be completing; neither API is proof that application data was persisted or device work was finished.

▶️ 3. Run it

💡 Story question: Which output tells you that the broker accepted the client, and what does a clean Ctrl+C demonstrate about lifecycle ownership?

Run the process with valid lab credentials, observe the acceptance evidence, then stop it and compare graceful disconnect with a process that simply disappears.

The backend reads its username and password from configs/lab.toml:

go run ./cmd/backend

go run ./cmd/backend compiles the package and runs it in the foreground. ./ means relative to the project root. Ctrl+C sends an interrupt to the foreground program; signal.NotifyContext turns it into context cancellation, allowing your cleanup code to run.

The TOML file contains the disposable local lab password. Use only lab credentials. Expect connected and connection ready; callback log order can vary. Press Ctrl+C and verify exit within the shutdown deadline.

Run syntax checks after creating the file:

gofmt -w cmd/backend/main.go
go build ./cmd/backend
go vet ./cmd/backend

gofmt -w rewrites the named source file using Go formatting. go build checks compilation and linking; go vet checks suspicious code patterns without running the application. A successful vet run is normally quiet.

go build emits a binary in the current directory; remove or ignore it as appropriate. A build verifies syntax/types, not connection behavior.

⚠️ 4. Observe the protocol and failures

💡 Story question: How can you tell whether a failure happened before MQTT, during broker authentication, or after an established connection was lost?

Change one condition at a time and compare the connection callbacks, reason information, retry behavior, and shutdown outcome.

CONNECT carries your client ID, authentication fields, keepalive, and session properties. A successful CONNACK is connection evidence. Bad credentials produce an unsuccessful connection attempt; the rejected CONNACK is handled by the connection-error path rather than your success callback.

Stop the broker before running the program. Expect retry diagnostics and initial-wait failure near ten seconds, followed by cleanup.
Start the broker; run the program; stop and restart the broker. Expect connection-down and connection-up events. You have not restored subscriptions yet because this program has none.
Supply a wrong password. Explain why retrying cannot repair an incorrect credential.
Run two copies using the same client ID. Observe connection takeover/reconnect behavior, then stop both. Unique IDs are necessary for concurrently running clients.
Press Ctrl+C while the broker is unavailable. Cancellation should interrupt the wait; do not wait for the full startup deadline.

🔎 Cautions and troubleshooting

AutoPaho handles connection management; it does not invent topic authorization, business retries, or durable command storage.
Do not call log.Fatal from callbacks. It terminates the process without running deferred cleanup.
Callbacks should be short. Never add an indefinite channel send, database operation, or sleep to OnConnectionDown.
Keep client IDs stable when you later want session resumption, but distinct across different simultaneously running clients.
Logging entire config structs or URL userinfo can reveal passwords. These examples keep credentials separate from the URL.
A connection-up event is not subscription readiness, and neither is device readiness.

Completion: Explain every field in cfg, distinguish the three contexts, and demonstrate successful startup, failed startup, reconnect, and cancellation.

✅ Result — the story so far

The backend now joins the broker as an authenticated MQTT 5 client. AutoPaho starts and manages the connection; the application waits until CONNACK confirms the broker accepted it, then keeps running and retries after a dropped connection. The backend has no subscriptions yet, so it is connected but is not listening for device messages.

❓ Questions to answer before Section 3

Why can NewConnection return before a usable broker connection exists?
What would happen if you passed startup to NewConnection?
Why does shutdown use context.Background() as its parent?
What does returning false from OnConnectionDown change?

Check your reasoning after answering

NewConnection starts a background lifecycle; CONNACK is later evidence. A startup context passed as the manager lifetime would stop the manager when that context expires or is canceled. Shutdown needs a fresh bounded context because the signal context is already canceled. Returning false from OnConnectionDown stops further automatic reconnect attempts for that manager; it does not repair the connection or preserve application state.

🔗 Sources

Next: Section 3.

14 KiB Raw Blame History