Refactor MQTT course and bound backend message processing

Move lesson source into the teaching structure, align the course path with the learner's Go and DevOps experience, and add the interactive lesson guides. Add advanced DevOps lifecycle study as a separate section.

Bound MQTT message work and drain accepted jobs during shutdown; align the backend implementation with the revised course through Section 6.

teaching/lessons/0016-mqtt5-controls.html

@@ -0,0 +1,100 @@
+<!doctype html>
+<html lang="en"><head>
+  <meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>Lesson 16 — Advanced MQTT 5 Controls</title>
+  <link rel="stylesheet" href="../assets/lesson.css"><script src="../assets/quiz.js" defer></script>
+</head><body><main class="page">
+  <p class="eyebrow">Go MQTT Vending Backend · Lesson 16 · 15 minutes</p>
+  <h1>Choose MQTT controls from measured behavior</h1>
+  <p class="dek">Experiment with subscription options, shared consumers, flow limits, packet sizes, aliases, and reason codes without confusing protocol controls with application policy.</p>
+  <div class="meta"><span class="pill">🎯 One skill: measure before adopting a feature</span><span class="pill">🧠 Predict packet effects</span><span class="pill">🧪 Version-pinned lab</span></div>
+  <section class="card callout"><p class="question">❓ Start with the system story</p><p>Two ingestion processes share a subscription. A command response reaches the instance that did not create the original process-local waiter. Which state design still lets that instance process the result correctly?</p><p>Answer with the shared durable source of truth, then explore the protocol controls.</p></section>
+  <h2>📬 Subscription options shape delivery</h2>
+  <table><thead><tr><th>Option</th><th>Changes</th><th>Does not do</th></tr></thead><tbody><tr><td>No Local</td><td>Suppresses same-client-originated publications for that subscription</td><td>Authorize a topic or filter a device identity</td></tr><tr><td>Retain As Published</td><td>Preserves incoming RETAIN bit on forwarded live traffic</td><td>Choose whether broker stores retained state</td></tr><tr><td>Retain Handling</td><td>Controls retained replay when subscription is established</td><td>Block future live retained publications</td></tr><tr><td>UNSUBSCRIBE</td><td>Removes the broker subscription after UNSUBACK</td><td>Erase retained state or unregister a local Go handler</td></tr></tbody></table>
+  <p>Change one option at a time with a retained status topic. Record whether the publication was a retained replay or live delivery and inspect the packet flags. Never set No Local as a substitute for ACLs.</p>
+  <form class="card quiz" data-quiz data-correct="a" data-explanation="Shared consumers distribute publications, but any instance may receive one; durable shared command state and idempotent result processing are needed instead of local waiter affinity." data-retry="A shared subscription doesn't route a response back to the instance that created an in-memory waiter."><p><strong>🧠 Check scaling behavior:</strong> What makes a shared-subscription response safe when it reaches a different ingestion instance?</p><label><input type="radio" name="q" value="a"> Persisted command state and durable idempotency shared by the instances</label><br><label><input type="radio" name="q" value="b"> A larger process-local waiter map</label><br><label><input type="radio" name="q" value="c"> Assuming per-device ordering and instance affinity</label><br><button type="submit">Check answer</button><p data-feedback aria-live="polite"></p></form>
+  <h2>⚖️ Shared subscriptions and limits</h2>
+  <p>Run two ingestion instances in one shared group. Observe distribution and what happens when one stops. Do not assume per-device ordering or affinity; both workers must use durable idempotency. MQTT Receive Maximum limits unacknowledged QoS 1/2 protocol exchanges in a direction. It is not your Go queue capacity, and it does not limit QoS 0 in the same way.</p>
+  <p>Advertised Maximum Packet Size limits packet size in the direction advertised. Test client-to-broker and broker-to-client separately. Compare protocol inflight count with application queue depth; they can differ sharply if callbacks acknowledge before queued work is durable.</p>
+  <h2>🏷️ Aliases, reason codes, and support matrix</h2>
+  <p>Topic Alias mappings are scoped to a network connection. A reconnect needs a new mapping even if the MQTT session resumes. Confirm the pinned Paho and Mosquitto support and lifecycle reset behavior before treating aliases as an optimization; a TopicAlias field existing in a type is not evidence that a full flow works.</p>
+  <p>Interpret reason codes in their packet context. PUBACK “No matching subscribers” is not a device response. A SUBACK result applies to its filter. A context deadline means the local wait ended and may leave business outcome uncertain. Use bounded reconnect backoff with jitter for transient failures; repair authentication/configuration errors instead of retrying faster.</p>
+  <h2>🧭 File overview: <code>internal/mqtt/controls.go</code></h2>
+  <p>Create this complete helper file to build MQTT 5 CONNECT and SUBSCRIBE options without mixing broker protocol limits with local queue capacity. The caller still inspects actual CONNACK/SUBACK results and checks version-matched Paho support.</p>
+  <h3>🧰 <code>NewConnectProperties</code> — short summary</h3><p>Builds validated client Receive Maximum and Maximum Packet Size properties.</p>
+  <h3>📬 <code>NewSubscription</code> — short summary</h3><p>Builds one subscription and rejects invalid QoS, Retain Handling, and No Local on a shared filter.</p>
+  <h3>🔎 <code>CapabilitiesFromConnack</code> — short summary</h3><p>Copies optional server capability values safely, preserving nil as “not advertised.”</p>
+  <details class="card" open><summary><strong>📄 Complete file contents</strong></summary><pre><code>package mqtt
+
+import (
+	"fmt"
+	"strings"
+
+	"example.com/mqtt5-learning/internal/contracts"
+	"github.com/eclipse/paho.golang/paho"
+)
+
+type ServerCapabilities struct {
+	SessionPresent bool
+	MaximumQoS *byte
+	ReceiveMaximum *uint16
+	MaximumPacketSize *uint32
+	TopicAliasMaximum *uint16
+	SharedSubscriptions bool
+	RetainedMessages bool
+}
+
+func NewConnectProperties(receiveMaximum uint16, maximumPacketSize uint32) (*paho.ConnectProperties, error) {
+	if receiveMaximum == 0 { return nil, fmt.Errorf("receive maximum must be at least one") }
+	if maximumPacketSize == 0 { return nil, fmt.Errorf("maximum packet size must be at least one byte") }
+	packetLimit := maximumPacketSize
+	flightLimit := receiveMaximum
+	return &amp;paho.ConnectProperties{ReceiveMaximum: &amp;flightLimit, MaximumPacketSize: &amp;packetLimit}, nil
+}
+
+func NewSubscription(filter contracts.TopicFilter, qos, retainHandling byte, noLocal, retainAsPublished bool) (paho.SubscribeOptions, error) {
+	value := filter.String()
+	if value == "" { return paho.SubscribeOptions{}, fmt.Errorf("subscription filter is required") }
+	if qos &gt; 2 { return paho.SubscribeOptions{}, fmt.Errorf("QoS must be 0, 1, or 2") }
+	if retainHandling &gt; 2 { return paho.SubscribeOptions{}, fmt.Errorf("Retain Handling must be 0, 1, or 2") }
+	if noLocal &amp;&amp; strings.HasPrefix(value, "$share/") {
+		return paho.SubscribeOptions{}, fmt.Errorf("No Local cannot be used with a shared subscription")
+	}
+	return paho.SubscribeOptions{
+		Topic: value, QoS: qos, RetainHandling: retainHandling,
+		NoLocal: noLocal, RetainAsPublished: retainAsPublished,
+	}, nil
+}
+
+func CapabilitiesFromConnack(ack *paho.Connack) ServerCapabilities {
+	if ack == nil { return ServerCapabilities{} }
+	result := ServerCapabilities{SessionPresent: ack.SessionPresent}
+	if ack.Properties == nil { return result }
+	result.MaximumQoS = copyByte(ack.Properties.MaximumQoS)
+	result.ReceiveMaximum = copyUint16(ack.Properties.ReceiveMaximum)
+	result.MaximumPacketSize = copyUint32(ack.Properties.MaximumPacketSize)
+	result.TopicAliasMaximum = copyUint16(ack.Properties.TopicAliasMaximum)
+	result.SharedSubscriptions = ack.Properties.SharedSubAvailable
+	result.RetainedMessages = ack.Properties.RetainAvailable
+	return result
+}
+
+func copyByte(value *byte) *byte {
+	if value == nil { return nil }; copy := *value; return &amp;copy
+}
+func copyUint16(value *uint16) *uint16 {
+	if value == nil { return nil }; copy := *value; return &amp;copy
+}
+func copyUint32(value *uint32) *uint32 {
+	if value == nil { return nil }; copy := *value; return &amp;copy
+}
+</code></pre></details>
+  <div class="card"><h3>🔎 Details: limits and capability defaults</h3><p>The helper does not resize your application queue or guarantee that the broker supports a requested behavior. The server's absent optional numeric properties stay nil; callers must apply the MQTT-defined defaults for the relevant feature. This small struct records the fields needed by the experiment and does not claim to capture every negotiated capability. Topic Alias state must be rebuilt after a new network connection.</p></div>
+  <h2>🧪 Build a feature decision matrix</h2>
+  <ol><li>Record Paho and broker versions and the exact option/API field.</li><li>For each experiment record purpose, packet and direction, prediction, actual observation, and whether you would use the feature.</li><li>Test No Local, Retain As Published, and each Retain Handling value with stored and live messages.</li><li>Run two shared consumers; stop one during traffic and inspect distribution, duplicates, and result processing.</li><li>Test Receive Maximum and packet-size negotiation in both directions. Measure protocol inflight and application queue separately.</li><li>Review Topic Alias and reason-code paths against the pinned source; mark reading-only items as not experimentally verified.</li></ol>
+  <div class="card"><h3>✅ Done when</h3><p>Your matrix distinguishes MQTT specification, broker, client-library, and application behavior. Every feature has evidence or is marked unverified; no optimization is selected just because the API exposes a field.</p></div>
+  <details class="card"><summary><strong>📋 Short self-check answers</strong></summary><ol><li>Retain Handling controls replay at subscribe time; Retain As Published controls a flag on forwarded live traffic.</li><li>Shared subscriptions distribute work and do not guarantee affinity; durable state lets any instance process safely.</li><li>Receive Maximum limits in-flight QoS 1/2 packets, not an application's total queue.</li><li>Aliases need to be re-established after reconnect.</li></ol></details>
+  <p>📚 Detailed experiments and Paho v0.23.0 source pointers: <a href="../course-source/16-mqtt5-controls.md">Lesson 16 source lesson</a>. Primary references: <a href="https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html">OASIS MQTT 5.0 Standard</a> and <a href="https://pkg.go.dev/github.com/eclipse/paho.golang/paho@v0.23.0">Paho v0.23.0 API</a>.</p>
+  <p>Ask me to map an observed control or reason code to its packet and owning layer.</p>
+  <footer><p>Previous: <a href="0015-process-boundaries.html">Lesson 15 — process boundaries</a> · Next: <a href="0017-operations-diagnostics.html">Lesson 17 — operate and diagnose</a>.</p></footer>
+</main></body></html>