Refactor MQTT course and bound backend message processing
Move lesson source into the teaching structure, align the course path with the learner's Go and DevOps experience, and add the interactive lesson guides. Add advanced DevOps lifecycle study as a separate section. Bound MQTT message work and drain accepted jobs during shutdown; align the backend implementation with the revised course through Section 6.
This commit is contained in:
@@ -0,0 +1,745 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Lesson 7 — Match Command Results to the Right Request</title>
|
||||
<link rel="stylesheet" href="../assets/lesson.css"><script src="../assets/quiz.js" defer></script>
|
||||
</head>
|
||||
<body><main class="page">
|
||||
<p class="eyebrow">Go MQTT Vending Backend · Lesson 7 · 8 minutes</p>
|
||||
<h1>Match command results to the right request</h1>
|
||||
<p class="dek">Keep broker acknowledgement, device result, and timeout uncertainty separate while several vending commands are in flight.</p>
|
||||
<div class="meta"><span class="pill">🎯 One skill: correlate safely</span><span class="pill">🧠 Predict a fast response</span><span class="pill">🧪 Simulated device</span></div>
|
||||
|
||||
<section class="card callout"><p class="question">❓ Start with the story</p><p>The backend publishes command <code>C-42</code>. PUBACK arrives, but no device result arrives before the deadline. Can the backend say the command failed or that the machine did nothing?</p><p>State what is known, what is unknown, and which ID can tie a later result to this command.</p></section>
|
||||
<section class="card"><h2>📍 Checkout status</h2><p>The current codebase does not yet contain <code>internal/commands</code> or <code>internal/simulator</code>, and its <code>mqtt.Message</code> has no Response Topic or Correlation Data fields. Section 7 creates those files and extends the MQTT message copy; complete source files are included below.</p><p><strong>Durability boundary:</strong> command records and simulator deduplication are process-local. Restarting either process loses its in-memory state. Sections 13–14 add durable records and recoverable publication intent.</p></section>
|
||||
<h2>🗺️ File overview: command state outlives a packet</h2>
|
||||
<p><code>internal/commands/store.go</code> owns command records and state transitions. <code>service.go</code> registers state before publishing and records timeouts. The MQTT adapter carries Response Topic and Correlation Data; the simulator checks its reply destination and suppresses repeated effects by command ID.</p>
|
||||
<div class="flow" aria-label="command result lifecycle"><div class="step">Create stable ID<br><strong>pending record</strong></div><span class="arrow">→</span><div class="step">Publish MQTT<br><strong>PUBACK evidence</strong></div><span class="arrow">→</span><div class="step">Match result<br><strong>device evidence</strong></div><span class="arrow">→</span><div class="step">No result by deadline<br><strong>unknown</strong></div></div>
|
||||
|
||||
<article class="card"><h3>🚀 `CreateAndPublish` — short summary</h3><p>Creates correlation bytes, stores pending state, publishes with a deadline, then records broker acknowledgement separately.</p><h3>🧩 Details: insert before publish</h3><pre><code>record := Record{
|
||||
CommandID: commandID,
|
||||
DeviceID: deviceID,
|
||||
CorrelationData: correlation,
|
||||
Status: Pending,
|
||||
Deadline: now.Add(timeout),
|
||||
}
|
||||
if err := store.Insert(record); err != nil {
|
||||
return Record{}, err
|
||||
}
|
||||
|
||||
// Only after pending state exists:
|
||||
replyTopic, err := contracts.DeviceTopic(deviceID, contracts.KindResponses)
|
||||
if err != nil { return Record{}, err }
|
||||
reason, err := publisher.PublishCommand(ctx, deviceID, command, replyTopic, correlation)
|
||||
if err != nil {
|
||||
return store.MarkUnknownIfPending(deviceID, commandID, "publish outcome uncertain")
|
||||
}
|
||||
return store.MarkBrokerAck(deviceID, commandID, reason)</code></pre><p>A device can reply before the publish call returns, so pending state must exist first. If the publish call times out, the backend may not know whether the broker accepted it; mark the outcome unknown rather than asserting no effect. A late response still needs an explicit policy.</p></article>
|
||||
|
||||
<h2>🪪 Three identifiers have different jobs</h2>
|
||||
<article class="card"><ul><li><strong>MQTT packet ID:</strong> protocol bookkeeping for an exchange; it can be reused.</li><li><strong>Correlation Data:</strong> opaque bytes to match one response to one pending request; it is not authorization.</li><li><strong>Application command ID:</strong> stable business identity across retries; it supports device-side deduplication and status lookup.</li></ul><p>The responder must also check that the Response Topic is authorized for that device. Matching identity is not proof of identity; credentials and broker ACLs remain separate controls.</p></article>
|
||||
<h2>🧭 Keep the state meanings honest</h2>
|
||||
<table><thead><tr><th>Evidence or event</th><th>Safe interpretation</th></tr></thead><tbody><tr><td>Created and stored</td><td>Pending publish attempt</td></tr><tr><td>Successful PUBACK</td><td>Broker acknowledged this publish hop</td></tr><tr><td>Validated device result</td><td>Apply accepted, completed, or rejected evidence</td></tr><tr><td>Deadline with no result</td><td>Unknown outcome; the effect may or may not have happened</td></tr></tbody></table>
|
||||
<p>Conceptually, the lifecycle is <code>created → pending delivery → accepted → completed/rejected</code>; a deadline moves unresolved work to <code>unknown</code>. A valid late result may resolve <code>unknown</code> here, while <code>completed</code> and <code>rejected</code> remain terminal. The reference store inserts directly as <code>Pending</code> before PUBLISH, so creation is not a separate queryable status. <code>BrokerAcked</code> is orthogonal to application status.</p>
|
||||
|
||||
<h2>🧭 File and function map</h2>
|
||||
<p><strong>File overview:</strong> <code>commands/store.go</code> owns in-memory records and transition checks; <code>commands/service.go</code> coordinates state and publishing; <code>mqtt/command_publisher.go</code> translates a command into an MQTT 5 packet; <code>commands/receive.go</code> validates incoming results. The simulator files validate, deduplicate, and answer commands. The modified <code>mqtt/client.go</code> copies MQTT 5 response properties into the app-owned message.</p>
|
||||
<div class="two-col">
|
||||
<article class="card"><h3>📦 <code>internal/commands/store.go</code></h3>
|
||||
<p><strong>Function summaries:</strong> <code>NewStore</code> creates the table; <code>Insert</code>, <code>Get</code>, <code>Update</code>, and <code>List</code> manage records; <code>MarkBrokerAck</code> records transport evidence; <code>MarkUnknownIfPending</code> records uncertainty; <code>ApplyResult</code> matches device evidence.</p>
|
||||
<h4>🔎 Details</h4><p>Keys include both device and command ID so equal IDs from different devices do not collide. The lock protects concurrent HTTP, MQTT, and timeout activity. Correlation bytes are copied at storage boundaries. <code>ApplyResult</code> checks device scope and correlation before changing state, and preserves terminal <code>Completed</code>/<code>Rejected</code> results against duplicates. Broker acknowledgement remains an independent field, not a command status.</p>
|
||||
</article>
|
||||
<article class="card"><h3>🧠 <code>internal/commands/service.go</code></h3>
|
||||
<p><strong>Function summaries:</strong> <code>NewService</code> wires the store, publisher, clock, and response timeout; <code>CreateAndPublish</code> stores pending state before MQTT publish; <code>HandleResult</code> delegates validated result matching; <code>Expire</code> marks overdue work unknown; <code>Get</code> reads current state.</p>
|
||||
<h4>🔎 Details</h4><p>The pending insert must happen before publishing because a fast simulator can answer before <code>PublishCommand</code> returns. A publish timeout can mean “ack not observed,” not “broker definitely did not accept it,” so the service marks the outcome unknown. <code>Expire</code> uses the same uncertainty meaning and never reports that the physical action did not occur. In this teaching version, the clock and records are process-local.</p>
|
||||
</article>
|
||||
<article class="card"><h3>📡 MQTT adapter and result receiver</h3>
|
||||
<p><strong><code>PublishCommand</code> summary:</strong> validates the device and reply destination, encodes the contract, publishes QoS 1 with Response Topic and Correlation Data, and returns the broker reason code.</p>
|
||||
<p><strong><code>HandleMessage</code> summary:</strong> parses the response route, decodes a <code>CommandResult</code>, and passes copied correlation bytes to the service.</p>
|
||||
<p><strong><code>mqtt.Start</code> summary:</strong> the Section 7 target copies Response Topic and Correlation Data inside the callback, while the packet properties are still available.</p>
|
||||
<h4>🔎 Details</h4><p>The adapter verifies that the reply topic belongs to the target device before publishing. A response is not accepted just because it arrived on a response-shaped topic: the contract must validate and the store must match its device and correlation to a pending record. Correlation matches a request; broker ACLs and device credentials authorize who can publish.</p>
|
||||
</article>
|
||||
<article class="card"><h3>🤖 Simulator files</h3>
|
||||
<p><strong><code>AuthorizedResponseTopic</code> summary:</strong> permits only that device's response topic. <strong><code>Deduplicator.Apply</code> summary:</strong> runs an effect once per device/command ID while its bounded cache retains the result. <strong><code>Respond</code> summary:</strong> validates route, payload, retained flag, and response destination, then publishes the result with the original correlation bytes. <strong><code>run</code> summary:</strong> owns MQTT startup, heartbeat publication, command receive, and cancellation.</p>
|
||||
<h4>🔎 Details</h4><p>The cache fingerprints command content. Reusing an ID with changed content is an error; repeating identical content replays the saved result without repeating the effect. Because the cache is in memory and bounded, this is a simulator guarantee only while the entry remains cached and the process stays alive—not production durable exactly-once execution.</p>
|
||||
</article>
|
||||
</div>
|
||||
|
||||
<h2>🧪 Check your model</h2>
|
||||
<form class="card quiz" data-quiz data-correct="c" data-explanation="Correct. PUBACK is transport evidence for the publication exchange; it does not establish device delivery or business completion." data-retry="Which component sends PUBACK, and which component creates an application result?"><fieldset class="question-block"><legend><strong>1. What can the backend conclude from PUBACK?</strong></legend><label class="option"><input type="radio" name="q1" value="a"> The device accepted the vending command for execution</label><label class="option"><input type="radio" name="q1" value="b"> The device completed the vending command successfully now</label><label class="option"><input type="radio" name="q1" value="c"> The broker acknowledged this client's publication hop only</label></fieldset><button type="submit">Check answer</button><p class="feedback" data-feedback aria-live="polite"></p></form>
|
||||
<form class="card quiz" data-quiz data-correct="a" data-explanation="Yes. No response means the backend lacks a device outcome. A timeout records uncertainty; it does not prove non-execution." data-retry="What does the lack of a response say about a physical effect that may have happened?"><fieldset class="question-block"><legend><strong>2. What should timeout mean without a response?</strong></legend><label class="option"><input type="radio" name="q2" value="a"> Backend outcome is unknown; device work may have happened</label><label class="option"><input type="radio" name="q2" value="b"> The command definitely never reached the vending device now</label><label class="option"><input type="radio" name="q2" value="c"> The broker can never undo any command delivered already</label></fieldset><button type="submit">Check answer</button><p class="feedback" data-feedback aria-live="polite"></p></form>
|
||||
<form class="card quiz" data-quiz data-correct="b" data-explanation="Correct. Correlation matches a response to a request. Topic authorization and device identity need separate checks." data-retry="Can matching bytes prove a device is allowed to publish to a chosen response topic?"><fieldset class="question-block"><legend><strong>3. What does Correlation Data not provide?</strong></legend><label class="option"><input type="radio" name="q3" value="a"> A match between one response and one pending request</label><label class="option"><input type="radio" name="q3" value="b"> Authorization for the response topic chosen by device itself</label><label class="option"><input type="radio" name="q3" value="c"> Opaque request metadata copied into the result packet unchanged</label></fieldset><button type="submit">Check answer</button><p class="feedback" data-feedback aria-live="polite"></p></form>
|
||||
|
||||
<h2>🛠️ Your tiny practice</h2><div class="card"><p>Trace this race: pending record inserted → PUBLISH begins → device responds → PUBACK returns. Which state fields change at each step? Then let a command time out, deliver a valid late result with matching device and correlation, and verify the reference policy resolves <code>unknown</code> without regressing <code>completed</code> or <code>rejected</code>. Finally send the same command ID twice and explain which component prevents the simulated effect from running twice.</p><p>Compare with <a href="../course-source/07-command-response.md">Section 7's complete command store and simulator code ↗</a>.</p></div>
|
||||
|
||||
|
||||
<details class="card" open><summary><strong>📦 Section 7 complete command and simulator files</strong></summary><p>Complete source files for this lesson are included here. The code blocks earlier in the lesson focus on the behavior being taught; use these full files when implementing it.</p><article class="card"><h3>📄 internal/commands/store.go</h3><pre><code>package commands
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"example.com/mqtt5-learning/internal/contracts"
|
||||
)
|
||||
|
||||
type Status string
|
||||
|
||||
// Status records application progress independently from BrokerAcked.
|
||||
const (
|
||||
Pending Status = "pending"
|
||||
Accepted Status = "accepted"
|
||||
Completed Status = "completed"
|
||||
Rejected Status = "rejected"
|
||||
Unknown Status = "unknown"
|
||||
)
|
||||
|
||||
type Record struct {
|
||||
CommandID string
|
||||
DeviceID string
|
||||
CorrelationData []byte
|
||||
CreatedAt time.Time
|
||||
Deadline time.Time
|
||||
Status Status
|
||||
BrokerAcked bool
|
||||
AckReasonCode byte
|
||||
Detail string
|
||||
}
|
||||
|
||||
type Store struct {
|
||||
mu sync.RWMutex
|
||||
records map[string]Record
|
||||
}
|
||||
|
||||
// NewStore creates the in-memory command record table.
|
||||
func NewStore() *Store { return &Store{records: make(map[string]Record)} }
|
||||
|
||||
func key(deviceID, commandID string) string { return deviceID + "/" + commandID }
|
||||
|
||||
func (s *Store) Insert(record Record) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
k := key(record.DeviceID, record.CommandID)
|
||||
if _, exists := s.records[k]; exists {
|
||||
return fmt.Errorf("command already exists")
|
||||
}
|
||||
record.CorrelationData = append([]byte(nil), record.CorrelationData...)
|
||||
s.records[k] = record
|
||||
return nil
|
||||
}
|
||||
|
||||
// Get and Update copy correlation bytes so callers cannot mutate stored state.
|
||||
func (s *Store) Get(deviceID, commandID string) (Record, bool) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
record, ok := s.records[key(deviceID, commandID)]
|
||||
record.CorrelationData = append([]byte(nil), record.CorrelationData...)
|
||||
return record, ok
|
||||
}
|
||||
|
||||
func (s *Store) Update(record Record) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
k := key(record.DeviceID, record.CommandID)
|
||||
if _, exists := s.records[k]; !exists {
|
||||
return fmt.Errorf("command not found")
|
||||
}
|
||||
record.CorrelationData = append([]byte(nil), record.CorrelationData...)
|
||||
s.records[k] = record
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) MarkBrokerAck(deviceID, commandID string, reason byte) (Record, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
k := key(deviceID, commandID)
|
||||
record, ok := s.records[k]
|
||||
if !ok {
|
||||
return Record{}, fmt.Errorf("command not found")
|
||||
}
|
||||
record.BrokerAcked, record.AckReasonCode = true, reason
|
||||
s.records[k] = record
|
||||
return record, nil
|
||||
}
|
||||
|
||||
// MarkUnknownIfPending preserves any application result that arrived first.
|
||||
func (s *Store) MarkUnknownIfPending(deviceID, commandID, detail string) (Record, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
k := key(deviceID, commandID)
|
||||
record, ok := s.records[k]
|
||||
if !ok {
|
||||
return Record{}, fmt.Errorf("command not found")
|
||||
}
|
||||
if record.Status == Pending || record.Status == Accepted {
|
||||
record.Status, record.Detail = Unknown, detail
|
||||
}
|
||||
s.records[k] = record
|
||||
return record, nil
|
||||
}
|
||||
|
||||
// ApplyResult checks both device scope and correlation before changing status.
|
||||
func (s *Store) ApplyResult(deviceID string, correlation []byte, result contracts.CommandResult) (Record, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
k := key(deviceID, result.CommandID)
|
||||
record, ok := s.records[k]
|
||||
if !ok {
|
||||
return Record{}, fmt.Errorf("no pending command matches result")
|
||||
}
|
||||
if result.DeviceID != record.DeviceID || !bytes.Equal(correlation, record.CorrelationData) {
|
||||
return Record{}, fmt.Errorf("result identity or correlation does not match pending command")
|
||||
}
|
||||
if record.Status == Completed || record.Status == Rejected {
|
||||
return record, nil
|
||||
}
|
||||
switch result.Status {
|
||||
case "accepted":
|
||||
record.Status = Accepted
|
||||
case "completed":
|
||||
record.Status = Completed
|
||||
case "rejected":
|
||||
record.Status = Rejected
|
||||
default:
|
||||
return Record{}, fmt.Errorf("unsupported result status %q", result.Status)
|
||||
}
|
||||
record.Detail = result.Detail
|
||||
s.records[k] = record
|
||||
return record, nil
|
||||
}
|
||||
|
||||
func (s *Store) List() []Record {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
result := make([]Record, 0, len(s.records))
|
||||
for _, record := range s.records {
|
||||
record.CorrelationData = append([]byte(nil), record.CorrelationData...)
|
||||
result = append(result, record)
|
||||
}
|
||||
return result
|
||||
}
|
||||
</code></pre></article>
|
||||
<article class="card"><h3>📄 internal/commands/service.go</h3><pre><code>package commands
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"example.com/mqtt5-learning/internal/contracts"
|
||||
)
|
||||
|
||||
type Publisher interface {
|
||||
PublishCommand(context.Context, string, contracts.Command, contracts.TopicName, []byte) (byte, error)
|
||||
}
|
||||
|
||||
// Service coordinates command identity, storage, publication, and deadlines.
|
||||
type Service struct {
|
||||
store *Store
|
||||
publisher Publisher
|
||||
now func() time.Time
|
||||
timeout time.Duration
|
||||
}
|
||||
|
||||
func NewService(store *Store, publisher Publisher, timeout time.Duration) *Service {
|
||||
return &Service{store: store, publisher: publisher, now: func() time.Time { return time.Now().UTC() }, timeout: timeout}
|
||||
}
|
||||
|
||||
func (s *Service) CreateAndPublish(ctx context.Context, deviceID, commandID string) (Record, error) {
|
||||
responseTopic, err := contracts.DeviceTopic(deviceID, contracts.KindResponses)
|
||||
if err != nil { return Record{}, fmt.Errorf("build response topic: %w", err) }
|
||||
// Store pending state first so a fast device response can match immediately.
|
||||
now := s.now()
|
||||
correlation := make([]byte, 16)
|
||||
if _, err := rand.Read(correlation); err != nil {
|
||||
return Record{}, fmt.Errorf("create correlation data: %w", err)
|
||||
}
|
||||
record := Record{CommandID: commandID, DeviceID: deviceID, CorrelationData: correlation, CreatedAt: now, Deadline: now.Add(s.timeout), Status: Pending}
|
||||
if err := s.store.Insert(record); err != nil {
|
||||
return Record{}, err
|
||||
}
|
||||
command := contracts.Command{SchemaVersion: contracts.SchemaVersion, MessageID: "message-" + commandID, DeviceID: deviceID, OccurredAt: now.Format(time.RFC3339), CommandID: commandID, Command: "ping"}
|
||||
ctx, cancel := context.WithDeadline(ctx, record.Deadline)
|
||||
defer cancel()
|
||||
reason, err := s.publisher.PublishCommand(ctx, deviceID, command, responseTopic, append([]byte(nil), correlation...))
|
||||
if err != nil {
|
||||
updated, _ := s.store.MarkUnknownIfPending(deviceID, commandID, "publish outcome uncertain: "+err.Error())
|
||||
return updated, err
|
||||
}
|
||||
updated, err := s.store.MarkBrokerAck(deviceID, commandID, reason)
|
||||
return updated, err
|
||||
}
|
||||
|
||||
func (s *Service) HandleResult(deviceID string, correlation []byte, result contracts.CommandResult) (Record, error) {
|
||||
return s.store.ApplyResult(deviceID, correlation, result)
|
||||
}
|
||||
|
||||
// Expire records uncertainty; it does not claim that the device did no work.
|
||||
func (s *Service) Expire(now time.Time) {
|
||||
for _, record := range s.store.List() {
|
||||
if (record.Status == Pending || record.Status == Accepted) && !now.Before(record.Deadline) {
|
||||
_, _ = s.store.MarkUnknownIfPending(record.DeviceID, record.CommandID, "response deadline expired; execution outcome is unknown")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) Get(commandID string) (Record, bool) {
|
||||
for _, record := range s.store.List() {
|
||||
if record.CommandID == commandID {
|
||||
return record, true
|
||||
}
|
||||
}
|
||||
return Record{}, false
|
||||
}
|
||||
</code></pre></article>
|
||||
<article class="card"><h3>📄 internal/mqtt/command_publisher.go</h3><pre><code>package mqtt
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"example.com/mqtt5-learning/internal/contracts"
|
||||
"github.com/eclipse/paho.golang/autopaho"
|
||||
"github.com/eclipse/paho.golang/paho"
|
||||
)
|
||||
|
||||
type CommandPublisher struct {
|
||||
Manager *autopaho.ConnectionManager
|
||||
}
|
||||
|
||||
func (p CommandPublisher) PublishCommand(ctx context.Context, deviceID string, command contracts.Command, responseTopic contracts.TopicName, correlation []byte) (byte, error) {
|
||||
// Authorize the reply destination before putting it in the MQTT packet.
|
||||
authorized, err := contracts.DeviceTopic(deviceID, contracts.KindResponses)
|
||||
if err != nil { return 0, fmt.Errorf("build response topic: %w", err) }
|
||||
if responseTopic != authorized {
|
||||
return 0, fmt.Errorf("unauthorized response topic")
|
||||
}
|
||||
if command.DeviceID != deviceID {
|
||||
return 0, fmt.Errorf("command device does not match publish destination")
|
||||
}
|
||||
commandTopic, err := contracts.DeviceTopic(deviceID, contracts.KindCommands)
|
||||
if err != nil { return 0, fmt.Errorf("build command topic: %w", err) }
|
||||
payload, err := json.Marshal(command)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("encode command: %w", err)
|
||||
}
|
||||
operation, cancel := context.WithTimeout(ctx, 3*time.Second)
|
||||
defer cancel()
|
||||
// Correlation Data is copied so the packet owns stable bytes for this publish.
|
||||
ack, err := p.Manager.Publish(operation, &paho.Publish{
|
||||
Topic: commandTopic.String(),
|
||||
QoS: 1,
|
||||
Payload: payload,
|
||||
Properties: &paho.PublishProperties{
|
||||
ContentType: "application/json",
|
||||
ResponseTopic: responseTopic.String(),
|
||||
CorrelationData: append([]byte(nil), correlation...),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("publish command: %w", err)
|
||||
}
|
||||
if ack == nil {
|
||||
return 0, fmt.Errorf("missing QoS 1 PUBACK")
|
||||
}
|
||||
if ack.ReasonCode >= 0x80 {
|
||||
return ack.ReasonCode, fmt.Errorf("broker rejected publish: reason 0x%02x", ack.ReasonCode)
|
||||
}
|
||||
return ack.ReasonCode, nil
|
||||
}
|
||||
</code></pre></article>
|
||||
<article class="card"><h3>📄 internal/commands/receive.go</h3><pre><code>package commands
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"example.com/mqtt5-learning/internal/contracts"
|
||||
"example.com/mqtt5-learning/internal/mqtt"
|
||||
)
|
||||
|
||||
func HandleMessage(service *Service, message mqtt.Message) error {
|
||||
// Validate the route and result body before matching the pending request.
|
||||
topic, err := contracts.ParseTopic(message.Topic)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if topic.Kind != contracts.KindResponses {
|
||||
return fmt.Errorf("expected a response topic")
|
||||
}
|
||||
result, err := contracts.Decode[contracts.CommandResult](message.Payload, topic, time.Now().UTC())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = service.HandleResult(topic.DeviceID, append([]byte(nil), message.CorrelationData...), result)
|
||||
return err
|
||||
}
|
||||
</code></pre></article>
|
||||
<article class="card"><h3>📄 internal/simulator/dedup.go</h3><pre><code>package simulator
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sync"
|
||||
|
||||
"example.com/mqtt5-learning/internal/contracts"
|
||||
)
|
||||
|
||||
type cachedResult struct {
|
||||
fingerprint [32]byte
|
||||
result contracts.CommandResult
|
||||
}
|
||||
|
||||
type Deduplicator struct {
|
||||
mu sync.Mutex
|
||||
capacity int
|
||||
results map[string]cachedResult
|
||||
}
|
||||
|
||||
// NewDeduplicator bounds how many command results this simulator retains.
|
||||
func NewDeduplicator(capacity int) *Deduplicator {
|
||||
if capacity < 1 {
|
||||
capacity = 1
|
||||
}
|
||||
return &Deduplicator{capacity: capacity, results: make(map[string]cachedResult)}
|
||||
}
|
||||
|
||||
func AuthorizedResponseTopic(deviceID, requested string) bool {
|
||||
expected, err := contracts.DeviceTopic(deviceID, contracts.KindResponses)
|
||||
return err == nil && requested == expected.String()
|
||||
}
|
||||
|
||||
// Apply executes effect at most once for a device/command ID while this process lives.
|
||||
func (d *Deduplicator) Apply(command contracts.Command, effect func() (contracts.CommandResult, error)) (contracts.CommandResult, bool, error) {
|
||||
if effect == nil {
|
||||
return contracts.CommandResult{}, false, fmt.Errorf("effect is required")
|
||||
}
|
||||
encoded, err := json.Marshal(command)
|
||||
if err != nil {
|
||||
return contracts.CommandResult{}, false, fmt.Errorf("fingerprint command: %w", err)
|
||||
}
|
||||
fingerprint := sha256.Sum256(encoded)
|
||||
key := command.DeviceID + "/" + command.CommandID
|
||||
d.mu.Lock()
|
||||
defer d.mu.Unlock()
|
||||
if previous, ok := d.results[key]; ok {
|
||||
if previous.fingerprint != fingerprint {
|
||||
return contracts.CommandResult{}, false, fmt.Errorf("command ID was reused with different content")
|
||||
}
|
||||
return previous.result, true, nil
|
||||
}
|
||||
if len(d.results) >= d.capacity {
|
||||
return contracts.CommandResult{}, false, fmt.Errorf("deduplication cache is full")
|
||||
}
|
||||
result, err := effect()
|
||||
if err != nil {
|
||||
return contracts.CommandResult{}, false, err
|
||||
}
|
||||
d.results[key] = cachedResult{fingerprint: fingerprint, result: result}
|
||||
return result, false, nil
|
||||
}
|
||||
</code></pre></article>
|
||||
<article class="card"><h3>📄 internal/simulator/respond.go</h3><pre><code>package simulator
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"example.com/mqtt5-learning/internal/contracts"
|
||||
"example.com/mqtt5-learning/internal/mqtt"
|
||||
"github.com/eclipse/paho.golang/autopaho"
|
||||
"github.com/eclipse/paho.golang/paho"
|
||||
)
|
||||
|
||||
func Respond(ctx context.Context, manager *autopaho.ConnectionManager, cache *Deduplicator, deviceID string, message mqtt.Message) error {
|
||||
// Reject retained or misrouted requests before validating or applying an effect.
|
||||
if message.Retain {
|
||||
return fmt.Errorf("retained commands are rejected")
|
||||
}
|
||||
topic, err := contracts.ParseTopic(message.Topic)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if topic.Kind != contracts.KindCommands || topic.DeviceID != deviceID {
|
||||
return fmt.Errorf("command topic is not authorized for this device")
|
||||
}
|
||||
command, err := contracts.Decode[contracts.Command](message.Payload, topic, time.Now().UTC())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !AuthorizedResponseTopic(deviceID, message.ResponseTopic) {
|
||||
return fmt.Errorf("unauthorized response topic")
|
||||
}
|
||||
// Cache the simulated result by stable command ID to suppress repeat effects.
|
||||
result, _, err := cache.Apply(command, func() (contracts.CommandResult, error) {
|
||||
return contracts.CommandResult{SchemaVersion: contracts.SchemaVersion, MessageID: "result-" + command.CommandID, DeviceID: deviceID, OccurredAt: time.Now().UTC().Format(time.RFC3339), CommandID: command.CommandID, Status: "completed"}, nil
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
payload, err := json.Marshal(result)
|
||||
if err != nil {
|
||||
return fmt.Errorf("encode result: %w", err)
|
||||
}
|
||||
operation, cancel := context.WithTimeout(ctx, 3*time.Second)
|
||||
defer cancel()
|
||||
// Reply only to the authorized destination and echo correlation bytes unchanged.
|
||||
ack, err := manager.Publish(operation, &paho.Publish{Topic: message.ResponseTopic, QoS: 1, Payload: payload, Properties: &paho.PublishProperties{ContentType: "application/json", CorrelationData: append([]byte(nil), message.CorrelationData...)}})
|
||||
if err != nil {
|
||||
return fmt.Errorf("publish response: %w", err)
|
||||
}
|
||||
if ack == nil {
|
||||
return fmt.Errorf("missing response PUBACK")
|
||||
}
|
||||
if ack.ReasonCode >= 0x80 {
|
||||
return fmt.Errorf("response publish rejected: reason 0x%02x", ack.ReasonCode)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
</code></pre></article>
|
||||
<article class="card"><h3>📄 cmd/simulator/main.go</h3><pre><code>package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"example.com/mqtt5-learning/internal/config"
|
||||
"example.com/mqtt5-learning/internal/contracts"
|
||||
"example.com/mqtt5-learning/internal/mqtt"
|
||||
"example.com/mqtt5-learning/internal/simulator"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if err := run(); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func run() error {
|
||||
cfg, err := config.Load(os.Args[1:])
|
||||
if err != nil {
|
||||
return fmt.Errorf("load config: %w", err)
|
||||
}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
commandFilter, err := contracts.DeviceSubscription(cfg.MQTT.DeviceID, contracts.KindCommands)
|
||||
if err != nil { return err }
|
||||
telemetryTopic, err := contracts.DeviceTopic(cfg.MQTT.DeviceID, contracts.KindTelemetry)
|
||||
if err != nil { return err }
|
||||
inbox := make(chan mqtt.Message, 16)
|
||||
manager, err := mqtt.Start(ctx, cfg, []contracts.TopicFilter{commandFilter}, inbox)
|
||||
if err != nil {
|
||||
return fmt.Errorf("start MQTT: %w", err)
|
||||
}
|
||||
defer mqtt.Stop(manager)
|
||||
// Keep recent command results so repeated delivery does not repeat the effect.
|
||||
cache := simulator.NewDeduplicator(1024)
|
||||
ticker := time.NewTicker(3 * time.Second)
|
||||
defer ticker.Stop()
|
||||
sequence := 0
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil
|
||||
case <-ticker.C:
|
||||
sequence++
|
||||
heartbeat := contracts.Heartbeat{SchemaVersion: contracts.SchemaVersion, MessageID: fmt.Sprintf("heartbeat-%d-%d", time.Now().Unix(), sequence), DeviceID: cfg.MQTT.DeviceID, OccurredAt: time.Now().UTC().Format(time.RFC3339)}
|
||||
payload, err := json.Marshal(heartbeat)
|
||||
if err != nil {
|
||||
return fmt.Errorf("encode heartbeat: %w", err)
|
||||
}
|
||||
if err := mqtt.Publish(ctx, manager, telemetryTopic, payload); err != nil {
|
||||
log.Printf("heartbeat publish: %v", err)
|
||||
}
|
||||
case message := <-inbox:
|
||||
if err := simulator.Respond(ctx, manager, cache, cfg.MQTT.DeviceID, message); err != nil {
|
||||
log.Printf("command rejected or response failed topic=%q reason=%v", message.Topic, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre></article><article class="card"><h3>📄 internal/mqtt/client.go — complete file with Section 7 message properties</h3><p>This target version extends the existing MQTT transport message and copies MQTT 5 response properties inside the callback, before the received packet goes out of scope.</p><pre><code>package mqtt
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"net/url"
|
||||
"time"
|
||||
|
||||
"example.com/mqtt5-learning/internal/config"
|
||||
"example.com/mqtt5-learning/internal/contracts"
|
||||
"github.com/eclipse/paho.golang/autopaho"
|
||||
"github.com/eclipse/paho.golang/paho"
|
||||
)
|
||||
|
||||
type Message struct {
|
||||
Topic string
|
||||
Payload []byte
|
||||
Retain bool
|
||||
ResponseTopic string
|
||||
CorrelationData []byte
|
||||
}
|
||||
|
||||
func Start(ctx context.Context, cfg config.Config, topics []contracts.TopicFilter, inbox chan<- Message) (*autopaho.ConnectionManager, error) {
|
||||
broker, err := url.Parse(cfg.MQTT.URL)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
initialSubscription := make(chan error, 1)
|
||||
manager, err := autopaho.NewConnection(ctx, autopaho.ClientConfig{
|
||||
ServerUrls: []*url.URL{broker},
|
||||
KeepAlive: 20,
|
||||
CleanStartOnInitialConnection: true,
|
||||
SessionExpiryInterval: 0,
|
||||
ConnectUsername: cfg.MQTT.Username,
|
||||
ConnectPassword: []byte(cfg.MQTT.Password),
|
||||
OnConnectionUp: func(cm *autopaho.ConnectionManager, _ *paho.Connack) {
|
||||
// This runs again on reconnect; reestablish broker subscriptions.
|
||||
subCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
subscriptions := make([]paho.SubscribeOptions, 0, len(topics))
|
||||
for _, topic := range topics {
|
||||
subscriptions = append(subscriptions, paho.SubscribeOptions{Topic: topic.String(), QoS: 1, RetainAsPublished: true})
|
||||
}
|
||||
|
||||
ack, subErr := cm.Subscribe(subCtx, &paho.Subscribe{Subscriptions: subscriptions})
|
||||
if subErr == nil {
|
||||
if ack == nil || len(ack.Reasons) != len(topics) {
|
||||
subErr = fmt.Errorf("missing or incomplete SUBACK")
|
||||
} else {
|
||||
for i, reason := range ack.Reasons {
|
||||
if reason > 1 { // We requested QoS 1; valid grants are 0 or 1.
|
||||
subErr = fmt.Errorf("subscription %s rejected: 0x%02x", topics[i], reason)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if subErr != nil {
|
||||
log.Printf("subscribe failed: %v", subErr)
|
||||
} else {
|
||||
log.Printf("subscriptions ready: %v", topics)
|
||||
}
|
||||
select {
|
||||
case initialSubscription <- subErr:
|
||||
default:
|
||||
}
|
||||
},
|
||||
OnConnectionDown: func() bool { log.Print("disconnected; reconnecting"); return true },
|
||||
OnConnectError: func(err error) { log.Printf("connect error: %v", err) },
|
||||
ClientID: cfg.MQTT.ClientID,
|
||||
OnClientError: func(err error) { log.Printf("client error: %v", err) },
|
||||
OnServerDisconnect: func(d *paho.Disconnect) { log.Printf("disconnect reason=0x%02x", d.ReasonCode) },
|
||||
OnPublishReceived: []func(paho.PublishReceived) (bool, error){
|
||||
func(received paho.PublishReceived) (bool, error) {
|
||||
packet := received.Packet
|
||||
message := Message{Topic: packet.Topic, Payload: append([]byte(nil), packet.Payload...), Retain: packet.Retain}
|
||||
if properties := packet.Properties; properties != nil {
|
||||
message.ResponseTopic = properties.ResponseTopic
|
||||
message.CorrelationData = append([]byte(nil), properties.CorrelationData...)
|
||||
}
|
||||
select {
|
||||
case inbox <- message:
|
||||
default:
|
||||
// Deliberately lossy early demo: automatic MQTT ACK is still possible.
|
||||
log.Printf("lab inbox full; message dropped topic=%s", packet.Topic)
|
||||
}
|
||||
return true, nil
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("start AutoPaho: %w", err)
|
||||
}
|
||||
startup, cancel := context.WithTimeout(ctx, cfg.ConnectTimeout())
|
||||
defer cancel()
|
||||
if err = manager.AwaitConnection(startup); err != nil {
|
||||
Stop(manager)
|
||||
return nil, fmt.Errorf("await connection: %w", err)
|
||||
}
|
||||
select {
|
||||
case err = <-initialSubscription:
|
||||
if err != nil {
|
||||
Stop(manager)
|
||||
return nil, err
|
||||
}
|
||||
case <-startup.Done():
|
||||
Stop(manager)
|
||||
return nil, fmt.Errorf("await subscriptions: %w", startup.Err())
|
||||
}
|
||||
return manager, nil
|
||||
}
|
||||
|
||||
func Publish(ctx context.Context, manager *autopaho.ConnectionManager, topic contracts.TopicName, payload []byte) error {
|
||||
operation, cancel := context.WithTimeout(ctx, 3*time.Second)
|
||||
defer cancel()
|
||||
|
||||
ack, err := manager.Publish(operation, &paho.Publish{
|
||||
Topic: topic.String(),
|
||||
Payload: payload,
|
||||
QoS: 1,
|
||||
Retain: false,
|
||||
Properties: &paho.PublishProperties{
|
||||
ContentType: "application/json",
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("publish: %w", err)
|
||||
}
|
||||
if ack == nil {
|
||||
return fmt.Errorf("missing Qos 1 publication response")
|
||||
}
|
||||
if ack.ReasonCode >= 0x80 {
|
||||
return fmt.Errorf("publish rejected: 0x%02x", ack.ReasonCode)
|
||||
}
|
||||
|
||||
log.Printf("broker publish acknowledgement topic=%s reason=0x%02x", topic, ack.ReasonCode)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func Stop(manager *autopaho.ConnectionManager) {
|
||||
shutdown, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
if err := manager.Disconnect(shutdown); err != nil {
|
||||
log.Printf("shutdown did not finish within deadline: %v", err)
|
||||
}
|
||||
}
|
||||
</code></pre></article>
|
||||
</details>
|
||||
|
||||
<h2>🏗️ Build and run the command-state stage</h2>
|
||||
<p>After saving the Section 7 files and updating <code>internal/mqtt/client.go</code>, check the repository and build both processes from its root:</p>
|
||||
<pre><code>go test ./...
|
||||
go test -race ./...
|
||||
go build -o /tmp/mqtt5-lab-backend ./cmd/backend
|
||||
go build -o /tmp/mqtt5-lab-simulator ./cmd/simulator</code></pre>
|
||||
<p>These commands run the tests that currently exist and check compilation. The race detector only covers schedules exercised by tests. Section 7 has no HTTP command entrypoint yet, so starting the processes verifies MQTT startup, subscriptions, and simulator heartbeats; it does not trigger a command/response round trip by itself.</p>
|
||||
<p>For that smoke run, start the named broker and inspect its state:</p>
|
||||
<pre><code>docker compose -p mqtt5-learning up -d broker
|
||||
docker compose -p mqtt5-learning ps</code></pre>
|
||||
<p>Then run these in separate terminals:</p>
|
||||
<pre><code>/tmp/mqtt5-lab-simulator -username simulator -client-id lab-simulator-device-001</code></pre>
|
||||
<pre><code>/tmp/mqtt5-lab-backend -username backend -client-id lab-backend</code></pre>
|
||||
<p>Expect successful connection/subscription logs and simulator heartbeats. Stop both with Ctrl+C, then stop the broker using <code>docker compose -p mqtt5-learning stop broker</code>. Lesson 8 adds the HTTP endpoint that can create a ping and exercise correlation end to end. Ask me a follow-up if your build or startup evidence is unexpected.</p>
|
||||
|
||||
<section class="card callout"><h2>✅ Takeaway</h2><p><strong>Register before publish. Match device, command ID, and correlation. Treat a missing result as unknown.</strong></p><p>Tell me how you handled the race; we’ll carry that state into an HTTP endpoint next.</p></section>
|
||||
<footer><p>Previous: <a href="0006-bounded-message-work.html">Lesson 6</a> · Next: <a href="0008-http-acceptance-vs-completion.html">Lesson 8 — expose command state over HTTP</a> · <a href="../reference/message-processing-lifecycle.html">lifecycle reference</a></p><p>Sources: <a href="https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html">OASIS MQTT 5.0 Standard</a>; <a href="https://pkg.go.dev/github.com/eclipse/paho.golang/paho@v0.23.0">Paho Go v0.23.0 API</a>; <a href="../course-source/07-command-response.md">repository Section 7</a>.</p></footer>
|
||||
</main></body></html>
|
||||
Reference in New Issue
Block a user