mqtt-5-learning/teaching/lessons/0006-bounded-message-work.html

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Lesson 6 — Bound Message Work</title>
  <link rel="stylesheet" href="../assets/lesson.css"><script src="../assets/quiz.js" defer></script>
</head>
<body><main class="page">
  <p class="eyebrow">Go MQTT Vending Backend · Lesson 6 · 8 minutes</p>
  <h1>Bound message work; make overload visible</h1>
  <p class="dek">Move application work out of the MQTT receive callback without creating an unlimited backlog or hiding queue rejection.</p>
  <div class="meta"><span class="pill">🎯 One skill: reason about admission</span><span class="pill">🧠 Predict overload behavior</span><span class="pill">🧪 In-memory queue</span></div>

  <section class="card callout"><p class="question">❓ Start with the story</p><p>A burst of vending heartbeats arrives while workers are slow. The callback returns, but the telemetry queue is full. What happened to the packet, and what can the backend truthfully claim?</p><p>Pause before reading: separate MQTT delivery acknowledgement from the backend's queue decision.</p></section>
  <section class="card"><h2>📍 Files to implement in the codebase</h2><p><code>internal/dispatch/pool.go</code> is already present. Read it first: this lesson teaches its admission, worker, and shutdown behavior, then aligns the routing, pool coordination, and both process entrypoints with the listings below.</p><table><thead><tr><th>Path</th><th>Action</th><th>What you will learn or change</th></tr></thead><tbody><tr><td><code>internal/dispatch/pool.go</code></td><td>Read and understand existing file</td><td>Fixed workers, bounded queue, immediate admission outcomes, counters, bounded drain</td></tr><tr><td><code>internal/dispatch/receiver.go</code></td><td>Review and align</td><td>Route by parsed topic kind and copy payload ownership before enqueue</td></tr><tr><td><code>internal/dispatch/shutdown.go</code></td><td>Review and align</td><td>Stop every pool before waiting under one shared deadline</td></tr><tr><td><code>cmd/backend/main.go</code></td><td>Replace with complete target file below</td><td>Wire Section 5A service to separate bounded pools and preserve MQTT during drain</td></tr><tr><td><code>cmd/simulator/main.go</code></td><td>Replace with complete target file below</td><td>Handle device commands through bounded work and drain before disconnect</td></tr></tbody></table><p>The routing and shutdown files are already present in this checkout. Compare them with the complete listings below and preserve the same admission and error semantics.</p></section>
  <h2>📄 File overview: `pool.go` controls finite work</h2>
  <p><code>internal/dispatch/pool.go</code> owns a fixed worker set and a bounded jobs channel. It limits simultaneous work and waiting work. It does not make queued messages durable, and it cannot ask MQTT to reverse an acknowledgement already sent.</p>
  <div class="flow" aria-label="dispatch lifecycle"><div class="step">MQTT callback<br><strong>copy + submit</strong></div><span class="arrow">→</span><div class="step">Finite queue<br><strong>admit or reject</strong></div><span class="arrow">→</span><div class="step">Fixed workers<br><strong>run service</strong></div></div>

  <article class="card">
    <h3>📥 `Submit` — short summary</h3>
    <p>Returns immediately with an explicit outcome: accepted, full, or stopping.</p>
    <pre><code>func (p *Pool) Submit(ctx context.Context, job Job) (Outcome, error) {
    if job == nil { return Stopping, errors.New("nil job") }
    if err := ctx.Err(); err != nil { return Stopping, err }

    p.mu.Lock()
    defer p.mu.Unlock()
    if !p.accepting { return Stopping, nil }
    select {
    case p.jobs &lt;- job:
        p.accepted.Add(1)
        return Accepted, nil
    default:
        p.rejected.Add(1)
        return Full, nil
    }
}</code></pre>
    <p class="small">This is the focused <code>Submit</code> method excerpt from <code>Pool</code>. The complete file defines the pool fields, <code>Outcome</code> values, worker loop, and shutdown behavior.</p>
    <h3>🧩 Details: why each branch exists</h3>
    <ul><li>Reject nil or already-cancelled work before touching the queue.</li><li>The mutex makes the accepting check and channel send atomic with shutdown closing the channel.</li><li>The <code>default</code> branch means “no slot now”: this method never waits for capacity.</li><li><code>Accepted</code> means queued in memory, not completed or saved to disk.</li><li><code>Full</code> is an application overload decision. With automatic acknowledgement, returning it does not undo PUBACK or request redelivery.</li></ul>
  </article>

  <h2>⚖️ Choose a policy per traffic class</h2>
  <div class="two-col"><article class="card"><h3>📈 Telemetry</h3><p>If the business allows it, drop a stale heartbeat and count the rejection. Freshness will age out. Make the loss visible.</p></article><article class="card"><h3>🧾 Sales and commands</h3><p>Silently dropping a report or command is usually unacceptable. Return an explicit failure and add durable admission before claiming crash-safe processing.</p></article></div>
  <p>Separate pools can prevent telemetry from consuming every waiting slot reserved for important work. That is capacity isolation, not durability. Shutdown order matters too: stop producers, stop admission, drain workers to a deadline while needed transport remains alive, then disconnect.</p>
  <section class="card"><h2>🛑 What does <code>ctx.Err()</code> contain during shutdown?</h2><p>The <code>ctx</code> passed to <code>Pool.Shutdown</code> is the drain context. If its deadline expires before workers finish, <code>ctx.Err()</code> returns <code>context.DeadlineExceeded</code>. If something cancels it earlier, it returns <code>context.Canceled</code>. If workers finish first, <code>Pool.Shutdown</code> returns <code>nil</code> instead. On timeout, the pool uses that context error as its shutdown result and cancels the worker job context. Cancellation is cooperative, so a handler must check its context or use context-aware I/O to stop promptly.</p></section>

  <h2>🧭 File and function map</h2>
  <p><strong>File overview:</strong> Section 6 adds three pieces: <code>pool.go</code> owns bounded execution, <code>receiver.go</code> routes each message to the right pool, and <code>shutdown.go</code> closes all pools before waiting for their workers. The backend entrypoint below wires those pieces to the Section 5A service and existing MQTT inbox.</p>
  <div class="two-col">
    <article class="card"><h3>📄 <code>internal/dispatch/pool.go</code></h3>
      <p><strong><code>New</code> summary:</strong> starts a fixed worker count and a queue with a fixed capacity.</p>
      <p><strong><code>Submit</code> summary:</strong> reports <code>Accepted</code>, <code>Full</code>, or <code>Stopping</code> without waiting for queue room.</p>
      <p><strong><code>StopAdmission</code> / <code>Shutdown</code> summary:</strong> close the submission path, wait for workers up to a deadline, then cancel remaining work.</p>
      <p><strong><code>Stats</code> / <code>worker</code> summary:</strong> expose admission and completion counts; workers run jobs and count success or failure.</p>
      <h4>🔎 Details</h4><p>The mutex protects both the accepting flag and channel close, preventing a send-on-closed-channel race. The nonblocking <code>select</code> makes overload immediate. Closing the jobs channel lets workers finish buffered jobs; the shared deadline keeps draining bounded. If it expires, cancellation is cooperative: a job that ignores its context can keep running. Counters describe process-local events, not durable business records.</p>
    </article>
    <article class="card"><h3>📄 <code>internal/dispatch/receiver.go</code></h3>
      <p><strong><code>Receiver.Submit</code> summary:</strong> parses the topic, selects the telemetry or important pool, copies payload bytes, and submits a closure that calls the application processor.</p>
      <h4>🔎 Details</h4><p>Routing on a parsed <code>Kind</code> keeps traffic policy out of the MQTT callback. Commands, sales, and command responses use the important pool; only telemetry uses the telemetry pool. Copying the payload transfers ownership: the worker no longer depends on Paho or a reused inbox buffer. The job closure captures that owned copy and runs later with the worker's context.</p>
    </article>
    <article class="card"><h3>📄 <code>internal/dispatch/shutdown.go</code></h3>
      <p><strong><code>Shutdown</code> summary:</strong> stops submissions to every pool first, then drains them using one shared deadline.</p>
      <h4>🔎 Details</h4><p>Stopping all pools before waiting prevents one pool from accepting new work while the other drains. Reusing one context bounds the total shutdown time instead of allowing a fresh timeout for each pool. <code>errors.Join</code> preserves timeout failures from multiple pools. Keep MQTT alive during this interval if jobs need to publish their result; disconnect only after the drain attempt.</p>
    </article>
  </div>
  <h3>🔌 Integration overview: edit <code>cmd/backend/main.go</code></h3>
  <p><strong><code>run</code> summary:</strong> builds the service and pools once, starts MQTT, sends each inbox message through <code>Receiver</code>, and drains workers when the signal context ends.</p>
  <p><strong>Details:</strong> the signal context stops the main loop, while a separate transport context keeps MQTT available during the bounded drain. The signal case itself only logs and returns; as <code>run</code> returns, deferred cleanup executes in reverse registration order. The dispatcher drain defer was registered after <code>mqtt.Stop</code>, so it runs first. Pool jobs use an independent parent context so the signal does not cancel them before draining. Once drain completes or times out, <code>mqtt.Stop</code> disconnects the client. The inbox and callback remain bounded; if more messages arrive during shutdown they can still be dropped under automatic acknowledgement, which this lesson reports rather than hiding.</p>
  <details class="card" open><summary><strong>🧩 Complete Section 6 backend wiring: <code>cmd/backend/main.go</code></strong></summary><p>This is the full target entrypoint layered on the Section 5A service. The three dispatcher files follow it below.</p><pre><code>package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"
	"log/slog"
	"os"
	"os/signal"
	"syscall"
	"time"

	"example.com/mqtt5-learning/internal/config"
	"example.com/mqtt5-learning/internal/contracts"
	"example.com/mqtt5-learning/internal/deviceview"
	"example.com/mqtt5-learning/internal/dispatch"
	"example.com/mqtt5-learning/internal/mqtt"
)

func main() {
	if err := run(); err != nil {
		log.Fatal(err)
	}
}

func run() error {
	cfg, err := config.Load(os.Args[1:])
	if err != nil {
		return fmt.Errorf("failed to load config: %w", err)
	}

	lifecycleCtx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
	defer stop()
	transportCtx, stopTransport := context.WithCancel(context.Background())
	defer stopTransport()

	store := deviceview.NewStore()
	service, err := deviceview.NewService(store, slog.Default(), 90*time.Second)
	if err != nil {
		return fmt.Errorf("create device-view service: %w", err)
	}

	telemetryFilter, err := contracts.DeviceFilter(contracts.KindTelemetry)
	if err != nil { return err }
	responsesFilter, err := contracts.DeviceFilter(contracts.KindResponses)
	if err != nil { return err }
	commandTopic, err := contracts.DeviceTopic(cfg.MQTT.DeviceID, contracts.KindCommands)
	if err != nil { return err }
	inbox := make(chan mqtt.Message, 16)
	manager, err := mqtt.Start(transportCtx, cfg, []contracts.TopicFilter{telemetryFilter, responsesFilter}, inbox)
	if err != nil {
		return fmt.Errorf("failed to start MQTT client: %w", err)
	}
	defer mqtt.Stop(manager)

	telemetryPool := dispatch.New(context.Background(), 2, 16)
	importantPool := dispatch.New(context.Background(), 2, 32)
	receiver := &amp;dispatch.Receiver{
		Telemetry: telemetryPool,
		Important: importantPool,
		Process:   service.HandleMessage,
	}
	defer func() {
		drainCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
		defer cancel()
		started := time.Now()
		if err := dispatch.Shutdown(drainCtx, telemetryPool, importantPool); err != nil {
			log.Printf("dispatcher shutdown incomplete: %v", err)
		}
		log.Printf("dispatcher drain finished elapsed=%s telemetry=%+v important=%+v",
			time.Since(started), telemetryPool.Stats(), importantPool.Stats())
	}()

	ticker := time.NewTicker(10 * time.Second)
	defer ticker.Stop()
	runID := time.Now().UnixNano()
	sequence := 0

	for {
		select {
		case &lt;-lifecycleCtx.Done():
			log.Print("shutting down; draining accepted work before MQTT disconnect")
			return nil
		case message := &lt;-inbox:
			outcome, submitErr := receiver.Submit(lifecycleCtx, message)
			if submitErr != nil || outcome != dispatch.Accepted {
				slog.Warn("message admission rejected", "topic", message.Topic, "outcome", outcome, "error", submitErr)
			}
		case &lt;-ticker.C:
			sequence++
			id := fmt.Sprintf("ping-%d-%d", runID, sequence)
			payload, marshalErr := json.Marshal(contracts.Command{
				SchemaVersion: contracts.SchemaVersion,
				MessageID: id,
				DeviceID: cfg.MQTT.DeviceID,
				OccurredAt: time.Now().UTC().Format(time.RFC3339),
				CommandID: id,
				Command: "ping",
			})
			if marshalErr != nil {
				return fmt.Errorf("failed to marshal payload: %w", marshalErr)
			}
			if publishErr := mqtt.Publish(lifecycleCtx, manager, commandTopic, payload); publishErr != nil {
				log.Printf("command publication failed or outcome uncertain: %v", publishErr)
			}
		}
	}
}
</code></pre></details>
  <h3>🤖 Simulator integration overview: edit <code>cmd/simulator/main.go</code></h3>
  <p><strong><code>run</code> summary:</strong> starts the device client, publishes heartbeats, and submits received commands to bounded workers instead of handling them inline in the main loop.</p>
  <p><strong><code>respond</code> summary:</strong> rejects retained or misrouted commands, validates the contract, and publishes the pong using the worker context.</p>
  <p><strong>Details:</strong> the simulator attempts to drain accepted command jobs before closing MQTT so a worker can still publish its response. The drain has a five-second limit; if it expires, remaining in-memory work may not complete. A full important queue is logged as an application rejection; automatic acknowledgement may already have completed the subscriber delivery, so this RAM-only example cannot promise redelivery.</p>
  <details class="card" open><summary><strong>🧩 Complete Section 6 simulator wiring: <code>cmd/simulator/main.go</code></strong></summary><p>This complete target file carries the Section 5 contract-aware simulator forward and moves inbound command work to the dispatcher.</p><pre><code>package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"
	"os"
	"os/signal"
	"sync/atomic"
	"syscall"
	"time"

	"example.com/mqtt5-learning/internal/config"
	"example.com/mqtt5-learning/internal/contracts"
	"example.com/mqtt5-learning/internal/dispatch"
	"example.com/mqtt5-learning/internal/mqtt"
	"github.com/eclipse/paho.golang/autopaho"
)

func main() {
	if err := run(); err != nil {
		log.Fatal(err)
	}
}

func run() error {
	cfg, err := config.Load(os.Args[1:])
	if err != nil {
		return fmt.Errorf("load config: %w", err)
	}
	lifecycleCtx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
	defer stop()
	transportCtx, stopTransport := context.WithCancel(context.Background())
	defer stopTransport()

	commandFilter, err := contracts.DeviceSubscription(cfg.MQTT.DeviceID, contracts.KindCommands)
	if err != nil { return err }
	telemetryTopic, err := contracts.DeviceTopic(cfg.MQTT.DeviceID, contracts.KindTelemetry)
	if err != nil { return err }
	inbox := make(chan mqtt.Message, 16)
	manager, err := mqtt.Start(transportCtx, cfg, []contracts.TopicFilter{commandFilter}, inbox)
	if err != nil {
		return fmt.Errorf("start MQTT: %w", err)
	}
	defer mqtt.Stop(manager)

	runID := time.Now().UnixNano()
	telemetryPool := dispatch.New(context.Background(), 2, 16)
	importantPool := dispatch.New(context.Background(), 2, 16)
	var resultSequence atomic.Uint64
	receiver := &amp;dispatch.Receiver{
		Telemetry: telemetryPool,
		Important: importantPool,
		Process: func(jobCtx context.Context, message mqtt.Message) error {
			resultID := fmt.Sprintf("result-%d-%d", runID, resultSequence.Add(1))
			return respond(jobCtx, manager, cfg.MQTT.DeviceID, resultID, message)
		},
	}
	defer func() {
		drainCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
		defer cancel()
		started := time.Now()
		if err := dispatch.Shutdown(drainCtx, telemetryPool, importantPool); err != nil {
			log.Printf("dispatcher shutdown incomplete: %v", err)
		}
		log.Printf("dispatcher drain finished elapsed=%s telemetry=%+v important=%+v",
			time.Since(started), telemetryPool.Stats(), importantPool.Stats())
	}()

	ticker := time.NewTicker(3 * time.Second)
	defer ticker.Stop()
	sequence := uint64(0)
	for {
		select {
		case &lt;-lifecycleCtx.Done():
			log.Print("shutting down; draining accepted command work before MQTT disconnect")
			return nil
		case &lt;-ticker.C:
			sequence++
			heartbeat := contracts.Heartbeat{
				SchemaVersion: contracts.SchemaVersion,
				MessageID:     fmt.Sprintf("heartbeat-%d-%d", time.Now().Unix(), sequence),
				DeviceID:      cfg.MQTT.DeviceID,
				OccurredAt:    time.Now().UTC().Format(time.RFC3339),
			}
			payload, marshalErr := json.Marshal(heartbeat)
			if marshalErr != nil {
				return fmt.Errorf("encode heartbeat: %w", marshalErr)
			}
			if publishErr := mqtt.Publish(lifecycleCtx, manager, telemetryTopic, payload); publishErr != nil {
				log.Printf("heartbeat publish: %v", publishErr)
			}
		case message := &lt;-inbox:
			outcome, submitErr := receiver.Submit(lifecycleCtx, message)
			if submitErr != nil || outcome != dispatch.Accepted {
				log.Printf("command admission rejected outcome=%d topic=%q error=%v", outcome, message.Topic, submitErr)
			}
		}
	}
}

func respond(ctx context.Context, manager *autopaho.ConnectionManager, deviceID, resultMessageID string, message mqtt.Message) error {
	if message.Retain {
		return fmt.Errorf("retained command rejected")
	}
	topic, err := contracts.ParseTopic(message.Topic)
	if err != nil {
		return err
	}
	if topic.Kind != contracts.KindCommands || topic.DeviceID != deviceID {
		return fmt.Errorf("command topic is not authorized for this device")
	}
	command, err := contracts.Decode[contracts.Command](message.Payload, topic, time.Now().UTC())
	if err != nil {
		return err
	}
	result := contracts.CommandResult{
		SchemaVersion: contracts.SchemaVersion,
		MessageID:     resultMessageID,
		DeviceID:      deviceID,
		OccurredAt:    time.Now().UTC().Format(time.RFC3339),
		CommandID:     command.CommandID,
		Status:        "completed",
	}
	payload, err := json.Marshal(result)
	if err != nil {
		return fmt.Errorf("encode result: %w", err)
	}
	responseTopic, err := contracts.DeviceTopic(deviceID, contracts.KindResponses)
	if err != nil { return fmt.Errorf("build response topic: %w", err) }
	return mqtt.Publish(ctx, manager, responseTopic, payload)
}
</code></pre></details>

  <h2>🧪 Check your model</h2>
  <form class="card quiz" data-quiz data-correct="b" data-explanation="Correct. Full describes local queue admission; automatic MQTT acknowledgement is a separate protocol decision and is not rolled back." data-retry="Ask what Submit controls and what Paho controls. Are those the same state machine?"><fieldset class="question-block"><legend><strong>1. The bounded queue returns `Full`. What does that change?</strong></legend><label class="option"><input type="radio" name="q1" value="a"> The broker must retransmit this message automatically now</label><label class="option"><input type="radio" name="q1" value="b"> The app rejected admission; PUBACK may already stand</label><label class="option"><input type="radio" name="q1" value="c"> The device command has completed with a failure</label></fieldset><button type="submit">Check answer</button><p class="feedback" data-feedback aria-live="polite"></p></form>
  <form class="card quiz" data-quiz data-correct="c" data-explanation="Right. Accepted means the queue received an in-memory job. A worker has not necessarily completed it, and a process crash can lose it." data-retry="Which counter records enqueue, and which evidence records worker completion?"><fieldset class="question-block"><legend><strong>2. What does `Accepted` mean for a queue job?</strong></legend><label class="option"><input type="radio" name="q2" value="a"> The worker completed the vending business operation</label><label class="option"><input type="radio" name="q2" value="b"> The broker durably stored the application result</label><label class="option"><input type="radio" name="q2" value="c"> The job entered the process's in-memory queue</label></fieldset><button type="submit">Check answer</button><p class="feedback" data-feedback aria-live="polite"></p></form>
  <form class="card quiz" data-quiz data-correct="a" data-explanation="Correct. Stop admission, drain workers while needed transport remains available, then close it. A drain deadline can still leave RAM-only work unfinished." data-retry="Could a worker publish a response after MQTT has already been disconnected?"><fieldset class="question-block"><legend><strong>3. What should shutdown do before MQTT disconnect?</strong></legend><label class="option"><input type="radio" name="q3" value="a"> Stop admission, drain workers, then close the transport</label><label class="option"><input type="radio" name="q3" value="b"> Disconnect MQTT, then let workers finish replies later</label><label class="option"><input type="radio" name="q3" value="c"> Keep admitting jobs until the process exits fully</label></fieldset><button type="submit">Check answer</button><p class="feedback" data-feedback aria-live="polite"></p></form>

  <h2>🛠️ Your tiny practice</h2><div class="card"><p>When capacity is full, write one sentence and one metric for each class: heartbeat, sales report, command result. Then state the exact loss window created by automatic acknowledgement plus an in-memory queue. Compare with <a href="../course-source/06-concurrency-cancellation.md">Section 6's dispatcher and shutdown code ↗</a>.</p></div>

  <details class="card" open><summary><strong>📦 Section 6 complete dispatcher files</strong></summary><p>Complete source files for the added dispatcher package. The complete backend integration appears immediately above.</p><article class="card"><h3>📄 internal/dispatch/pool.go</h3><pre><code>package dispatch

import (
	"context"
	"errors"
	"sync"
	"sync/atomic"
)

type Outcome uint8

// Outcomes make queue admission success and overload visible to callers.
const (
	Accepted Outcome = iota
	Full
	Stopping
)

type Job func(context.Context) error

type Stats struct {
	Accepted  uint64
	Rejected  uint64
	Completed uint64
	Failed    uint64
}

// Pool owns a bounded queue, worker lifecycle, and admission counters.
type Pool struct {
	mu        sync.Mutex
	accepting bool
	jobs      chan Job
	ctx       context.Context
	cancel    context.CancelFunc
	workers   sync.WaitGroup
	done      chan struct{}

	accepted  atomic.Uint64
	rejected  atomic.Uint64
	completed atomic.Uint64
	failed    atomic.Uint64
}

// New starts a fixed worker set and ties job cancellation to parent.
func New(parent context.Context, workers, capacity int) *Pool {
	if workers &lt; 1 {
		workers = 1
	}
	if capacity &lt; 0 {
		capacity = 0
	}
	ctx, cancel := context.WithCancel(parent)
	p := &amp;Pool{accepting: true, jobs: make(chan Job, capacity), ctx: ctx, cancel: cancel, done: make(chan struct{})}
	p.workers.Add(workers)
	for i := 0; i &lt; workers; i++ {
		go p.worker()
	}
	go func() { p.workers.Wait(); close(p.done) }()
	return p
}

// Submit never waits for queue capacity. The caller gets an explicit Full result.
// The caller context can cancel admission before it enters the queue.
func (p *Pool) Submit(ctx context.Context, job Job) (Outcome, error) {
	if job == nil {
		return Stopping, errors.New("nil job")
	}
	if err := ctx.Err(); err != nil {
		return Stopping, err
	}
	p.mu.Lock()
	defer p.mu.Unlock()
	if !p.accepting {
		return Stopping, nil
	}
	select {
	case p.jobs &lt;- job:
		p.accepted.Add(1)
		return Accepted, nil
	default:
		p.rejected.Add(1)
		return Full, nil
	}
}

func (p *Pool) StopAdmission() {
	p.mu.Lock()
	defer p.mu.Unlock()
	if p.accepting {
		p.accepting = false
		close(p.jobs)
	}
}

// Shutdown stops new submissions and drains queued work until ctx expires.
// Jobs must observe their context so deadline cancellation can stop slow work.
func (p *Pool) Shutdown(ctx context.Context) error {
	p.StopAdmission()
	select {
	case &lt;-p.done:
		p.cancel()
		return nil
	case &lt;-ctx.Done():
		p.cancel()
		return ctx.Err()
	}
}

func (p *Pool) Stats() Stats {
	return Stats{Accepted: p.accepted.Load(), Rejected: p.rejected.Load(), Completed: p.completed.Load(), Failed: p.failed.Load()}
}

// Workers execute admitted jobs and count their final result.
func (p *Pool) worker() {
	defer p.workers.Done()
	for {
		select {
		case &lt;-p.ctx.Done():
			return
		case job, ok := &lt;-p.jobs:
			if !ok {
				return
			}
			if err := job(p.ctx); err != nil {
				p.failed.Add(1)
				continue
			}
			p.completed.Add(1)
		}
	}
}
</code></pre></article>
<article class="card"><h3>📄 internal/dispatch/receiver.go</h3><pre><code>package dispatch

import (
	"context"
	"fmt"

	"example.com/mqtt5-learning/internal/contracts"
	"example.com/mqtt5-learning/internal/mqtt"
)

type Processor func(context.Context, mqtt.Message) error

// Receiver validates the route, selects a pool, and copies payload ownership.
type Receiver struct {
	Telemetry *Pool
	Important *Pool
	Process   Processor
}

func (r *Receiver) Submit(ctx context.Context, message mqtt.Message) (Outcome, error) {
	if r.Process == nil {
		return Stopping, fmt.Errorf("message processor is required")
	}
	topic, err := contracts.ParseTopic(message.Topic)
	if err != nil {
		return Stopping, err
	}
	pool := r.Telemetry
	if topic.Kind == contracts.KindCommands || topic.Kind == contracts.KindResponses || topic.Kind == contracts.KindSales {
		pool = r.Important
	}
	if pool == nil {
		return Stopping, fmt.Errorf("no queue configured for topic kind %q", topic.Kind)
	}
	copyOfMessage := message
	copyOfMessage.Payload = append([]byte(nil), message.Payload...)
	return pool.Submit(ctx, func(jobCtx context.Context) error { return r.Process(jobCtx, copyOfMessage) })
}
</code></pre></article>
<article class="card"><h3>📄 internal/dispatch/shutdown.go</h3><pre><code>package dispatch

import (
	"context"
	"errors"
)

func Shutdown(ctx context.Context, pools ...*Pool) error {
	// Close admission on every pool before waiting for any worker to drain.
	for _, pool := range pools {
		if pool != nil {
			pool.StopAdmission()
		}
	}
	var result error
	// Share one deadline across the drain so total shutdown time stays bounded.
	for _, pool := range pools {
		if pool != nil {
			result = errors.Join(result, pool.Shutdown(ctx))
		}
	}
	return result
}
</code></pre></article></details>

  <h2>🏗️ Build and smoke-run the updated processes</h2>
  <p>After saving all three <code>internal/dispatch</code> files and wiring both entrypoints, run these commands from the repository root:</p>
  <pre><code>go test ./...
go test -race ./...
go build -o /tmp/mqtt5-lab-backend ./cmd/backend
go build -o /tmp/mqtt5-lab-simulator ./cmd/simulator</code></pre>
  <p>The race detector can report races only on paths the tests execute; it does not prove every worker schedule safe. The builds confirm both programs use the updated wiring.</p>
  <p>Start only the named lab broker, then run the two binaries in separate terminals:</p>
  <pre><code>docker compose -p mqtt5-learning up -d broker
docker compose -p mqtt5-learning ps</code></pre>
  <pre><code>/tmp/mqtt5-lab-simulator -username simulator -client-id lab-simulator-device-001</code></pre>
  <pre><code>/tmp/mqtt5-lab-backend -username backend -client-id lab-backend</code></pre>
  <p>Confirm heartbeats and ping responses still complete while passing through worker queues. On Ctrl+C, observe the shutdown summary and compare accepted, rejected, completed, failed, and drain-time values if your wiring logs them. This ordinary traffic is a smoke run, not an overload test; use a finite burst larger than active workers plus queue capacity to observe <code>Full</code> outcomes. Stop the clients, then stop the lab broker with <code>docker compose -p mqtt5-learning stop broker</code>.</p>
  <p>Ask me a follow-up if the counters or shutdown behavior differ from your prediction.</p>

  <section class="card callout"><h2>✅ Takeaway</h2><p><strong>A bounded queue protects process memory; an explicit overload policy protects business meaning. Neither makes work durable.</strong></p><p>Tell me which message class you would admit first and why.</p></section>
  <footer><p>Previous: <a href="0005a-backend-message-processing.html">Lesson 5A</a> · Next: <a href="0007-command-state.html">Lesson 7 — command identity and state</a> · <a href="../reference/message-processing-lifecycle.html">lifecycle reference</a></p><p>Sources: <a href="https://pkg.go.dev/context">Go `context`</a>; <a href="https://pkg.go.dev/sync">Go `sync`</a>; <a href="https://pkg.go.dev/github.com/eclipse/paho.golang/autopaho/examples/queue@v0.23.0">Paho Go v0.23.0 queue example</a>; <a href="../course-source/06-concurrency-cancellation.md">repository Section 6</a>.</p></footer>
</main></body></html>