feat: expand MQTT contract and reliability lessons
This commit is contained in:
@@ -2,9 +2,9 @@
|
||||
|
||||
## 🎬 Start with the system story
|
||||
|
||||
> 💡 **Story question:** The broker can route a message because its topic matches. How does the Go backend decide that the topic and payload describe a valid message from the expected device?
|
||||
> **❓ Question:** **The broker can route a message because its topic matches. How does the Go backend decide that the topic and payload describe a valid message from the expected device?**
|
||||
|
||||
Routing is not validation or identity proof. A message arrives as a topic plus bytes; the application must parse the route, decode the payload, and check that fields and identity make sense before changing state. A device ID inside JSON is a claim, so broker authentication and topic authorization remain separate safeguards.
|
||||
**💬 Answer:** The topic provides the route, but it does not verify the payload. The backend parses the topic, validates the JSON, and compares the claimed device ID before business handling. Broker authentication and ACLs separately control who may publish to a topic.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -20,20 +20,24 @@ flowchart LR
|
||||
|
||||
Prerequisite: Section 4 exchanges heartbeats and ping/pong. The clients share the disposable password in `configs/lab.toml`; select the simulator account with `-username simulator` when starting it.
|
||||
|
||||
Run all commands from `/home/ryan/Media/Code/Code/robot-market/mqtt-5-learning`. Create files at the project-relative paths shown below, not under `lessons/`. Continue the same application and pinned Paho v0.23.0 dependency. These are guided implementation exercises; attempt each step before requesting a full solution.
|
||||
Run all commands from `/home/ryan/Media/Code/Code/robot-market/mqtt-5-learning`. Save the complete code blocks at their project-relative paths, not under `lessons/`. This lesson provides a full working implementation with the pinned Paho v0.23.0 dependency; use the exercises to inspect and verify its behavior.
|
||||
|
||||
## 📨 Follow one message through the trust boundaries
|
||||
|
||||
Suppose the backend receives topic `lab/v1/devices/device-001/sales` with a body claiming `device_id: device-002`. The broker may have correctly authorized the sender to publish to the topic. It has not established that the JSON is truthful. Reject the mismatch before selecting a device record or changing a balance.
|
||||
**❓ Question:** **The backend receives a sales report on device-001's topic, but its JSON says `device_id: device-002`. What could go wrong if the backend accepts it?**
|
||||
|
||||
**💬 Answer:** The report could be applied to the wrong device. The broker may allow the publish, but it does not compare the topic with the JSON. The backend compares both IDs and rejects a mismatch before changing data.
|
||||
|
||||
```text
|
||||
PUBLISH → size limit → topic parser → JSON decoder → schema checks
|
||||
→ topic/body identity check → business validation → handler
|
||||
```
|
||||
|
||||
Each step answers a different question. The size limit bounds decoding cost. Parsing establishes the intended route. JSON decoding proves structure only. Business validation rejects a syntactically valid negative quantity. None of these replaces broker authentication.
|
||||
The backend applies several checks because each catches a different problem. A size limit bounds the payload it decodes. The topic parser identifies the device and message type. JSON decoding checks the data's shape, while business validation checks its values. For example, a sales report can be valid JSON but contain a negative quantity. The backend rejects that value. These checks do not replace broker authentication or ACLs.
|
||||
|
||||
The 16 KiB application limit applies to `Payload`, not the entire MQTT packet. Topic and properties add bytes; the library has already received the packet before this validator runs. Lesson 16 adds protocol packet-size limits to bound an earlier layer.
|
||||
**❓ Question:** **Does the 16 KiB application limit cap the whole MQTT packet?**
|
||||
|
||||
**💬 Answer:** No. It caps the payload the application decodes. The topic and properties add packet bytes, and Paho has already received the packet before this check runs. Lesson 16 covers an earlier packet-size limit.
|
||||
|
||||
### 📨 Map encoding metadata into Paho
|
||||
|
||||
@@ -51,9 +55,11 @@ properties := &paho.PublishProperties{
|
||||
_ = properties
|
||||
```
|
||||
|
||||
Payload Format Indicator 1 declares UTF-8 character data. It does not mean JSON, schema version 1, or trusted content. Content Type describes the media type. For example, `hello` is UTF-8 but fails your JSON object contract. Property pointers distinguish an omitted property from an explicitly supplied value; inspect nil before dereferencing incoming fields.
|
||||
**❓ Question:** **A message declares JSON, but its payload is `hello`. Is it valid JSON?**
|
||||
|
||||
Keep these distinctions in your transport type when needed: packet metadata belongs alongside payload bytes, while `schema_version` and `command_id` belong to your application contract. Do not make handlers depend on a mutable packet owned elsewhere merely to read metadata.
|
||||
**💬 Answer:** No. Content Type describes the media type; Payload Format Indicator 1 says the bytes are UTF-8 text. Neither validates the content or proves its schema. Decode the bytes, and check an incoming property pointer for nil before reading it; nil means that property was omitted.
|
||||
|
||||
Keep MQTT properties with the received transport message when the application needs them. Fields such as `schema_version` and `command_id` belong in the JSON contract type. This lets handlers use the needed data without depending on a packet object that another part of the program may change.
|
||||
|
||||
### 🧪 Predict, modify, explain
|
||||
|
||||
@@ -62,49 +68,508 @@ Send a valid heartbeat, then change only its body device ID, then change only it
|
||||
<details>
|
||||
<summary>Why can an invalid message still get a successful MQTT acknowledgement?</summary>
|
||||
|
||||
MQTT receipt and application validity are separate. With the early automatic-acknowledgement helper, returning from the callback allows acknowledgement even when later validation rejects the queued message. Record rejection in application evidence; do not report the PUBACK as successful business processing. Only respond to a command after establishing a safe, authorized response destination.
|
||||
**❓ Question:** **The broker delivers a sales report with quantity `-3`. Can the MQTT client acknowledge it even if the application rejects it?**
|
||||
|
||||
**💬 Answer:** Yes. In this helper, the callback can return and the MQTT client may acknowledge delivery. The backend then records the validation failure and rejects the sale. PUBACK confirms the MQTT delivery step; it does not mean the sale succeeded.
|
||||
|
||||
</details>
|
||||
|
||||
## 📋 1. Create the contract package
|
||||
|
||||
> 💡 **Story question:** What shared vocabulary lets the backend and device agree on the shape and meaning of each message?
|
||||
> **❓ Question:** **What shared vocabulary lets the backend and device agree on the shape and meaning of each message?**
|
||||
|
||||
The contract package defines those wire-level names and fields so neither client silently invents a different JSON shape.
|
||||
The contract package gives both programs the same field names and meanings. For example, if the simulator sends `device` while the backend expects `device_id`, the message may be rejected or misunderstood.
|
||||
|
||||
```sh
|
||||
mkdir -p internal/contracts
|
||||
```
|
||||
|
||||
Create `internal/contracts/messages.go`, `topics.go`, `decode.go`, and `decode_test.go`. Edit the existing backend and simulator files to use this package. These are exercises: implement the functions described below; this lesson does not supply a complete replacement program.
|
||||
The complete implementation for this step is shown below. Save each block at its stated path, then update the backend and simulator to use these contracts before they handle or publish messages.
|
||||
|
||||
Define JSON structs for Heartbeat, Command, CommandResult, and SalesReport. Use `schema_version`, `message_id`, `device_id`, and `occurred_at` consistently. Command also needs `command_id` and `command`; CommandResult needs the same command ID and a status. A sales report needs a stable report ID and synthetic line items with positive quantities and integer minor-unit amounts. Keep command identity stable across retries; an MQTT packet identifier is not that identity.
|
||||
Save the first block as `internal/contracts/messages.go`. It defines the versioned messages, topic parser, and value checks. A command ID remains stable across retries; an MQTT packet identifier identifies only one protocol exchange.
|
||||
|
||||
```go
|
||||
package contracts
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const SchemaVersion = 1
|
||||
|
||||
// Kind values match the final topic segment used to route each message.
|
||||
type Kind string
|
||||
|
||||
const (
|
||||
KindTelemetry Kind = "telemetry"
|
||||
KindCommands Kind = "commands"
|
||||
KindResponses Kind = "responses"
|
||||
KindSales Kind = "sales"
|
||||
)
|
||||
|
||||
// These types define the JSON fields shared by backend and simulator.
|
||||
type Heartbeat struct {
|
||||
SchemaVersion int `json:"schema_version"`
|
||||
MessageID string `json:"message_id"`
|
||||
DeviceID string `json:"device_id"`
|
||||
OccurredAt string `json:"occurred_at"`
|
||||
Temperature *float64 `json:"temperature,omitempty"`
|
||||
}
|
||||
|
||||
type Command struct {
|
||||
SchemaVersion int `json:"schema_version"`
|
||||
MessageID string `json:"message_id"`
|
||||
DeviceID string `json:"device_id"`
|
||||
OccurredAt string `json:"occurred_at"`
|
||||
CommandID string `json:"command_id"`
|
||||
Command string `json:"command"`
|
||||
}
|
||||
|
||||
type CommandResult struct {
|
||||
SchemaVersion int `json:"schema_version"`
|
||||
MessageID string `json:"message_id"`
|
||||
DeviceID string `json:"device_id"`
|
||||
OccurredAt string `json:"occurred_at"`
|
||||
CommandID string `json:"command_id"`
|
||||
Status string `json:"status"`
|
||||
Detail string `json:"detail,omitempty"`
|
||||
}
|
||||
|
||||
type SalesLine struct {
|
||||
SKU string `json:"sku"`
|
||||
Quantity int64 `json:"quantity"`
|
||||
AmountMinor int64 `json:"amount_minor"`
|
||||
}
|
||||
|
||||
type SalesReport struct {
|
||||
SchemaVersion int `json:"schema_version"`
|
||||
ReportID string `json:"report_id"`
|
||||
DeviceID string `json:"device_id"`
|
||||
OccurredAt string `json:"occurred_at"`
|
||||
Items []SalesLine `json:"items"`
|
||||
}
|
||||
|
||||
// validID limits identifiers to safe, single-segment topic values.
|
||||
func validID(value string) bool {
|
||||
if value == "" || len(value) > 128 {
|
||||
return false
|
||||
}
|
||||
for _, r := range value {
|
||||
if !(r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9' || strings.ContainsRune("._-", r)) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// validateEnvelope checks fields shared by every versioned message.
|
||||
func validateEnvelope(topic Topic, version int, messageID, deviceID, occurredAt string, now time.Time) error {
|
||||
if version != SchemaVersion {
|
||||
return fmt.Errorf("unsupported schema_version %d", version)
|
||||
}
|
||||
if !validID(messageID) || !validID(deviceID) {
|
||||
return fmt.Errorf("message_id and device_id must be valid IDs")
|
||||
}
|
||||
if topic.DeviceID != deviceID {
|
||||
return fmt.Errorf("topic device %q does not match payload device_id %q", topic.DeviceID, deviceID)
|
||||
}
|
||||
stamp, err := time.Parse(time.RFC3339, occurredAt)
|
||||
if err != nil {
|
||||
return fmt.Errorf("occurred_at must be RFC3339: %w", err)
|
||||
}
|
||||
if stamp.After(now.Add(30 * time.Second)) {
|
||||
return fmt.Errorf("occurred_at is more than 30 seconds in the future")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Each Validate method adds checks specific to one message kind.
|
||||
func (m Heartbeat) Validate(topic Topic, now time.Time) error {
|
||||
if topic.Kind != KindTelemetry {
|
||||
return fmt.Errorf("heartbeat requires telemetry topic")
|
||||
}
|
||||
if err := validateEnvelope(topic, m.SchemaVersion, m.MessageID, m.DeviceID, m.OccurredAt, now); err != nil {
|
||||
return err
|
||||
}
|
||||
if m.Temperature != nil && (*m.Temperature < -50 || *m.Temperature > 100) {
|
||||
return fmt.Errorf("temperature is outside the supported range")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m Command) Validate(topic Topic, now time.Time) error {
|
||||
if topic.Kind != KindCommands {
|
||||
return fmt.Errorf("command requires commands topic")
|
||||
}
|
||||
if err := validateEnvelope(topic, m.SchemaVersion, m.MessageID, m.DeviceID, m.OccurredAt, now); err != nil {
|
||||
return err
|
||||
}
|
||||
if !validID(m.CommandID) || m.Command != "ping" {
|
||||
return fmt.Errorf("command_id is invalid or command is unsupported")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m CommandResult) Validate(topic Topic, now time.Time) error {
|
||||
if topic.Kind != KindResponses {
|
||||
return fmt.Errorf("command result requires responses topic")
|
||||
}
|
||||
if err := validateEnvelope(topic, m.SchemaVersion, m.MessageID, m.DeviceID, m.OccurredAt, now); err != nil {
|
||||
return err
|
||||
}
|
||||
if !validID(m.CommandID) || (m.Status != "accepted" && m.Status != "completed" && m.Status != "rejected") {
|
||||
return fmt.Errorf("command result has invalid command_id or status")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m SalesReport) Validate(topic Topic, now time.Time) error {
|
||||
if topic.Kind != KindSales {
|
||||
return fmt.Errorf("sales report requires sales topic")
|
||||
}
|
||||
if err := validateEnvelope(topic, m.SchemaVersion, m.ReportID, m.DeviceID, m.OccurredAt, now); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(m.Items) == 0 || len(m.Items) > 100 {
|
||||
return fmt.Errorf("sales report must contain 1 to 100 items")
|
||||
}
|
||||
for i, item := range m.Items {
|
||||
if !validID(item.SKU) || item.Quantity <= 0 || item.AmountMinor < 0 {
|
||||
return fmt.Errorf("item %d has invalid SKU, quantity, or amount_minor", i)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ParseTopic turns broker routing metadata into values handlers can compare.
|
||||
func ParseTopic(value string) (Topic, error) {
|
||||
parts := strings.Split(value, "/")
|
||||
if len(parts) != 5 || parts[0] != "lab" || parts[1] != "v1" || parts[2] != "devices" || !validID(parts[3]) {
|
||||
return Topic{}, fmt.Errorf("topic must be lab/v1/devices/{deviceID}/{kind}")
|
||||
}
|
||||
kind := Kind(parts[4])
|
||||
switch kind {
|
||||
case KindTelemetry, KindCommands, KindResponses, KindSales:
|
||||
return Topic{DeviceID: parts[3], Kind: kind}, nil
|
||||
default:
|
||||
return Topic{}, fmt.Errorf("unsupported topic kind %q", kind)
|
||||
}
|
||||
}
|
||||
|
||||
// Topic is the parsed device scope and message kind from an MQTT topic.
|
||||
type Topic struct {
|
||||
DeviceID string
|
||||
Kind Kind
|
||||
}
|
||||
```
|
||||
|
||||
`internal/contracts/decode.go` bounds the payload before decoding, rejects unknown fields, requires the JSON value to consume the entire input, and calls the decoded contract's `Validate` method before returning it. `MQTTMessage` restricts the generic decoder to types with that method. The parsed topic and clock are explicit inputs so validation can check route/body identity and remain deterministic in tests.
|
||||
|
||||
```go
|
||||
package contracts
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"time"
|
||||
)
|
||||
|
||||
const MaxPayloadBytes = 16 * 1024
|
||||
|
||||
// MQTTMessage is implemented by each versioned message contract.
|
||||
type MQTTMessage interface {
|
||||
Validate(topic Topic, now time.Time) error
|
||||
}
|
||||
|
||||
// Decode bounds input, rejects unknown fields, requires one JSON value,
|
||||
// and validates it against the topic before returning it.
|
||||
func Decode[T MQTTMessage](payload []byte, topic Topic, now time.Time) (T, error) {
|
||||
var value T
|
||||
if len(payload) > MaxPayloadBytes {
|
||||
return value, fmt.Errorf("payload exceeds %d bytes", MaxPayloadBytes)
|
||||
}
|
||||
decoder := json.NewDecoder(bytes.NewReader(payload))
|
||||
decoder.DisallowUnknownFields()
|
||||
if err := decoder.Decode(&value); err != nil {
|
||||
return value, fmt.Errorf("decode JSON: %w", err)
|
||||
}
|
||||
var trailing any
|
||||
// Decode reads one JSON value at a time. A second call distinguishes
|
||||
// whitespace (EOF) from a second value or malformed trailing bytes.
|
||||
if err := decoder.Decode(&trailing); err != io.EOF {
|
||||
if err == nil {
|
||||
return value, fmt.Errorf("payload contains more than one JSON value")
|
||||
}
|
||||
return value, fmt.Errorf("trailing data after JSON value: %w", err)
|
||||
}
|
||||
if err := value.Validate(topic, now); err != nil {
|
||||
return value, fmt.Errorf("validate message: %w", err)
|
||||
}
|
||||
return value, nil
|
||||
}
|
||||
```
|
||||
|
||||
`internal/contracts/decode_test.go` checks malformed boundaries without a broker:
|
||||
|
||||
```go
|
||||
package contracts
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Topic tests reject malformed routes and unsupported message kinds.
|
||||
func TestParseTopic(t *testing.T) {
|
||||
topic, err := ParseTopic("lab/v1/devices/device-001/sales")
|
||||
if err != nil { t.Fatal(err) }
|
||||
if topic.DeviceID != "device-001" || topic.Kind != KindSales {
|
||||
t.Fatalf("unexpected topic: %#v", topic)
|
||||
}
|
||||
for _, invalid := range []string{"lab/v1/devices/+/sales", "lab/v1/devices/device-001/unknown", "lab/v1/devices/device-001/sales/extra"} {
|
||||
if _, err := ParseTopic(invalid); err == nil { t.Errorf("ParseTopic(%q) succeeded", invalid) }
|
||||
}
|
||||
}
|
||||
|
||||
// These cases cover strict JSON shape and single-value decoding.
|
||||
func TestDecodeRejectsUnknownAndTrailingValues(t *testing.T) {
|
||||
topic := Topic{DeviceID: "device-001", Kind: KindTelemetry}
|
||||
now := time.Date(2026, 10, 13, 12, 0, 0, 0, time.UTC)
|
||||
if _, err := Decode[Heartbeat]([]byte(`{"schema_version":1,"unknown":true}`), topic, now); err == nil {
|
||||
t.Fatal("unknown field was accepted")
|
||||
}
|
||||
if _, err := Decode[Heartbeat]([]byte(`{"schema_version":1,"message_id":"m-1","device_id":"device-001","occurred_at":"2026-10-13T12:00:00Z"} {"message_id":"m-2"}`), topic, now); err == nil {
|
||||
t.Fatal("trailing JSON value was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
// This case ensures the application limit is applied before decoding.
|
||||
func TestDecodeRejectsOversizedPayload(t *testing.T) {
|
||||
payload := []byte(`"` + strings.Repeat("x", MaxPayloadBytes) + `"`)
|
||||
if _, err := Decode[Heartbeat](payload, topic, now); err == nil {
|
||||
t.Fatal("oversized payload was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
// A body cannot override the device selected by the topic.
|
||||
func TestCommandRejectsTopicIdentityMismatch(t *testing.T) {
|
||||
topic, err := ParseTopic("lab/v1/devices/device-001/commands")
|
||||
if err != nil { t.Fatal(err) }
|
||||
command := Command{SchemaVersion: SchemaVersion, MessageID: "m-1", DeviceID: "device-002", OccurredAt: time.Now().UTC().Format(time.RFC3339), CommandID: "c-1", Command: "ping"}
|
||||
if err := command.Validate(topic, time.Now().UTC()); err == nil {
|
||||
t.Fatal("topic/body identity mismatch was accepted")
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🏷️ 2. Parse topics before routing
|
||||
|
||||
> 💡 **Story question:** A message arrives with both a topic and a payload. Why decide which device and message kind the topic represents before running a handler?
|
||||
> **❓ Question:** **A message arrives with both a topic and a payload. Why decide which device and message kind the topic represents before running a handler?**
|
||||
|
||||
Parsing the topic establishes the intended route from broker metadata; it also gives validation a value to compare against any device identity claimed in the payload.
|
||||
|
||||
Implement a parser accepting exactly `lab/v1/devices/{deviceID}/{kind}`. Initially permit telemetry, commands, responses, and sales. Reject empty device IDs, extra segments, wildcards, and unknown kinds. Return a typed device ID and kind rather than making every handler split strings itself. Extend the lab ACL explicitly for sales only when you start publishing reports.
|
||||
`contracts.ParseTopic` in `messages.go` accepts exactly `lab/v1/devices/{deviceID}/{kind}`. It rejects empty IDs, extra segments, wildcard characters, and unknown message kinds. Call it before decoding so the validator can compare the route's device ID with the payload claim. Add the sales ACL rules only when you publish sales reports.
|
||||
|
||||
## ✅ 3. Decode and validate
|
||||
|
||||
> 💡 **Story question:** JSON decoding succeeded. What else must be true before this message is safe for application state?
|
||||
> **❓ Question:** **JSON decoding succeeded. What else must be true before this message is safe for application state?**
|
||||
|
||||
Structure is only one check. Bound the input, enforce the supported schema and required IDs, validate timestamps and values, and compare topic identity with the body claim.
|
||||
**💬 Answer:** Decoding checks syntax, not whether the message is safe to use. The backend also checks size, schema, required IDs, timestamps, values, and agreement between the topic and claimed device before business handling.
|
||||
|
||||
Apply a 16 KiB payload limit before decoding. Use `json.Decoder` with `DisallowUnknownFields`, then require EOF after the first JSON value. Validate schema version 1, required IDs, supported commands, topic/body device agreement, RFC3339 timestamps, and field lengths. Choose and document a future-clock tolerance, such as 30 seconds; inject the current time into validation to keep tests deterministic. Historical sales may legitimately be old, so do not apply heartbeat freshness rules to every report.
|
||||
`contracts.Decode[T]` in `decode.go` applies the 16 KiB payload limit, rejects unknown JSON fields, requires EOF after one value, and calls `Validate(topic, now)` before returning. `MQTTMessage` restricts `T` to contract types that implement validation. Validation checks schema version 1, required IDs, supported commands, topic/body device agreement, RFC3339 timestamps, and message-specific values. Passing `now` explicitly lets callers use a fixed clock in tests. Historical sales are allowed; the validator does not apply heartbeat freshness rules to reports.
|
||||
|
||||
Treat Content Type and Payload Format Indicator as descriptive metadata. Validate bytes even when they claim to contain JSON. A payload device ID is a claim; broker credentials and ACLs supply the separate authorization boundary.
|
||||
Metadata describes the payload, and the ID in it is only a claim. The backend validates the bytes and compares that ID with the topic. Broker credentials and ACLs separately authorize which sender may publish to the topic.
|
||||
|
||||
## 🧪 4. Integrate and verify
|
||||
|
||||
> 💡 **Story question:** Where should validation sit so malformed messages cannot trigger a response or change state?
|
||||
> **❓ Question:** **Where should validation sit so malformed messages cannot trigger a response or change state?**
|
||||
|
||||
Run it between transport reception and business handling. Verify both accepted messages and rejection cases, and inspect broker ACLs separately when testing sender permissions.
|
||||
Run validation after receiving the message but before business handling. Otherwise, a malformed command could already change state or trigger a reply. Test accepted and rejected messages, then test broker ACL permissions separately; application validation does not decide who may publish to a topic.
|
||||
|
||||
Replace Section 4's anonymous JSON structs in both programs with these contracts. Validate before updating state or sending a result. Log the error category and bounded identifiers, without dumping rejected payloads. Unsupported or malformed commands must have no simulated effect.
|
||||
For one sales message, the processing order is:
|
||||
|
||||
```go
|
||||
topic, err := contracts.ParseTopic(message.Topic)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
report, err := contracts.Decode[contracts.SalesReport](message.Payload, topic, time.Now().UTC())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// Only a validated report reaches the business handler.
|
||||
```
|
||||
|
||||
`Decode` reads one JSON value per call. After decoding the expected report, it calls `Decode` again: `io.EOF` means only whitespace remained, another successful decode means there was a second JSON value, and a different error means malformed trailing bytes. Only after that structural check does it call the report's `Validate` method.
|
||||
|
||||
Replace Section 4's anonymous JSON structs in both programs with these contracts. The complete executable files are below; each validates inbound data before logging, changing state, or sending a result. Rejections log a bounded error, not the full payload.
|
||||
|
||||
`cmd/backend/main.go`:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"example.com/mqtt5-learning/internal/config"
|
||||
"example.com/mqtt5-learning/internal/contracts"
|
||||
"example.com/mqtt5-learning/internal/mqtt"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if err := run(); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func run() error {
|
||||
cfg, err := config.Load(os.Args[1:])
|
||||
if err != nil {
|
||||
return fmt.Errorf("load config: %w", err)
|
||||
}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
// Start subscriptions before handling any delivered message.
|
||||
inbox := make(chan mqtt.Message, 16)
|
||||
manager, err := mqtt.Start(ctx, cfg, []string{"lab/v1/devices/+/telemetry", "lab/v1/devices/+/responses", "lab/v1/devices/+/sales"}, inbox)
|
||||
if err != nil {
|
||||
return fmt.Errorf("start MQTT: %w", err)
|
||||
}
|
||||
defer mqtt.Stop(manager)
|
||||
|
||||
ticker := time.NewTicker(10 * time.Second)
|
||||
defer ticker.Stop()
|
||||
sequence := 0
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil
|
||||
case message := <-inbox:
|
||||
if err := handleMessage(message); err != nil {
|
||||
log.Printf("rejected topic=%q reason=%v", message.Topic, err)
|
||||
}
|
||||
case <-ticker.C:
|
||||
sequence++
|
||||
id := fmt.Sprintf("ping-%d-%d", time.Now().UTC().Unix(), sequence)
|
||||
command := contracts.Command{SchemaVersion: contracts.SchemaVersion, MessageID: id, DeviceID: cfg.MQTT.DeviceID, OccurredAt: time.Now().UTC().Format(time.RFC3339), CommandID: id, Command: "ping"}
|
||||
payload, err := json.Marshal(command)
|
||||
if err != nil {
|
||||
return fmt.Errorf("encode command: %w", err)
|
||||
}
|
||||
if err := mqtt.Publish(ctx, manager, "lab/v1/devices/"+cfg.MQTT.DeviceID+"/commands", payload); err != nil {
|
||||
log.Printf("command publish failed or uncertain id=%s: %v", id, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func handleMessage(message mqtt.Message) error {
|
||||
// Parse and validate before a message can reach application handling.
|
||||
topic, err := contracts.ParseTopic(message.Topic)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
switch topic.Kind {
|
||||
case contracts.KindTelemetry:
|
||||
value, err := contracts.Decode[contracts.Heartbeat](message.Payload, topic, now)
|
||||
if err != nil { return err }
|
||||
log.Printf("valid heartbeat device=%s message=%s", value.DeviceID, value.MessageID)
|
||||
case contracts.KindResponses:
|
||||
value, err := contracts.Decode[contracts.CommandResult](message.Payload, topic, now)
|
||||
if err != nil { return err }
|
||||
log.Printf("command result device=%s command=%s status=%s", value.DeviceID, value.CommandID, value.Status)
|
||||
case contracts.KindSales:
|
||||
value, err := contracts.Decode[contracts.SalesReport](message.Payload, topic, now)
|
||||
if err != nil { return err }
|
||||
log.Printf("valid sales report device=%s report=%s items=%d", value.DeviceID, value.ReportID, len(value.Items))
|
||||
default:
|
||||
return fmt.Errorf("backend does not consume topic kind %q", topic.Kind)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
`cmd/simulator/main.go`:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"example.com/mqtt5-learning/internal/config"
|
||||
"example.com/mqtt5-learning/internal/contracts"
|
||||
"example.com/mqtt5-learning/internal/mqtt"
|
||||
"github.com/eclipse/paho.golang/autopaho"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if err := run(); err != nil { log.Fatal(err) }
|
||||
}
|
||||
|
||||
func run() error {
|
||||
cfg, err := config.Load(os.Args[1:])
|
||||
if err != nil { return fmt.Errorf("load config: %w", err) }
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
// Publish this device's heartbeat and subscribe only to its command topic.
|
||||
base := "lab/v1/devices/" + cfg.MQTT.DeviceID
|
||||
inbox := make(chan mqtt.Message, 16)
|
||||
manager, err := mqtt.Start(ctx, cfg, []string{base+"/commands"}, inbox)
|
||||
if err != nil { return fmt.Errorf("start MQTT: %w", err) }
|
||||
defer mqtt.Stop(manager)
|
||||
ticker := time.NewTicker(3 * time.Second)
|
||||
defer ticker.Stop()
|
||||
sequence := 0
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil
|
||||
case <-ticker.C:
|
||||
sequence++
|
||||
heartbeat := contracts.Heartbeat{SchemaVersion: contracts.SchemaVersion, MessageID: fmt.Sprintf("heartbeat-%d-%d", time.Now().Unix(), sequence), DeviceID: cfg.MQTT.DeviceID, OccurredAt: time.Now().UTC().Format(time.RFC3339)}
|
||||
payload, err := json.Marshal(heartbeat)
|
||||
if err != nil { return fmt.Errorf("encode heartbeat: %w", err) }
|
||||
if err := mqtt.Publish(ctx, manager, base+"/telemetry", payload); err != nil { log.Printf("heartbeat publish: %v", err) }
|
||||
case message := <-inbox:
|
||||
if message.Retain { log.Printf("reject retained command topic=%q", message.Topic); continue }
|
||||
if err := respond(ctx, manager, cfg.MQTT.DeviceID, message); err != nil { log.Printf("reject command topic=%q reason=%v", message.Topic, err) }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func respond(ctx context.Context, manager *autopaho.ConnectionManager, deviceID string, message mqtt.Message) error {
|
||||
// Check route and body identity before constructing a command result.
|
||||
topic, err := contracts.ParseTopic(message.Topic)
|
||||
if err != nil { return err }
|
||||
command, err := contracts.Decode[contracts.Command](message.Payload, topic, time.Now().UTC())
|
||||
if err != nil { return err }
|
||||
if topic.DeviceID != deviceID { return fmt.Errorf("command is for another device") }
|
||||
result := contracts.CommandResult{SchemaVersion: contracts.SchemaVersion, MessageID: "result-"+command.CommandID, DeviceID: deviceID, OccurredAt: time.Now().UTC().Format(time.RFC3339), CommandID: command.CommandID, Status: "completed"}
|
||||
payload, err := json.Marshal(result)
|
||||
if err != nil { return fmt.Errorf("encode result: %w", err) }
|
||||
return mqtt.Publish(ctx, manager, "lab/v1/devices/"+deviceID+"/responses", payload)
|
||||
}
|
||||
```
|
||||
|
||||
```sh
|
||||
gofmt -w internal/contracts cmd
|
||||
@@ -114,11 +579,11 @@ go test ./internal/contracts
|
||||
go test ./...
|
||||
```
|
||||
|
||||
`mkdir -p` creates the package directory; you still create the listed `.go` files yourself. `gofmt -w` rewrites source formatting, `go test ./internal/contracts` isolates contract tests, and `go test ./...` checks the whole evolving module for integration regressions. Go discovers test functions named `Test...` in `_test.go` files. No broker is needed for pure parser/validator tests.
|
||||
`mkdir -p` creates the package directory. `gofmt -w` formats the complete files above. `go test ./internal/contracts` runs the supplied pure parser/validator tests without a broker; `go test ./...` checks the rest of the application too.
|
||||
|
||||
When adding sales, edit `lab/mosquitto/acl`: add `topic read lab/v1/devices/+/sales` under backend and `topic write lab/v1/devices/device-001/sales` under simulator. Restart the lab broker to read the changed ACL, check logs, and test allowed and cross-device-denied publications. Permissions enable a topic direction; they do not replace sales validation.
|
||||
|
||||
Test valid messages, malformed JSON, trailing JSON, unknown fields, missing IDs, unsupported versions, identity mismatch, unknown commands, invalid timestamps, and payloads exactly at and above the limit. Keep the simulator and backend contracts in sync when changing the wire format.
|
||||
The supplied tests cover topic parsing, malformed JSON boundaries, payload size, and topic/body identity. Add a case when you change a contract so the test captures the new rule. Keep the simulator and backend contracts in sync when changing the wire format.
|
||||
|
||||
**Done when:** Both programs still exchange ping/pong, every rejection has a predictable reason, and invalid sales or commands leave state unchanged. Explain why schema validation does not authenticate a device.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user