130 lines
9.9 KiB
Markdown
130 lines
9.9 KiB
Markdown
# 📋 Section 5 — Design contracts and validate inbound messages
|
|
|
|
## 🎬 Start with the system story
|
|
|
|
> 💡 **Story question:** The broker can route a message because its topic matches. How does the Go backend decide that the topic and payload describe a valid message from the expected device?
|
|
|
|
Routing is not validation or identity proof. A message arrives as a topic plus bytes; the application must parse the route, decode the payload, and check that fields and identity make sense before changing state. A device ID inside JSON is a claim, so broker authentication and topic authorization remain separate safeguards.
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
B[Broker routes matching topic] --> P[Paho delivers topic + bytes]
|
|
P --> T[Parse topic and device scope]
|
|
T --> J[Decode versioned JSON]
|
|
J --> V[Validate identity, IDs, size, time]
|
|
V -->|valid| H[Business handler]
|
|
V -->|invalid| E[Safe rejection / metric]
|
|
```
|
|
|
|
**Before each implementation step, ask:** What can be trusted at this boundary, and what must be checked before the application acts? Follow [Lesson 0](00-system-lifecycle.md) for the complete message path.
|
|
|
|
Prerequisite: Section 4 exchanges heartbeats and ping/pong. The clients share the disposable password in `configs/lab.toml`; select the simulator account with `-username simulator` when starting it.
|
|
|
|
Run all commands from `/home/ryan/Media/Code/Code/robot-market/mqtt-5-learning`. Create files at the project-relative paths shown below, not under `lessons/`. Continue the same application and pinned Paho v0.23.0 dependency. These are guided implementation exercises; attempt each step before requesting a full solution.
|
|
|
|
## 📨 Follow one message through the trust boundaries
|
|
|
|
Suppose the backend receives topic `lab/v1/devices/device-001/sales` with a body claiming `device_id: device-002`. The broker may have correctly authorized the sender to publish to the topic. It has not established that the JSON is truthful. Reject the mismatch before selecting a device record or changing a balance.
|
|
|
|
```text
|
|
PUBLISH → size limit → topic parser → JSON decoder → schema checks
|
|
→ topic/body identity check → business validation → handler
|
|
```
|
|
|
|
Each step answers a different question. The size limit bounds decoding cost. Parsing establishes the intended route. JSON decoding proves structure only. Business validation rejects a syntactically valid negative quantity. None of these replaces broker authentication.
|
|
|
|
The 16 KiB application limit applies to `Payload`, not the entire MQTT packet. Topic and properties add bytes; the library has already received the packet before this validator runs. Lesson 16 adds protocol packet-size limits to bound an earlier layer.
|
|
|
|
### 📨 Map encoding metadata into Paho
|
|
|
|
This fragment belongs where you build a `paho.Publish`; `payload` is the JSON produced by your contract encoder:
|
|
|
|
```go
|
|
utf8Payload := byte(1)
|
|
|
|
// Describe the payload's encoding and media type for MQTT 5 consumers.
|
|
properties := &paho.PublishProperties{
|
|
ContentType: "application/json",
|
|
PayloadFormat: &utf8Payload,
|
|
}
|
|
// Attach these properties to the PUBLISH packet built by your transport helper.
|
|
_ = properties
|
|
```
|
|
|
|
Payload Format Indicator 1 declares UTF-8 character data. It does not mean JSON, schema version 1, or trusted content. Content Type describes the media type. For example, `hello` is UTF-8 but fails your JSON object contract. Property pointers distinguish an omitted property from an explicitly supplied value; inspect nil before dereferencing incoming fields.
|
|
|
|
Keep these distinctions in your transport type when needed: packet metadata belongs alongside payload bytes, while `schema_version` and `command_id` belong to your application contract. Do not make handlers depend on a mutable packet owned elsewhere merely to read metadata.
|
|
|
|
### 🧪 Predict, modify, explain
|
|
|
|
Send a valid heartbeat, then change only its body device ID, then change only its schema version. Predict which validation stage rejects each. Finally publish valid JSON with an extra trailing object: one successful decoder call is insufficient unless you also require EOF.
|
|
|
|
<details>
|
|
<summary>Why can an invalid message still get a successful MQTT acknowledgement?</summary>
|
|
|
|
MQTT receipt and application validity are separate. With the early automatic-acknowledgement helper, returning from the callback allows acknowledgement even when later validation rejects the queued message. Record rejection in application evidence; do not report the PUBACK as successful business processing. Only respond to a command after establishing a safe, authorized response destination.
|
|
|
|
</details>
|
|
|
|
## 📋 1. Create the contract package
|
|
|
|
> 💡 **Story question:** What shared vocabulary lets the backend and device agree on the shape and meaning of each message?
|
|
|
|
The contract package defines those wire-level names and fields so neither client silently invents a different JSON shape.
|
|
|
|
```sh
|
|
mkdir -p internal/contracts
|
|
```
|
|
|
|
Create `internal/contracts/messages.go`, `topics.go`, `decode.go`, and `decode_test.go`. Edit the existing backend and simulator files to use this package. These are exercises: implement the functions described below; this lesson does not supply a complete replacement program.
|
|
|
|
Define JSON structs for Heartbeat, Command, CommandResult, and SalesReport. Use `schema_version`, `message_id`, `device_id`, and `occurred_at` consistently. Command also needs `command_id` and `command`; CommandResult needs the same command ID and a status. A sales report needs a stable report ID and synthetic line items with positive quantities and integer minor-unit amounts. Keep command identity stable across retries; an MQTT packet identifier is not that identity.
|
|
|
|
## 🏷️ 2. Parse topics before routing
|
|
|
|
> 💡 **Story question:** A message arrives with both a topic and a payload. Why decide which device and message kind the topic represents before running a handler?
|
|
|
|
Parsing the topic establishes the intended route from broker metadata; it also gives validation a value to compare against any device identity claimed in the payload.
|
|
|
|
Implement a parser accepting exactly `lab/v1/devices/{deviceID}/{kind}`. Initially permit telemetry, commands, responses, and sales. Reject empty device IDs, extra segments, wildcards, and unknown kinds. Return a typed device ID and kind rather than making every handler split strings itself. Extend the lab ACL explicitly for sales only when you start publishing reports.
|
|
|
|
## ✅ 3. Decode and validate
|
|
|
|
> 💡 **Story question:** JSON decoding succeeded. What else must be true before this message is safe for application state?
|
|
|
|
Structure is only one check. Bound the input, enforce the supported schema and required IDs, validate timestamps and values, and compare topic identity with the body claim.
|
|
|
|
Apply a 16 KiB payload limit before decoding. Use `json.Decoder` with `DisallowUnknownFields`, then require EOF after the first JSON value. Validate schema version 1, required IDs, supported commands, topic/body device agreement, RFC3339 timestamps, and field lengths. Choose and document a future-clock tolerance, such as 30 seconds; inject the current time into validation to keep tests deterministic. Historical sales may legitimately be old, so do not apply heartbeat freshness rules to every report.
|
|
|
|
Treat Content Type and Payload Format Indicator as descriptive metadata. Validate bytes even when they claim to contain JSON. A payload device ID is a claim; broker credentials and ACLs supply the separate authorization boundary.
|
|
|
|
## 🧪 4. Integrate and verify
|
|
|
|
> 💡 **Story question:** Where should validation sit so malformed messages cannot trigger a response or change state?
|
|
|
|
Run it between transport reception and business handling. Verify both accepted messages and rejection cases, and inspect broker ACLs separately when testing sender permissions.
|
|
|
|
Replace Section 4's anonymous JSON structs in both programs with these contracts. Validate before updating state or sending a result. Log the error category and bounded identifiers, without dumping rejected payloads. Unsupported or malformed commands must have no simulated effect.
|
|
|
|
```sh
|
|
gofmt -w internal/contracts cmd
|
|
|
|
go test ./internal/contracts
|
|
|
|
go test ./...
|
|
```
|
|
|
|
`mkdir -p` creates the package directory; you still create the listed `.go` files yourself. `gofmt -w` rewrites source formatting, `go test ./internal/contracts` isolates contract tests, and `go test ./...` checks the whole evolving module for integration regressions. Go discovers test functions named `Test...` in `_test.go` files. No broker is needed for pure parser/validator tests.
|
|
|
|
When adding sales, edit `lab/mosquitto/acl`: add `topic read lab/v1/devices/+/sales` under backend and `topic write lab/v1/devices/device-001/sales` under simulator. Restart the lab broker to read the changed ACL, check logs, and test allowed and cross-device-denied publications. Permissions enable a topic direction; they do not replace sales validation.
|
|
|
|
Test valid messages, malformed JSON, trailing JSON, unknown fields, missing IDs, unsupported versions, identity mismatch, unknown commands, invalid timestamps, and payloads exactly at and above the limit. Keep the simulator and backend contracts in sync when changing the wire format.
|
|
|
|
**Done when:** Both programs still exchange ping/pong, every rejection has a predictable reason, and invalid sales or commands leave state unchanged. Explain why schema validation does not authenticate a device.
|
|
|
|
## ✅ Result — the story so far
|
|
|
|
The backend and simulator now have a shared vocabulary for heartbeats, commands, results, and sales reports. When a message arrives, the receiver checks its topic, size, JSON shape, schema version, identifiers, timestamp, and topic/body device match before changing application state. A broker-routed message can still be rejected by the application; successful MQTT delivery alone does not make its contents valid or trusted.
|
|
|
|
Previous: [04-publish-subscribe-simulator.md](04-publish-subscribe-simulator.md). Next: [06-concurrency-cancellation.md](06-concurrency-cancellation.md).
|