first commit

lessons/05-contracts-validation.md

@@ -0,0 +1,86 @@
+# Section 5 — Design contracts and validate inbound messages
+
+Prerequisite: Section 4 exchanges heartbeats and ping/pong.
+
+Run all commands from `/home/ryan/Media/Code/Code/robot-market/mqtt-5-learning`. Create files at the project-relative paths shown below, not under `lessons/`. Continue the same application and pinned Paho v0.23.0 dependency. These are guided implementation exercises; attempt each step before requesting a full solution.
+
+## Follow one message through the trust boundaries
+
+Suppose the backend receives topic `lab/v1/devices/device-001/sales` with a body claiming `device_id: device-002`. The broker may have correctly authorized the sender to publish to the topic. It has not established that the JSON is truthful. Reject the mismatch before selecting a device record or changing a balance.
+
+```text
+PUBLISH → size limit → topic parser → JSON decoder → schema checks
+        → topic/body identity check → business validation → handler
+```
+
+Each step answers a different question. The size limit bounds decoding cost. Parsing establishes the intended route. JSON decoding proves structure only. Business validation rejects a syntactically valid negative quantity. None of these replaces broker authentication.
+
+The 16 KiB application limit applies to `Payload`, not the entire MQTT packet. Topic and properties add bytes; the library has already received the packet before this validator runs. Lesson 16 adds protocol packet-size limits to bound an earlier layer.
+
+### Map encoding metadata into Paho
+
+This fragment belongs where you build a `paho.Publish`; `payload` is the JSON produced by your contract encoder:
+
+```go
+utf8Payload := byte(1)
+properties := &paho.PublishProperties{
+    ContentType:   "application/json",
+    PayloadFormat: &utf8Payload,
+}
+_ = properties // attach as Publish.Properties in your implementation
+```
+
+Payload Format Indicator 1 declares UTF-8 character data. It does not mean JSON, schema version 1, or trusted content. Content Type describes the media type. For example, `hello` is UTF-8 but fails your JSON object contract. Property pointers distinguish an omitted property from an explicitly supplied value; inspect nil before dereferencing incoming fields.
+
+Keep these distinctions in your transport type when needed: packet metadata belongs alongside payload bytes, while `schema_version` and `command_id` belong to your application contract. Do not make handlers depend on a mutable packet owned elsewhere merely to read metadata.
+
+### Predict, modify, explain
+
+Send a valid heartbeat, then change only its body device ID, then change only its schema version. Predict which validation stage rejects each. Finally publish valid JSON with an extra trailing object: one successful decoder call is insufficient unless you also require EOF.
+
+<details>
+<summary>Why can an invalid message still get a successful MQTT acknowledgement?</summary>
+
+MQTT receipt and application validity are separate. With the early automatic-acknowledgement helper, returning from the callback allows acknowledgement even when later validation rejects the queued message. Record rejection in application evidence; do not report the PUBACK as successful business processing. Only respond to a command after establishing a safe, authorized response destination.
+
+</details>
+
+## 1. Create the contract package
+
+```sh
+mkdir -p internal/contracts
+```
+
+Create `internal/contracts/messages.go`, `topics.go`, `decode.go`, and `decode_test.go`. Edit the existing backend and simulator files to use this package. These are exercises: implement the functions described below; this lesson does not supply a complete replacement program.
+
+Define JSON structs for Heartbeat, Command, CommandResult, and SalesReport. Use `schema_version`, `message_id`, `device_id`, and `occurred_at` consistently. Command also needs `command_id` and `command`; CommandResult needs the same command ID and a status. A sales report needs a stable report ID and synthetic line items with positive quantities and integer minor-unit amounts. Keep command identity stable across retries; an MQTT packet identifier is not that identity.
+
+## 2. Parse topics before routing
+
+Implement a parser accepting exactly `lab/v1/devices/{deviceID}/{kind}`. Initially permit telemetry, commands, responses, and sales. Reject empty device IDs, extra segments, wildcards, and unknown kinds. Return a typed device ID and kind rather than making every handler split strings itself. Extend the lab ACL explicitly for sales only when you start publishing reports.
+
+## 3. Decode and validate
+
+Apply a 16 KiB payload limit before decoding. Use `json.Decoder` with `DisallowUnknownFields`, then require EOF after the first JSON value. Validate schema version 1, required IDs, supported commands, topic/body device agreement, RFC3339 timestamps, and field lengths. Choose and document a future-clock tolerance, such as 30 seconds; inject the current time into validation to keep tests deterministic. Historical sales may legitimately be old, so do not apply heartbeat freshness rules to every report.
+
+Treat Content Type and Payload Format Indicator as descriptive metadata. Validate bytes even when they claim to contain JSON. A payload device ID is a claim; broker credentials and ACLs supply the separate authorization boundary.
+
+## 4. Integrate and verify
+
+Replace Section 4's anonymous JSON structs in both programs with these contracts. Validate before updating state or sending a result. Log the error category and bounded identifiers, without dumping rejected payloads. Unsupported or malformed commands must have no simulated effect.
+
+```sh
+gofmt -w internal/contracts cmd
+go test ./internal/contracts
+go test ./...
+```
+
+`mkdir -p` creates the package directory; you still create the listed `.go` files yourself. `gofmt -w` rewrites source formatting, `go test ./internal/contracts` isolates contract tests, and `go test ./...` checks the whole evolving module for integration regressions. Go discovers test functions named `Test...` in `_test.go` files. No broker is needed for pure parser/validator tests.
+
+When adding sales, edit `lab/mosquitto/acl`: add `topic read lab/v1/devices/+/sales` under backend and `topic write lab/v1/devices/device-001/sales` under simulator. Restart the lab broker to read the changed ACL, check logs, and test allowed and cross-device-denied publications. Permissions enable a topic direction; they do not replace sales validation.
+
+Test valid messages, malformed JSON, trailing JSON, unknown fields, missing IDs, unsupported versions, identity mismatch, unknown commands, invalid timestamps, and payloads exactly at and above the limit. Keep the simulator and backend contracts in sync when changing the wire format.
+
+**Done when:** Both programs still exchange ping/pong, every rejection has a predictable reason, and invalid sales or commands leave state unchanged. Explain why schema validation does not authenticate a device.
+
+Previous: [04-publish-subscribe-simulator.md](04-publish-subscribe-simulator.md). Next: [06-concurrency-cancellation.md](06-concurrency-cancellation.md).