117 lines
10 KiB
Markdown
117 lines
10 KiB
Markdown
# Section 10 — Expiry, presence, and retained state
|
|
|
|
Prerequisite: complete the session experiments in Lesson 9. Continue the same lab and pinned Paho v0.23.0. These are incremental exercises; the fragments below belong in your existing transport and simulator, not new standalone programs.
|
|
|
|
## 1. Give every timer one responsibility
|
|
|
|
**❓ Question:** **A ping is queued while a device is offline, then reaches it after the operator has changed the desired setting. Which timer can stop stale delivery, and which check must still run on the device?**
|
|
|
|
**💬 Answer:** MQTT Message Expiry limits how long the broker may forward the publication. The device must also check the application's absolute deadline before starting the action, because work may have spent time in a local queue or arrived just before expiry. Neither timer can reverse an action already started.
|
|
|
|
| Timer | What expires | What it cannot prove |
|
|
| --- | --- | --- |
|
|
| Go publish context | Your wait for the publication operation | That the device never received it |
|
|
| Application command deadline | Permission to begin a stale action under your contract | That an already-started action was undone |
|
|
| MQTT Message Expiry | A publication's remaining delivery lifetime | That a delivered message stops executing |
|
|
| Session Expiry | Broker session after disconnect | That application records were saved |
|
|
| Keep Alive | Acceptable MQTT connection inactivity | That the application is healthy |
|
|
| Will Delay | Delay before an eligible Will is published | That the device is physically broken |
|
|
|
|
A vending configuration command that was useful five minutes ago may be wrong now. Use Message Expiry to limit stale delivery and an application deadline to reject stale execution after delivery. A response timeout merely tells the caller it lacks a result. A PUBACK can arrive before the simulator validates the command; if the application rejects it, that MQTT acknowledgement remains valid for the protocol step it acknowledged.
|
|
|
|
This fragment builds MQTT metadata for a five-second publication lifetime:
|
|
|
|
```go
|
|
expirySeconds := uint32(5)
|
|
properties := &paho.PublishProperties{
|
|
ContentType: "application/json",
|
|
MessageExpiry: &expirySeconds,
|
|
}
|
|
```
|
|
|
|
Attach `properties` to the outbound PUBLISH. A nil MessageExpiry omits the property; an explicit zero is not an unlimited lifetime. The broker reduces the remaining lifetime when forwarding queued work. Time spent waiting in your own outgoing queue needs a separate policy: check the absolute application deadline before transmission and execution. Do not reissue an old command with a fresh five seconds indefinitely.
|
|
|
|
**Experiment:** preserve the simulator session for 60 seconds, disconnect it, publish one five-second QoS 1 ping, and reconnect after ten seconds. Expect no expired command delivery. Repeat with a much longer message expiry and reconnect within it: expect delivery if the other session/ACL prerequisites hold. Log IDs and timestamps, not just a message count. Allow timing margin around expiry boundaries.
|
|
|
|
## 2. Retained state is a latest value, not an event history
|
|
|
|
**❓ Question:** **A new simulator subscribes and receives a retained `dispense` command from last week. What design choice caused this replay?**
|
|
|
|
**💬 Answer:** A one-shot action was published with RETAIN set. Retained publications are stored by topic for future subscribers, so use retained messages for replaceable state such as desired configuration and nonretained messages for one-time actions.
|
|
|
|
A retained publication replaces the stored value for one exact topic. It can initialize a newly created subscription even when that client has never connected before. Offline session queues instead belong to an existing session and its subscriptions. Unsubscribing or deleting a session does not delete a retained value.
|
|
|
|
Use a desired-configuration topic for state such as revision 7 and a temperature setting. Use nonretained command topics for one-shot actions. A retained "dispense" would risk replay when someone subscribes later.
|
|
|
|
Extend the ACL deliberately before introducing topics: backend writes `lab/v1/devices/+/desired`, simulator reads its exact `.../device-001/desired`; simulator writes `.../device-001/reported`, backend reads `.../+/reported`. Reload/restart the lab and verify both allowed and denied directions. Extend Lesson 5's topic parser and contracts as well.
|
|
|
|
Implement desired and reported revision records. Store the highest applied revision in the simulator and explain the initial in-memory restart limit. Receiving revision 6 after 7 must not overwrite 7. A backend PUBACK for revision 7 proves neither application nor reported revision 7; wait for the device's report.
|
|
|
|
### Clear the retained telemetry from Lesson 4
|
|
|
|
Stop the simulator's periodic telemetry first so it cannot replace the value during your observation. Run this with the simulator's temporary lab password, using the same CLI credential caveat as Lesson 1:
|
|
|
|
```sh
|
|
mosquitto_pub -h 127.0.0.1 -p 18883 -V mqttv5 -u simulator -P '<simulator-lab-password>' -t 'lab/v1/devices/device-001/telemetry' -q 1 -r -n -d
|
|
```
|
|
|
|
`-r` sets RETAIN; `-n` supplies a zero-byte payload. Together they delete the retained value at that exact topic. An empty JSON object or the two characters `""` are nonempty payloads and do not delete it. Existing subscribers can see the empty publication; a fresh subscriber should receive no retained replay. Removing the stored value does not retract copies already delivered or queued elsewhere. Restore ordinary nonretained telemetry after checking.
|
|
|
|
## 3. Model presence as evidence
|
|
|
|
**❓ Question:** **The broker's retained presence says `online`, but the last heartbeat is old. What can the backend safely conclude?**
|
|
|
|
**💬 Answer:** It can conclude only that the broker currently stores an online assertion. It cannot conclude the device is healthy now. Track backend receipt time and heartbeat freshness, and compare a connection or boot identity so delayed state does not replace newer evidence.
|
|
|
|
The simulator supplies its Will during CONNECT. The broker later publishes it when the connection ends in a way that requires a Will; it is not code that the dead simulator executes. First add simulator write/backend read ACLs for its `.../presence` topic and teach the topic parser about it.
|
|
|
|
Add these fields to the simulator's AutoPaho configuration, alongside its existing settings:
|
|
|
|
```go
|
|
WillMessage: &paho.WillMessage{
|
|
Topic: "lab/v1/devices/device-001/presence",
|
|
QoS: 1,
|
|
Retain: true,
|
|
Payload: []byte(`{"status":"offline"}`),
|
|
},
|
|
WillProperties: &paho.WillProperties{
|
|
WillDelayInterval: paho.Uint32(5),
|
|
ContentType: "application/json",
|
|
},
|
|
```
|
|
|
|
`paho.Uint32(5)` supplies the optional MQTT property as a pointer; the unit is seconds. This is a minimal presence payload for the experiment. Extend it with your schema and a connection/boot identity for the real lesson implementation. A Will payload is prepared at connection time: a timestamp embedded there is not the later time of disconnection. Record backend receipt time separately.
|
|
|
|
Publish retained online presence after successful simulator startup. Publish an explicit retained offline value before a planned disconnect if observers need that signal. Normal DISCONNECT suppresses the Will, so otherwise a retained online value can remain after a clean shutdown. A special MQTT 5 disconnect reason can request the Will, but the early helper uses the normal path.
|
|
|
|
Will publication occurs when its delay elapses or the session ends, whichever happens first. With Session Expiry 0, ending the session can remove the intended five-second grace period. With a preserved session, resuming that session before the delay ends can suppress the pending Will. Test with stable identity, Clean Start false, and a session expiry longer than the delay.
|
|
|
|
### Separate experiments that look similar
|
|
|
|
| Action | What it tests | Prediction |
|
|
| --- | --- | --- |
|
|
| Graceful simulator shutdown | DISCONNECT and explicit offline policy | Normal disconnect does not itself publish the Will |
|
|
| Abruptly terminate the exact simulator binary | Lost client without DISCONNECT | Broker may publish Will after detection and applicable delay |
|
|
| Reconnect into preserved session before Will Delay | Transient outage | Pending Will can be suppressed |
|
|
| Stop broker | Broker availability | A stopped broker cannot publish Wills while it is down |
|
|
| Keep MQTT traffic alive but stall heartbeat production | Application health | Keep Alive can succeed while freshness fails |
|
|
|
|
For an abrupt test, build a dedicated lab binary with `go build -o /tmp/mqtt5-lab-simulator ./cmd/simulator`, run it with your usual simulator flags, and record its PID. Send `kill -KILL <recorded-lab-PID>` from another terminal only after verifying that PID. KILL prevents Go cleanup and DISCONNECT; it is different from Ctrl+C. Do not target all Go processes or assume a `go run` wrapper is the actual client. Restart the binary afterward and inspect both retained presence and fresh heartbeats.
|
|
|
|
A killed process usually closes its socket promptly. It does not reproduce a silent network black hole that must be detected through Keep Alive. Record the actual fault injected and avoid claiming the measured delay is a universal offline-detection time.
|
|
|
|
## 4. Explain the result before moving on
|
|
|
|
<details>
|
|
<summary>A fresh subscriber receives retained online, but no heartbeat follows. Is the machine healthy?</summary>
|
|
|
|
You only know the broker stored an online assertion. The producer may have exited cleanly without replacing it, or the retained value may be old. Combine presence with receipt time and heartbeat freshness; use connection/revision identity to reject stale evidence.
|
|
|
|
</details>
|
|
|
|
**Done when:** demonstrate an expired offline command, retained replacement and deletion, graceful versus abrupt presence behavior, and rejection of an older desired revision. Record predictions and observations in `journal/10-expiry-presence.md`.
|
|
|
|
Sources: [MQTT 5 packet and session rules](https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html), [Mosquitto publication flags](https://mosquitto.org/man/mosquitto_pub-1.html), and local `references/paho.golang-v0.23.0/paho/cp_publish.go`, `paho/cp_connect.go`, and `autopaho/auto.go`.
|
|
|
|
Previous: [Lesson 9](09-sessions-recovery.md). Next: [Lesson 11](11-security-boundaries.md).
|