mqtt-5-learning/teaching/course-source/16-mqtt5-controls.md

Section 16 — MQTT 5 controls you can explain and measure

Study this after Sections 12–15 in COURSE.md, or read it earlier as a conceptual preview. Continue using Paho v0.23.0 and the isolated broker. Each experiment changes one behavior; record the broker/client versions and observed result instead of assuming a field's existence proves complete support.

1. Subscription options change what arrives

❓ Question: A device publishes a retained status update and then subscribes to that same topic. Which option controls retained replay, and which option controls the RETAIN bit on forwarded live traffic?

💬 Answer: Retain Handling controls retained replay at subscription time. Retain As Published controls whether the broker preserves the incoming RETAIN bit when forwarding a live publication. Neither option changes the broker's ACL decision or filters payload identity.

These fields belong to each paho.SubscribeOptions entry, not to the PUBLISH payload:

Field	Meaning	Experiment
NoLocal	Suppress publications originating on the same client connection identity	With a lab identity authorized to publish and subscribe to one topic, compare false/true
RetainAsPublished	Preserve the incoming publication's RETAIN bit when forwarding live messages	Compare a live retained publication with replay after resubscription
RetainHandling: 0	Send matching retained state when subscribing	Resubscribe and observe replay
RetainHandling: 1	Send retained state only when this subscription is new	Repeat the same subscription in the existing session
RetainHandling: 2	Do not send retained state at subscription time	Subscribe after storing a retained value; then publish a live update

Retain Handling 2 does not prevent future live retained publications. Retain As Published does not decide whether the broker stores retained state. No Local is not an ACL or a device-ID filter, and must not be set for shared subscriptions. Lesson 4 deliberately enables Retain As Published so the simulator's retained-command guard also works for live forwarding.

An UNSUBSCRIBE removes a broker subscription after its UNSUBACK exchange; it does not unregister your local Go callback or erase retained values. Use manager.Unsubscribe(ctx, &paho.Unsubscribe{Topics: []string{filter}}) with an operation deadline and inspect the returned reasons. filter is the exact previously subscribed filter. Already delivered work may still be in the local queue.

2. Shared subscriptions distribute work

❓ Question: Two ingestion processes share a subscription group, and a response reaches the process that did not create the pending waiter. What fails?

💬 Answer: The receiving process may have no matching in-memory command record. A shared subscription distributes each matching delivery to a group member; it does not share process memory or guarantee device affinity. Keep response correlation state shared or route responses to a common durable store.

Ordinary subscriptions on two clients each receive a matching publication. Build the base filter with contracts.DeviceFilter(contracts.KindTelemetry), then wrap its string form as $share/ingest/ plus that filter to create the shared subscription. Both clients join the ingest group, and the broker selects a group member for each matching delivery. Convert the final shared filter to contracts.TopicFilter when passing it through the lab transport helper, or to string at the Paho packet boundary. Give each process a unique Client ID; sharing a group is different from sharing a Client ID.

Create two authorized lab consumers, check their SUBACK results, then publish numbered synthetic telemetry. Record which instance handles each ID. Stop one consumer and repeat. Use a fresh lab group/session for the experiment so old offline state does not obscure the result. Verify the broker's shared-subscription support and effective ACL behavior before drawing conclusions.

Distribution is not guaranteed round-robin, per-device affinity, or business exactly-once execution. A restarted consumer can repeat work, and parallel workers can finish out of order. Shared consumers need shared durable deduplication for important reports. Keep command-response routing deliberate: a response delivered to another instance will not find a waiter stored only in your instance's RAM.

3. Distinguish wire limits from application limits

❓ Question: The backend sets Receive Maximum to four. Does that cap the number of telemetry jobs waiting in its Go worker queue at four?

💬 Answer: No. This CONNECT limit constrains the broker's concurrent QoS 1/2 publications awaiting protocol acknowledgement on the broker-to-client direction. If callbacks acknowledge quickly after queueing, the application queue can continue to grow until its own bound is reached.

The CONNECT properties describe what this client can receive. The CONNACK properties describe what the server can receive or supports. Direction matters: changing the client's Receive Maximum constrains broker-to-client QoS 1/2 inflight publications, not the backend's outgoing command rate.

Add this field to your existing autopaho.ClientConfig; it is a fragment, with net/url and Paho imports supplied by that file:

ConnectPacketBuilder: func(cp *paho.Connect, _ *url.URL) (*paho.Connect, error) {
    if cp.Properties == nil {
        cp.Properties = &paho.ConnectProperties{}
    }
    receiveMaximum := uint16(4)
    maximumPacketSize := uint32(32768)
    cp.Properties.ReceiveMaximum = &receiveMaximum
    cp.Properties.MaximumPacketSize = &maximumPacketSize
    return cp, nil
},

The nil check preserves AutoPaho's existing properties, including configured session expiry. Replacing the whole property struct would silently discard them. Receive Maximum must be nonzero. Maximum Packet Size counts the complete MQTT packet, including topic and properties, not only JSON payload. Paho maps these fields to CONNECT on each connection.

In OnConnectionUp, inspect ack.Properties safely and record server limits/capabilities such as Receive Maximum, Maximum Packet Size, Maximum QoS, and retained/shared-subscription availability. A nil optional pointer is absence with a protocol-defined default, not automatically zero. Do not print the complete authentication configuration as a shortcut.

Experiment: compare QoS 1 inflight traffic with automatic acknowledgements and with Lesson 6's deliberately delayed manual acknowledgements. A queue of 16 can still fill even with Receive Maximum 4 if automatic ACK frees inflight slots as soon as callbacks return. Repeat at QoS 0: Receive Maximum is not its backpressure mechanism. Bound application queues and broker resources separately.

For packet-size testing, use harmless payloads below and above a deliberately small configured limit. Record whether the sender library rejects locally, the peer rejects/disconnects, or delivery is absent; the precise path depends on direction and support. No callback arrival alone does not identify the failure layer.

4. Properties carry context, not trust

❓ Question: A simulator sets User Property producer=device-001. Has the broker authenticated that device identity?

💬 Answer: No. User Properties are forwarded metadata. The broker authenticates the MQTT client and applies its topic ACL; the application validates the topic, payload, and any metadata it chooses to use. Never treat a property or payload ID as proof of the publisher.

User Properties are UTF-8 key/value pairs and may repeat a key. Use them for synthetic trace context or schema hints, with size/count limits. They are not authenticated claims by themselves and are not HTTP headers that the broker interprets for your application.

properties := &paho.PublishProperties{
    User: paho.UserProperties{
        {Key: "trace-id", Value: "lab-trace-001"},
        {Key: "producer", Value: "simulator"},
    },
}

Attach this fragment to a publication and copy the metadata you need at your receive boundary. Change producer to backend while publishing as the simulator: the string changes, but the authenticated identity does not. Avoid putting secrets in properties because observers may log them.

Subscription Identifiers let a subscriber label a subscription and identify which subscription(s) matched a received message. They do not identify the publisher or deduplicate business events. The pinned public PublishProperties.SubscriptionIdentifier is a single *int; overlapping-subscription cases require checking the library's representation/support rather than assuming it exposes every possible identifier as a list.

5. Topic aliases optimize one connection

❓ Question: The broker restarted and the MQTT session resumed. Can the client safely publish using an alias-only packet from before the restart?

💬 Answer: No. Topic Alias mappings belong to one network connection and direction. Reestablish the mapping after reconnect; session resumption does not restore it.

An alias associates a small integer with a topic name for one direction of one connection. The sender first establishes the mapping with the topic and alias; subsequent publications may use the alias to avoid resending the long topic string. This reduces bytes, not permission checks or payload size.

The receiver advertises its Topic Alias Maximum. Zero or absent support must not be treated as permission to send arbitrary aliases. A reconnect loses the mapping even when an MQTT session resumes. Never send an alias-only first publication after reconnect.

Read the pinned paho/extensions/topicaliases/ implementation and its tests, plus the receiving path in paho/client.go, before enabling aliases. The extension uses a publish hook to manage outbound mappings. Integrate reset/recreation into the connection lifecycle and verify both initial and reconnected traffic; the presence of a TopicAlias field alone is insufficient evidence. Keep the lab's default full-topic publications until that experiment passes.

6. Read failures as protocol evidence

❓ Question: A QoS 1 PUBACK has reason code 0x10 (No matching subscribers). Did the device accept the command?

💬 Answer: No. The reason describes the broker's handling of that publication hop. It is not an application response from the device. The command service must wait for and validate a correlated device response, and must preserve an unknown outcome if none arrives.

Reason codes belong to a packet context. A SUBACK contains a result for each filter. A QoS 1 PUBACK describes the immediate publication hop. A DISCONNECT may describe why the broker ended a connection. None is a device business status.

Observation	Interpretation	Next action
Publish response 0x10	No matching subscribers; a non-error MQTT result	Check peer session/subscription and await application evidence
Reason code at least 0x80 in these ACK results	Protocol-level failure	Classify the packet/reason before deciding to retry
Connection error without MQTT reason	May have failed before a packet response arrived	Inspect network/TLS and broker evidence
Context deadline exceeded	Local wait ended	Preserve outcome uncertainty if transmission may have happened

Use bounded reconnect backoff with jitter for transient failures so many devices do not reconnect in lockstep after an outage. The pinned ReconnectBackoff field accepts a function of attempt number; inspect the tagged backoff example before selecting parameters. Authentication/configuration errors need repair or escalation, not a faster retry loop. Reconnect retries a connection; it does not authorize repeating a business action under a new ID.

Why can Receive Maximum 4 coexist with 100 queued application items?

The inflight quota counts unacknowledged protocol deliveries, not all work the application has admitted. If the callback queues and acknowledges quickly, each ACK permits another delivery while workers remain behind. Separate durable admission, queue capacity, and protocol flow control.

Done when: for each feature record purpose, Go API field, packet/direction, prerequisite/ACL, prediction, actual observation, and whether you would use it. A reading-only feature stays marked “not experimentally verified.”

Sources: MQTT 5 specification and pinned local paho/cp_subscribe.go, paho/cp_unsubscribe.go, paho/cp_publish.go, paho/cp_connect.go, paho/cp_connack.go, and autopaho/backoff.go under references/paho.golang-v0.23.0/.

Next: Section 17 in COURSE.md.