mqtt-security-test/docs/sn-as-username-process.md

# Board identity over MQTT: the full process with SN as the broker username

**Status:** Design proposal. The enrollment part (raw board to FN) has been prototyped and verified in a Docker spike; the activation part (FN to SN) is **designed but not built or tested**. Nothing here is implemented in the backend yet.
**Builds on:** [`mqtt-dynamic-security-rnd.md`](./mqtt-dynamic-security-rnd.md) and [`enrollment-and-activation.md`](./enrollment-and-activation.md) (which compares the FN-username and SN-username models). This document is the complete description of the chosen direction: **SN as the username**, including every caution and security risk found so far.

## 1. Goal and scope

Turn a raw board into a board that holds only the MQTT access of the vending machine it is installed in, safely and durably:

```
raw board  ->  FN board (low access)  ->  SN board (machine access via roles)
```

In scope: enrollment, activation, normal operation, board replacement and retirement, compromise handling, broker configuration, backend data model, threat model.
Out of scope: the MQTT application topics themselves (see the protocol contract), firmware update delivery, TLS certificate issuance.

## 2. Terms

| Term | Meaning |
|---|---|
| **Board** | The physical controller. Identified by its **FN**. |
| **FN** | Backend-assigned identity of one physical board. Never reused, never reassigned. Lives in its own board table. |
| **SN / plate** | Identity of one vending machine (the plate is the SN). Pre-registered by an admin. Printed on the machine, so **not a secret**. |
| **Enrollment** | A raw board obtains an FN and a broker account. Over MQTT, with the Go backend, no other channel. |
| **Activation** | An operator links an FN to an SN; the board receives the machine-level broker account. |
| **Nonce** | Random single-use value that binds a reply to exactly one request. |
| **Random Client ID** | 128-bit CSPRNG value the board picks for enrollment only. Acts as an unguessable mailbox address. |
| **SAS** | Short authentication string: a short code the board shows and the operator compares with the screen. |
| **Dynamic Security (dynsec)** | Mosquitto plugin holding users, roles and ACLs, managed over MQTT. |

## 3. Design principles

1. **Authentication is separate from authorization.** Who may connect (a client with a secret) is not the same as what it may do (roles and ACLs).
2. **Trust is granted once, at activation, by a human.** Enrollment is deliberately open; an FN grants almost nothing.
3. **No secret ever crosses the network.** Passwords are derived independently on both sides from an X25519 key agreement.
4. **The backend proves who it is with a signature** whose public key is pinned in the board.
5. **Deny by default.** Every ACL type starts denied; only reviewed topic and action pairs are allowed.
6. **The machine owns its broker account.** A board is the current holder of that account; replacement rotates it.
7. **The database is the authority; the broker is reconciled to it.** A broker command succeeding does not prove the database committed, and the reverse.
8. **One-way doors are logged.** Every identity-changing step is audited with actor, time, FN, SN and broker result.

## 4. Components and namespaces

```mermaid
flowchart LR
    B[Board<br/>firmware with pinned backend public keys] -- MQTT over TLS --> M[Mosquitto 2.1+<br/>Dynamic Security]
    G[Go backend<br/>admin client + service client] -- MQTT --> M
    G --> D[(Database<br/>boards, devices, assignments, audit)]
    O[Operator UI<br/>RBAC] --> G
```

| Principal | Broker identity | May do |
|---|---|---|
| Raw board | anonymous (`enroll` role via the anonymous/unauthenticated group) | publish `enroll/request/%c`; subscribe and receive `enroll/response/%c` |
| FN board | username = FN, Client ID = FN, role `board-bootstrap` | publish `board/%u/hello` and `board/%u/activation/request`; subscribe and receive `board/%u/cmd` and `board/%u/activation/response` |
| SN board | username = SN, Client ID = current FN, role `vending-device` (+ capability roles) | topics under `device/%u/...` only, as listed in the protocol contract |
| Backend service | `backend-svc`, role `backend-service` | read enrollment requests, answer them, talk to FN and SN topics it must serve |
| Backend admin | `backend` (dynsec admin) | `$CONTROL/dynamic-security/#` only |

**Two backend identities on purpose:** the admin client manages the broker; the service client carries application traffic. A client whose roles are changed while connected is disconnected by the broker, so an identity must never modify itself, and a compromise of the service client must not give broker administration.

## 5. Lifecycle

```mermaid
stateDiagram-v2
    [*] --> RAW
    RAW --> ENROLLED: enrollment (FN created)
    ENROLLED --> EXPIRED: never logged in within N hours
    ENROLLED --> ACTIVATION_PENDING: board asks to activate
    ACTIVATION_PENDING --> ACTIVATING: operator approves (SAS confirmed)
    ACTIVATING --> ACTIVE: board logged in as SN (first SN login = ack)
    ACTIVATING --> ENROLLED: timeout, rolled back
    ACTIVE --> SUSPENDED: operator action
    SUSPENDED --> ACTIVE: resumed (credentials rotated)
    ACTIVE --> REPLACED: replacement approved
    ACTIVE --> RETIRED: end of life
    ENROLLED --> REVOKED: operator or security action
    ACTIVE --> REVOKED: operator or security action
```

The SN has its own state: `UNASSIGNED -> ASSIGNED(FN, generation) -> UNASSIGNED` on replacement or retirement. Invariant, enforced by a database unique constraint: **one SN has at most one active FN, and one FN is active on at most one SN.**

## 6. What the firmware contains

Identical on every board, nothing secret:

- Backend **primary and backup public signing keys** (Ed25519).
- Broker host and the CA certificate used to verify the broker's TLS certificate.
- Topic templates for enrollment, FN and SN topics.
- Crypto: X25519, HKDF-SHA256, Ed25519 verification, a hardware random number generator.
- Flash space for the stored identity (FN, SN, passwords, generation number).
- A way to show the SAS (display or service port).

Rejected: any fleet-wide secret in firmware. One flash dump would expose every board and rotation would need a fleet-wide OTA update.

## 7. Stage 0: enrollment (raw board becomes an FN board)

### 7.1 Broker side
Anonymous clients are allowed on a dedicated TLS listener and fall into the anonymous group (`setAnonymousGroup`); on Mosquitto 2.1 and later there is also a default `unauthenticated` group. Mosquitto does **not** require passwords. A shared static password would add no security because it would be public.

Role `enroll` (assigned to the anonymous group):

| ACL type | Topic |
|---|---|
| `publishClientSend` | `enroll/request/%c` |
| `subscribePattern` | `enroll/response/%c` |
| `publishClientReceive` | `enroll/response/%c` |
| `unsubscribePattern` | `enroll/response/%c` |

`%c` is the Client ID, so a board can only talk on its own mailbox. Only the backend service may read `enroll/request/+`.

### 7.2 Protocol

```mermaid
sequenceDiagram
    participant B as Board
    participant M as Mosquitto
    participant G as Go backend
    B->>B: random Client ID, X25519 pair (b, B), nonce n1
    B->>M: CONNECT anonymous, clientId=random (TLS)
    B->>M: SUBSCRIBE enroll/response/[cid]  (own topic only)
    B->>G: enroll/request/[cid] [B, n1, mac]
    G->>G: reserve [cid], assign FN, ephemeral (s, S)
    G->>G: shared = X25519(s, B), pw = HKDF(shared, salt=n1, info = label, FN)
    G->>M: dynsec createClient FN, pw, clientid=FN, role board-bootstrap
    G->>B: enroll/response/[cid] [FN, S, sig]  (not retained)
    B->>B: verify sig, shared = X25519(b, S), same pw
    B->>B: persist FN and pw to flash, then reconnect as FN
```

- `sig` = Ed25519 over `S || B || n1 || clientid || FN`. Binding the FN and the Client ID means a reply for another request, board or identity does not verify.
- The password is **hex of HKDF-SHA256(shared secret, salt = n1, info = "robot-market/mqtt-pw/v1|" + FN, 32 bytes)**. The label separates this use of the secret from any other.
- The first successful FN login is the enrollment acknowledgement.
- Dynamic Security stores only a salted hash of the password. The backend never stores the plaintext, so a re-issue means a new password, not a lookup.

### 7.3 Idempotency and retries
- The board persists FN and password to flash **before** reconnecting; a power loss in between must not lose the only copy.
- A lost response is handled by the board re-sending the **same** request. The backend keeps a per-Client-ID record: `processing` (duplicates are ignored) or `answered` (the cached public reply is re-sent; no new FN, no new password).
- A failed enrollment releases its reservation so the board can retry. (The prototype originally created an orphan FN when a retry arrived while the first request was still running; the in-flight reservation fixes that.)
- Unclaimed FNs (never logged in within N hours) are disabled and then deleted.

### 7.4 What an FN can do
Only `hello` and `activation/request` on its own FN topics. It cannot use enrollment topics, other FNs' topics, or any `device/...` topic. Because of this, a fake FN is harmless in itself, which is why enrollment can stay open.

## 8. Stage 1: FN board waiting

The board logs in as `FN`, Client ID `FN`, with the derived password, sends `board/<FN>/hello`, and requests activation when a machine is ready. It receives only `board/<FN>/cmd` and `board/<FN>/activation/response`.

## 9. Stage 2: activation (FN to SN) -- designed, not yet built

### 9.1 Preconditions
- The SN exists in the database as `UNASSIGNED`, created by an admin.
- The board is `ENROLLED` and online as its FN.
- An operator with the right RBAC role starts the activation in the application.

### 9.2 Flow

```mermaid
sequenceDiagram
    participant O as Operator (UI)
    participant G as Go backend
    participant M as Mosquitto
    participant B as Board (FN session)
    B->>B: fresh X25519 pair (b2, B2), nonce n2, SAS = digits(hash(B2, n2, FN))
    B->>G: board/[FN]/activation/request [B2, n2]
    G->>O: pending request FN, MAC label, SAS, time
    O->>G: choose SN, enter activation code, confirm SAS matches the board
    G->>G: DB: ACTIVATING (FN, SN, next generation), unique constraint
    G->>G: ephemeral (s2, S2), shared2, pw_sn = HKDF(shared2, salt=n2, info = label, SN, generation)
    G->>M: dynsec create or update client SN: pw_sn, role vending-device, clientid = FN
    G->>B: board/[FN]/activation/response [SN, next generation, S2, sig]
    B->>B: verify sig, derive pw_sn, persist SN, pw_sn and generation, reconnect
    B->>M: CONNECT username=SN, clientId=FN, password=pw_sn
    M-->>G: first message on device/[SN]/... (or presence)
    G->>G: DB: ACTIVE, disable the FN client, audit
```

### 9.3 Rules that make it safe
1. **SAS bound to the activation key.** The board shows a code derived from `B2` (and `n2`, FN). The operator compares it with what the backend received. An attacker who stole an FN password and sends a request with their own key produces a code that does not match what the real board displays. **This check is mandatory**, not optional; without it the operator could attach a stranger's FN to a real SN.
2. **Activation code** (from the existing activation flow): single use, expiring, bound to one SN, attempt-limited. It proves the operator is physically at the machine.
3. **SN is never trusted as proof.** It is printed on the machine; the operator chooses it, the board does not claim it.
4. **Fresh key pair per activation.** Reusing the enrollment key would link the two exchanges and weaken forward secrecy.
5. **Signed reply includes domain separation.** Sign `"robot-market/activate/v1" || S2 || B2 || n2 || FN || SN || generation`. (The enrollment prototype signs without a domain-separation prefix; add `"robot-market/enroll/v1"` there too so a signature from one flow can never be replayed in the other.)
6. **Generation counter.** Each activation of an SN increments a stored generation, mixed into the password derivation and delivered signed. It makes every password unique, gives auditors a monotonic value, and lets a board detect a stale reply.
7. **Client ID binding.** The SN client's Client ID is set to the current board's FN. A copied SN password does not work from a client with another Client ID. (A *clone* of the same flash image still has the right Client ID; see section 13.)
8. **Cutover is acknowledged, not assumed.** The FN client stays enabled until the first successful SN login. Disabling it earlier would strand a board that lost power before saving the new credentials. After the acknowledgement, the FN client is disabled (or deleted, per policy) and its live session dropped.
9. **Order of operations.** Database state `ACTIVATING` is committed first, then the broker command, then the reply, then `ACTIVE` after the acknowledgement. A reconciler repairs any crash in between (section 11).
10. **Activation requests expire** and are bounded in number per FN.

### 9.4 Failure cases

| Failure | Outcome |
|---|---|
| Response lost | Board times out and re-sends the request; backend resends the cached public reply (the password is the same). |
| Board loses power after the reply but before saving | FN still enabled, so it can request again; the backend re-derives with the same inputs or restarts the activation. |
| Broker command fails | State stays `ACTIVATING`; the reconciler retries idempotently or rolls back to `ENROLLED`. |
| Operator confirms the wrong SN | Detected before approval by the SAS and by the activation code binding; after approval, replacement flow reverts it (audited). |
| No SN login within the timeout | Roll back: remove or disable the SN credential, SN returns to `UNASSIGNED`, FN remains usable. |
| Two operators approve the same SN | The unique constraint rejects the second. |

## 10. Stage 3: normal operation

- The board connects as `SN` with Client ID `FN` and uses `device/<SN>/...` topics. One shared role `vending-device` uses `device/%u/...`, so the broker enforces that a board can only touch its own machine's topics.
- **Capability roles** (`news`, `updates`, ...) are additional shared roles added per machine type or entitlement. Adding a role to a connected client **disconnects it**, so plan for a reconnect.
- The backend subscribes with wildcards (`device/+/...`) and takes the SN from the topic. MQTT does not tell a subscriber who published, so the backend must not blindly trust the topic: the broker's ACL and Client ID binding guarantee the topic matches the username, and critical messages should also carry the SN and be cross-checked against the database.
- Messages that carry secrets or credentials are never published retained. Device clients use clean sessions so queued messages do not outlive an identity.

## 11. Backend data and reconciliation

Suggested tables (names illustrative):

| Table | Key fields |
|---|---|
| `boards` | FN (unique, never reused), enrolled_at, last_login_at, state, hardware_label (claim only), pubkey fingerprint |
| `enrollments` | client id, nonce, board FN, state (`processing`, `answered`, `confirmed`, `expired`), cached public reply |
| `devices` (existing) | UUID, SN (plate), state |
| `board_assignments` | SN, FN, generation, state, created/activated/ended timestamps, actor |
| `broker_outbox` | desired broker operation, idempotency key, attempts, result |
| `audit_log` | actor, action, FN, SN, generation, broker result, timestamp |

Reconciler rules:
- Every broker mutation is a recorded desired state, applied idempotently: create client is "create or update", disable is safe to repeat.
- A periodic job lists clients, roles and the anonymous group from the broker and compares them with the database; drift raises an alert.
- Passwords cannot be read back from the broker, so drift in a password is detected by the generation counter and by login failures, and corrected by rotating.

## 12. Replacement, transfer, retirement

**Replacing the board in machine SN (same machine, new hardware):**
1. Authorize the replacement; mark the old assignment `REPLACED`.
2. **Revoke first:** rotate or disable the SN credential and drop the old board's session. The broker must refuse the old board before any new board gets access.
3. The new board enrolls as a **new FN** (a board never inherits an FN).
4. Normal activation runs for the same SN: new key exchange, **new password**, Client ID rebound to the new FN, generation incremented.
5. Audit: old FN, new FN, SN, generation, actor, time, broker results.

A single "set a new password" action both revokes the old board and grants the new one, so the system cannot end up with two valid boards for one machine. Do not rely on a password change alone to drop an existing live session; verify the broker's behavior and explicitly disconnect or rebind (changing a client's roles or Client ID disconnects it, as observed).

**Moving a board to another machine:** treat the board as revoked on the old machine, then run a deliberate **re-enrollment** (new FN, new password) before activating on the new one. Never re-enable an old credential.

**Retirement / lost or stolen board:** disable the SN credential immediately (rotate), disable the FN, mark `REVOKED`, alert on any later connection attempt.

## 13. Threat model

| # | Threat | Mitigation | Residual risk |
|---|---|---|---|
| 1 | Eavesdropper on the network or broker reads the credential | The password is never transmitted; only public keys travel; TLS on top | Traffic metadata is visible. |
| 2 | Man in the middle swaps the backend's key | Ed25519 signature checked against the pinned public key | Depends on the pinned key staying secret on the server. |
| 3 | Replay of an old reply | Reply bound to the request nonce, Client ID, FN (and SN, generation); board rejects mismatches | None known if nonces come from a good RNG. |
| 4 | Another board reads my reply | `%c`-scoped ACL; random 128-bit Client ID; wildcard subscription denied | Weak RNG on the board would make Client IDs guessable. |
| 5 | Enrollment flood (anonymous) | Per-listener limits, backend rate limits and cap on pending enrollments, expiry of unclaimed FNs, payload size and schema checks | Some resource use under a determined flood; needs monitoring. |
| 6 | MAC spoofing to hijack or lock out a board | MAC is a label only; re-issue only while an FN has never logged in; later reset needs an operator | An attacker can still enroll fake FNs (harmless, section 7.4). |
| 7 | Attacker enrolls a fake FN and asks for activation | Operator must confirm the SAS derived from the activation key and enter the activation code | Relies on operators actually doing the check. |
| 8 | Stolen FN password | SAS makes activation hijack visible; FN can do nothing else | Attacker can disconnect the real board (nuisance). |
| 9 | Stolen or copied SN password | Client ID binding; rotation at replacement; revocation on report | A full flash clone has the correct Client ID too (see 11). |
| 10 | Firmware / flash dump | Flash encryption and secure boot; no fleet-wide secret; the pinned key is public | Without flash encryption, one board's credentials are readable. |
| 11 | Cloned board (same FN, SN, password) | Duplicate Client ID connections kick each other, which is detectable; alert on takeover events; rotate on suspicion | A silent clone can impersonate until detected. Secure element is the real fix. |
| 12 | Backend signing key leaked | Backup public key already in firmware; signing key in a secret store or HSM; separate key for firmware signing | Boards that have not yet seen a backup-signed message still trust the leaked key. |
| 13 | Dynamic Security admin credential stolen | Admin client separate from the service client; secret store; network restriction of the control topic; device roles never grant `$CONTROL` | Admin compromise is full broker compromise. |
| 14 | A device role accidentally allows `$CONTROL` or `#` | ACL review; automated test that connects as a device and attempts both | A mistake in a future role. |
| 15 | Broker compromise | Passwords are never on the wire (hashes only in the dynsec file); TLS terminates at the broker so traffic is visible there | All device data is exposed; credentials are not directly derivable but the file is sensitive. |
| 16 | Broker and database disagree | Outbox, idempotent commands, periodic reconciliation | Window between a crash and the next reconcile. |
| 17 | Two boards for one SN, or one board for two SNs | Database unique constraints plus broker-level credential rotation | None if constraints are enforced. |
| 18 | Race between operator approvals | Single transaction and constraint on the assignment | None. |
| 19 | Old board keeps access after replacement | Revoke first, rotate, explicit disconnect, Client ID rebinding | Depends on actually dropping the live session. |
| 20 | Low-order or all-zero X25519 results | Reject an all-zero shared secret (Go and libsodium do); verify the OpenSSL path | Verify per library. |
| 21 | Weak or unseeded RNG on the board | Use the hardware RNG; on ESP32 ensure the hardware generator has entropy (RF or bootloader entropy enabled) | Must be verified on the real chip. |
| 22 | Private keys left in memory | Zeroize secrets after use (the prototype does not yet) | Memory disclosure on the board. |
| 23 | TLS misconfiguration (no certificate check, expired certificate, no clock) | Pin the CA, verify the server name, get time before TLS (NTP or RTC) | Boards without a clock cannot validate certificates reliably. |
| 24 | Secrets in logs | Log only short fingerprints, never passwords, keys or nonces beyond what is public; redact in support tooling | Debug logging added later. |
| 25 | Retained messages or persistent sessions leaking across identities | No retained secrets; clean sessions; drop sessions on revoke | Misconfigured client. |
| 26 | Operator or insider abuse | RBAC, audit log, optional four-eyes for replacement | Authorized misuse remains possible but visible. |
| 27 | Protocol change in the field | Version field in every message; reject unknown versions | Needs a migration plan. |
| 28 | Domain confusion between enrollment and activation signatures | Distinct signed prefixes per flow | Prototype lacks the prefix; add it. |

## 14. Broker facts verified in the prototype

- **`%c` and `%u` substitution in dynsec ACLs works only in Mosquitto 2.1 or newer.** On 2.0.22, literal ACLs worked and substituted ones silently never matched. Use `eclipse-mosquitto:2.1-alpine` (2.1.2) or a pinned newer image.
- **Role changes disconnect the client**: `addClientRole` kicked the connected client. Use separate admin and service identities; expect a reconnect when roles change.
- `mosquitto_ctrl addRoleACL` requires an explicit `allow` or `deny`, otherwise it fails with "Invalid arguments".
- An ACL-denied subscription returns SUBACK 128; tests rely on that. A wildcard such as `enroll/response/#` is also denied.
- A pattern string must be the only item at its hierarchy level (`enroll/response/%c` is valid; `x/pre%c` is not).
- Client ID binding works: right password with another Client ID is refused.
- Not verified yet: whether a password change alone drops a live session; the `unauthenticated` group behavior on the deployed version; behavior of every device topic under the real protocol contract.

## 15. Operations

- Secrets: dynsec admin and service passwords and the signing keys live in a secret store, not in compose files or environment dumps (the prototype uses demo values).
- Keys: the backup signing key is stored offline; rotate the primary on a schedule and on suspicion; keep a separate key for firmware signing.
- Monitoring: alert on duplicate-Client-ID takeovers, refused logins, enrollment rate, pending-activation age, reconciler drift, and any connection by a revoked FN or SN.
- Testing: keep an automated test that connects as each role and asserts what is denied, including `$CONTROL/dynamic-security/#` and `#`.
- Retention: audit log is append-only and retained according to policy.

## 16. Open decisions

- Disable or delete the FN client after activation (recommended: disable, delete after a retention period).
- Whether activation also keeps the existing activation-code flow unchanged or merges it with the SAS step.
- Capability roles: which machine types or entitlements get `news`, `updates`, etc.
- Exact timeouts: unclaimed FN expiry, activation request expiry, SN-login acknowledgement window.
- Hardware: secure element or flash encryption availability, RNG behavior, clock source.
- Whether to add a four-eyes rule for replacement and retirement.
- Which Mosquitto 2.1.x version is deployed and pinned.

## 17. Implementation checklist

1. Pin Mosquitto 2.1 or newer; enable TLS on every listener; separate enrollment listener with limits.
2. Create the dynsec roles (`enroll`, `board-bootstrap`, `vending-device`, `backend-service`), anonymous group, default deny, and the admin and service clients.
3. Backend enrollment handler (in-flight reservation, caching, expiry), with the prototype as reference.
4. Database tables, constraints and outbox; reconciler.
5. Activation: operator UI, SAS, activation code, second exchange, SN client creation, acknowledgement, FN disable.
6. Replacement and retirement flows with revoke-first ordering.
7. Domain-separated signatures; generation counter; message versioning.
8. Board firmware: persistent identity, RNG, key zeroization, pinned primary and backup keys, TLS with CA pinning.
9. Automated role and ACL test suite; monitoring and alerts.
10. Extend the Docker prototype to cover activation and replacement before building the production version.