# The enrollment demo: what it is, what it does, what it checks

A throwaway prototype that proves the board enrollment design works across three languages and a real broker. It covers **Stage 0 and Stage 1 only** (a raw board becomes an FN board and uses its FN topics). Activation (FN to SN) is described in `docs/investigations/mqtt-board-identity/sn-as-username-process.md` in the backend repo but is **not built here**.

Everything is in `crypto-interop-spike/`.

## 1. The idea in one paragraph

A board with no credentials connects anonymously and sends a **public key**. The backend answers with its own public key and a **signature**. Each side combines the two public keys into the **same secret** and turns it into the broker password, so the password is never sent. The backend creates the broker account with that password, the board logs in with it, and from then on the board can only use its own FN topics.

## 2. What runs

```mermaid
flowchart LR
    subgraph Docker network
      V[viewer<br/>web page :8099]
      M[broker<br/>Mosquitto 2.1 + Dynamic Security]
      G[backend<br/>Go]
      C[board-c<br/>C + libsodium]
      P[board-cpp<br/>C++ + OpenSSL]
    end
    C -- MQTT --> M
    P -- MQTT --> M
    G -- MQTT --> M
    G -. log file .-> V
    C -. log file .-> V
    P -. log file .-> V
```

| Service | Language and libraries | What it does |
|---|---|---|
| `broker` | `eclipse-mosquitto:2.1-alpine` | MQTT broker with the Dynamic Security plugin; starts with only an admin client. |
| `backend` | Go (standard library crypto, paho MQTT) | Configures the broker, answers enrollment requests, creates FN accounts, acknowledges `hello`. |
| `board-c` | C, libsodium 1.0.20 or newer, libmosquitto | Simulated board. |
| `board-cpp` | C++17, OpenSSL 3, libmosquitto | A second simulated board using different libraries, to prove they interoperate. |
| `viewer` | Go standard library | Web page with a live view of all log lines. |

The two boards are intentionally written with different crypto libraries: if the backend (Go) and both boards derive the same password, the algorithms and byte formats really are compatible.

## 3. How to run it

```bash
cd /home/ryan/Media/Code/Code/robot-market/crypto-interop-spike/docker
./run-docker.sh        # builds, starts, streams logs, prints the result
```

- Open **http://localhost:8099** for the live view.
- `docker compose logs -f` shows the same lines in a terminal.
- Stop everything with `docker compose down` (run it in the `docker` directory, or pass `-f .../docker/compose.yaml`).
- Exit code 0 from `board-c` and `board-cpp` means all their checks passed.

**Timing** (set in `compose.yaml`): the backend waits 15 s, `board-c` 20 s, `board-cpp` 35 s before starting, and every step pauses 2 s (`STEP_DELAY`). These delays exist only so a person can follow along.

## 4. What happens, step by step

**Backend (once at start):**

1. Connects to the broker as the Dynamic Security admin (`backend`).
2. Provisions the broker: role `backend-service` with its own client `backend-svc`; role `enroll` for anonymous clients; role `board-bootstrap` for FN boards; default ACL access set to deny for everything else.
3. Connects the service client and subscribes to `enroll/request/+` and `board/+/hello`.

**Each board (ten steps):**

| Step | Board action |
|---|---|
| 1 | Generate a random Client ID, an X25519 key pair and a nonce from the CSPRNG. |
| 2 | Connect anonymously using that Client ID. |
| 3 | Subscribe to its own `enroll/response/<clientid>`; also try another board's topic and a wildcard. |
| 4 | Publish the request on `enroll/request/<clientid>` with its public key, nonce and MAC label. |
| 5 | Wait for the answer (re-send the same request if it is lost). |
| 6 | Verify the backend's Ed25519 signature with the pinned public key, plus tamper tests. |
| 7 | Derive the shared secret (X25519) and the broker password (HKDF-SHA256). |
| 8 | Log in as the FN with the derived password; also try wrong credentials. |
| 9 | Use the FN topics: subscribe, publish `hello`, wait for `ack-hello`; also try forbidden topics. |
| 10 | Print the result and exit with code 0 (all passed) or 1. |

**Backend per request** (when it sees a board's request): reserve the Client ID, assign `FN-00000N`, create its own temporary key, derive the password, log only its fingerprint, create the broker client (username = FN, Client ID = FN, role `board-bootstrap`), sign the answer, publish it.

## 5. Message formats and cryptography

- **Request** (`enroll/request/<clientid>`): lines `B=<64 hex>`, `n1=<32 hex>`, `mac=<text>`.
- **Response** (`enroll/response/<clientid>`): lines `fn=<FN>`, `S=<64 hex>`, `sig=<128 hex>`. No password, no secret.
- **Shared secret:** X25519 between the board's private key and the backend's temporary public key `S` (and the reverse on the backend).
- **Password:** hex of `HKDF-SHA256(shared, salt = n1, info = "robot-market/mqtt-pw/v1|" + FN, 32 bytes)`.
- **Signature:** Ed25519 over `S || B || n1 || clientid || FN` using the backend's key (seed in `compose.yaml`); the board holds the public half (`BACKEND_PUBKEY_HEX`), like firmware would.
- **Logs:** passwords are never logged; a short SHA-256 fingerprint (16 hex characters) is logged by both sides so they can be compared.

## 6. The 19 checks each board performs

| # | Check | What it proves |
|---|---|---|
| 1 | Anonymous CONNECT accepted | The enrollment listener allows credential-less boards. |
| 2 | Subscribe to own `enroll/response/<clientid>` granted | The `%c` ACL gives each board its own mailbox. |
| 3 | Subscribe to another board's response topic denied (SUBACK 128) | Boards cannot read each other's replies. |
| 4 | Wildcard `enroll/response/#` denied | A board cannot read every reply at once. |
| 5 | Response received | The request/response round trip works through the broker. |
| 6 | Response contains `fn`, `S`, `sig` | The message format parses. |
| 7 | S and signature decode to 32 and 64 bytes | Field sizes are correct. |
| 8 | Signature over `S || B || n1 || clientid || FN` is valid | The backend (Go, Ed25519) and the board's library agree on the signed bytes. |
| 9 | Tampered signature (1 bit) rejected | Forged signatures are caught. |
| 10 | Reply bound to a different nonce rejected | Replayed or mismatched replies are caught. |
| 11 | Reply with a swapped FN rejected | The FN is part of what is signed. |
| 12 | X25519(b, S) succeeds (non all-zero) | Key agreement works and degenerate results are rejected. |
| 13 | Login with a wrong password refused | The broker really checks the derived password. |
| 14 | Right password but a different Client ID refused | Client ID binding is enforced. |
| 15 | Login as FN with derived password and bound Client ID accepted | The board derived **exactly** the password the backend set. |
| 16 | Subscribe to own `board/<FN>/cmd` granted | `%u` substitution works for the FN role. |
| 17 | Subscribe to another FN's `cmd` denied | An FN board is limited to its own topics. |
| 18 | `enroll/response/#` denied for the FN board | The FN board has left the enrollment permissions behind. |
| 19 | Backend's `ack-hello` received | Publishing `hello` was permitted and the backend's reply reached only this board. |

Visible on the web page in addition: the two **fingerprints** (backend and board) must match, and the backend logs one FN per board with no duplicates.

## 7. Reading the web page

- **Three lanes**, one per program (board C, backend, board C++), in time order.
- **Cards** at the top: progress bar, checks passed and failed, final result, and a green chip when the board's password fingerprint equals the backend's.
- **"Now" box:** a plain-language explanation of the latest step.
- **Options:** follow live, only checks and steps, hide countdown lines. Click a long hex value to expand it.
- A refresh replays the whole run, because the server keeps every line.
- **Run again** button (shown only when the stack is started with `docker/compose.restart.yaml`, as `run-docker.sh` does; the server deployment does not use it): restarts the backend, both boards and the broker (clean Dynamic Security state), clears the logs and replays the whole run. It needs the Docker socket mounted into the `viewer` container (`/var/run/docker.sock`), which gives that container control of Docker on the host, so the page is bound to `127.0.0.1` and the restart endpoint requires a custom header that other web pages cannot send. Demo use only.

## 8. File map

| Path | Contents |
|---|---|
| `docker/compose.yaml` | The stack, timings, secrets (demo values), shared log volume. |
| `docker/broker/` | `mosquitto.conf`, `entrypoint.sh` (creates the admin client on every start). |
| `docker/backend/` | Go backend (`main.go`), vendored dependencies, Dockerfile. |
| `docker/board-c/` | C board (`main.c`), Dockerfile. |
| `docker/board-cpp/` | C++ board (`main.cpp`), Dockerfile. |
| `docker/viewer/` | Log viewer and restart button (`main.go`, `index.html`), Dockerfile. |
| `docker/run-docker.sh` | Build, run, stream logs, show results. |
| `go/`, `c/`, `c-libsodium/`, `cpp/`, `run.sh`, `run-libsodium.sh` | The earlier fixed-key programs that check the crypto libraries against the RFC test vectors and against each other (no networking). |

## 9. What it proved, and what it did not

**Proved**

- Go (standard library), libsodium and OpenSSL produce identical X25519, HKDF and Ed25519 results, including RFC 7748, RFC 5869 and RFC 8032 test vectors.
- The whole enrollment works over a real broker with Dynamic Security, with the password never transmitted.
- The ACLs behave as designed: own mailbox only, no wildcards, FN topics only, Client ID binding.
- Retries are safe: a repeated request does not create a second FN (the first version of the backend did; fixed with an in-flight reservation).

**Findings from building it**

- `%c`/`%u` substitution in dynsec ACLs does not work in Mosquitto 2.0.22; it works in 2.1.
- Changing a connected client's roles disconnects it, so the backend uses a separate admin client and service client.
- `mosquitto_ctrl addRoleACL` needs an explicit `allow` or `deny`.
- Go dependencies are vendored because the Docker build could not reach the Go module proxy in this environment.

**Not covered**

- Activation (FN to SN), operator approval, SAS, replacement and retirement.
- TLS, real secret storage (demo secrets are in `compose.yaml`), rate limiting and expiry of unclaimed FNs.
- Real hardware: nothing ran on an ESP32 or Orange Pi; the libsodium board was built against Alpine's libsodium, not the ESP-IDF component.
- Domain-separated signatures and zeroizing of secrets (recommended in the design doc, not in the prototype).
- Load, failure injection (broker restart, lost responses beyond the retry), and concurrency beyond two boards.