145 lines
10 KiB
Markdown
145 lines
10 KiB
Markdown
# 🌐 Section 8 — Expose device commands through HTTP
|
||
|
||
## 🎬 Start with the system story
|
||
|
||
> 💡 **Story question:** A user clicks “ping device” in an HTTP API. How can the API respond promptly while the MQTT command and device response happen asynchronously?
|
||
|
||
The HTTP handler validates the request and asks an application service to create a command. The service records its identity and pending status, publishes through the MQTT client, then exposes later device evidence through a status endpoint. HTTP request lifetime, MQTT delivery, and the device operation each have their own acknowledgement and failure boundary.
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant U as API caller
|
||
participant H as Go HTTP handler
|
||
participant S as Command service
|
||
participant B as MQTT broker
|
||
participant D as Device
|
||
U->>H: POST /devices/{id}/commands
|
||
H->>S: Validate and create command
|
||
S->>S: Store command ID and pending state
|
||
S->>B: Publish MQTT command
|
||
B-->>S: PUBACK for broker hop
|
||
S-->>H: Accepted with command ID
|
||
H-->>U: HTTP 202 + command ID
|
||
B->>D: Deliver command
|
||
D->>B: Publish application result
|
||
B->>S: Deliver result
|
||
U->>H: GET /commands/{id}
|
||
H->>S: Read current status
|
||
S-->>H: Accepted/completed/rejected/unknown
|
||
H-->>U: HTTP status representation
|
||
```
|
||
|
||
**Before each implementation step, ask:** Is this response evidence that the HTTP request was accepted, that the broker acknowledged a publication, or that the device completed the work? Follow [Lesson 0](00-system-lifecycle.md) for the complete MQTT lifecycle.
|
||
|
||
Prerequisite: Section 7 provides a command service and queryable in-memory states. The backend and simulator continue to read their shared disposable password from `configs/lab.toml`; select each broker username with `-username` when starting the process.
|
||
|
||
Run all commands from `/home/ryan/Media/Code/Code/robot-market/mqtt-5-learning`. Create files at the project-relative paths shown below, not under `lessons/`. Continue the same application and pinned Paho v0.23.0 dependency. These are guided implementation exercises; attempt each step before requesting a full solution.
|
||
|
||
## 🌐 Translate evidence between HTTP and MQTT
|
||
|
||
HTTP has one response to this request. MQTT has an asynchronous stream where the device may reply after the HTTP connection closes. Returning 202 after admission gives the caller a stable command ID; a later GET reports evidence accumulated by the command service.
|
||
|
||
```text
|
||
POST + retry key → admit command C → HTTP 202 + C
|
||
→ MQTT publication → device result → status C
|
||
GET C --------------------------------------------------------------→ status C
|
||
```
|
||
|
||
Do not convert PUBACK directly into HTTP "device completed". Also do not detach work using an unbounded `context.Background()` per command: accepted work needs the service lifecycle, its own deadline, and tracked shutdown. The HTTP request context controls parsing and admission, while the service controls work after acceptance.
|
||
|
||
Predict a lost HTTP response: the server admits C, but the caller never sees 202 and retries. The same retry key and payload should return C, not create D. Reusing the key with a changed payload is a conflict. Later, scope keys by authenticated caller so two users cannot accidentally share a command record.
|
||
|
||
<details>
|
||
<summary>If the backend restarts after returning 202, can this lesson recover C?</summary>
|
||
|
||
No. The lesson's in-memory command and retry-key maps disappear. HTTP 202 describes admission under the current implementation, not disk durability. Sections 13–15 add database admission/outbox and process recovery; until then, document this restart limit in the lab API.
|
||
|
||
</details>
|
||
|
||
## ⚙️ 1. Add HTTP configuration and handlers
|
||
|
||
> 💡 **Story question:** Which process accepts the user's HTTP request, and how does that process reach the command service without putting MQTT logic in the handler?
|
||
|
||
Wire the HTTP server to the application service and preserve bounded parsing, deadlines, and controlled shutdown.
|
||
|
||
Create `internal/httpapi/server.go`, `handlers.go`, and `handlers_test.go`. Extend `internal/config/config.go` and `configs/lab.toml` with a loopback listener such as `127.0.0.1:18080`, request-body limit, and server timeout settings. Preserve defaults < TOML < environment < explicit CLI flags for supported settings; keep `mqtt.password` TOML-only. Validate these settings before starting listeners.
|
||
|
||
Compose HTTP and MQTT in the existing `cmd/backend/main.go`. Keep the simulator separate. Section 15 introduces separate API and ingestion processes.
|
||
|
||
## 📋 2. Define the API contract
|
||
|
||
> 💡 **Story question:** What should the caller receive immediately when device work will finish later?
|
||
|
||
Return an accepted command ID, then provide a status resource that reports later evidence.
|
||
|
||
Implement `POST /devices/{id}/commands` for a JSON command such as `{"command":"ping"}` and `GET /commands/{id}` for status. Limit bodies, reject trailing JSON and unknown fields, validate device IDs, and return JSON errors consistently. Keep the listener local; this lesson does not implement production user authorization.
|
||
|
||
Return HTTP 202 with a command ID and a Location pointing to its status after service admission. Document malformed-input 400, oversized-body 413, unknown-command/device 404 or a consistent chosen alternative, conflict 409, and admission-unavailable 503 behavior. Transport uncertainty after acceptance belongs in command status rather than silently discarding an accepted command.
|
||
|
||
## 🌐 3. Implement HTTP retry identity
|
||
|
||
> 💡 **Story question:** If the server accepted a request but the HTTP response was lost, how can the caller retry without creating a second device action?
|
||
|
||
Use an HTTP idempotency key with stable request semantics; keep it distinct from the command ID and MQTT packet identifier.
|
||
|
||
Require an `Idempotency-Key` header for POST. Map the key and request fingerprint to the existing command ID. An identical retry returns the same ID; changed content under the same key returns 409. Serialize concurrent admission for the same key. Bound key retention and document that process restart loses this in-memory guarantee. This key, the stable device command ID, and MQTT packet IDs serve separate purposes.
|
||
|
||
Use the HTTP context for request parsing and admission. Once accepted, execute under the application's lifecycle and command deadline, so client disconnect does not cancel the command. Inject the command service through a small interface and keep MQTT calls out of handlers.
|
||
|
||
## 🧪 4. Run and verify
|
||
|
||
> 💡 **Story question:** What evidence demonstrates each boundary in the round trip—from HTTP acceptance to broker delivery to device result?
|
||
|
||
Observe the returned command ID, MQTT acknowledgement, simulator response, and status endpoint separately.
|
||
|
||
Start the simulator and backend in separate terminals from the project root. The test password is already loaded from `configs/lab.toml`; `-username` selects the account for each process:
|
||
|
||
```sh
|
||
go run ./cmd/simulator -username simulator -client-id lab-simulator-device-001
|
||
```
|
||
|
||
```sh
|
||
go run ./cmd/backend -username backend -client-id lab-backend
|
||
```
|
||
|
||
In a third terminal, send the HTTP command:
|
||
|
||
```sh
|
||
curl -i -X POST http://127.0.0.1:18080/devices/device-001/commands -H 'Content-Type: application/json' -H 'Idempotency-Key: lesson8-ping-001' --data '{"command":"ping"}'
|
||
|
||
# Replace cmd-example with the ID returned by the POST response.
|
||
curl -i http://127.0.0.1:18080/commands/cmd-example
|
||
```
|
||
|
||
`curl` is an HTTP client. `-i` prints response headers alongside the body; `-X POST` chooses the method; each `-H` sets a request header; `--data` sends the quoted JSON bytes. Content-Type tells the server how to interpret the body, while Idempotency-Key identifies this caller retry. This local exercise sends no production authorization token.
|
||
|
||
Copy the returned ID into `GET /commands/{id}`. Repeat the identical POST and confirm the same ID. Use a new key for a genuinely new command.
|
||
|
||
For example, if the returned ID is `cmd-example`, use `curl -i http://127.0.0.1:18080/commands/cmd-example`; without `--data` or an explicit method, curl performs GET. Replace that example ID with the actual returned value. HTTP 202 means admitted for asynchronous processing; inspect later status for device completion. `httptest` runs handlers against an in-process request/recorder or test server, letting you assert headers, status, and body without the live broker.
|
||
|
||
Set ReadHeaderTimeout, ReadTimeout, WriteTimeout, and IdleTimeout deliberately. Coordinate bounded HTTP shutdown with the MQTT and worker lifecycle.
|
||
|
||
```sh
|
||
mkdir -p internal/httpapi
|
||
|
||
gofmt -w internal cmd
|
||
|
||
go test -race ./...
|
||
|
||
go vet ./...
|
||
```
|
||
|
||
ReadHeaderTimeout bounds header reading; ReadTimeout bounds reading the request including its body; WriteTimeout bounds response writing; IdleTimeout bounds waiting for another request on an idle keep-alive connection. Choose values that fit this asynchronous API rather than waiting in POST for a device result. Server `Shutdown` stops accepting work and waits for active HTTP requests within its context deadline; your application must separately drain accepted command work.
|
||
|
||
`mkdir -p` creates the HTTP package directory, `gofmt` formats edits, and the test/race/vet commands check behavior, executed concurrent access, and suspicious static patterns respectively. Run directory creation before writing the files even though the check block collects the commands together.
|
||
|
||
Use `httptest` with a fake service for decoding, size limits, idempotency, unavailable admission, unknown IDs, and status responses. Add a service-level check that accepted work continues after the caller context is canceled.
|
||
|
||
**Done when:** HTTP creates a ping, status reflects device evidence, retries reuse its ID, and closing the caller connection does not erase accepted work.
|
||
|
||
## ✅ Result — the story so far
|
||
|
||
An API caller can now ask the backend to ping a device with HTTP. The handler validates the request and admits it through the command service; the caller receives HTTP 202 and a command ID without waiting for the device. The backend publishes the MQTT command, the simulator returns its application result, and a later GET reports the command's current evidence. Retrying the same POST with the same idempotency key returns the original command instead of starting a second effect. HTTP acceptance, MQTT acknowledgement, and device completion remain separate events.
|
||
|
||
Previous: [07-command-response.md](07-command-response.md). Next: [09-sessions-recovery.md](09-sessions-recovery.md).
|