In 1999, two engineers named Andy Stanford-Clark and Arlen Nipper faced a brutal engineering problem:
They needed to monitor crude oil pipelines s...
For further actions, you may consider blocking this person and/or reporting abuse
The distinction between protocol delivery guarantees and application processing is worth emphasizing. Even with a higher QoS, a subscriber may need idempotent processing when an operation can be delivered more than once.
Thanks Alex, well said. QoS only covers one hop, and the effective level is the lower of the publisher and subscriber sides. So the consumer should always dedupe, for example with a unique message ID in the payload.
Yes — and the dedupe key needs a retention policy that matches the business effect. A payment or provisioning action may need a longer ledger than telemetry, while still including publisher or session context so unrelated operations cannot collide.
The clearest takeaway for me is that a topic is a broadcast channel, not a queue, so a message with no subscriber simply disappears. That makes the persistent session exception important: do you treat it as a safety net for brief disconnects, or do you avoid relying on it because it only buffers for one client? I would also be interested in how you decide between a retained message and a separate state query for a dashboard that boots up later.
Thanks Arjun! I treat persistent sessions as a safety net for brief disconnects, not as storage. They are per client, and limited by session expiry and queue size. For anything that must not be lost, I'd put a durable queue or database behind the subscriber. For dashboards, retained messages work well for small last-known values like device status. For history, aggregates or larger state, I'd query a database instead.
That boundary is helpful: a persistent session for brief disconnects, a durable store behind the subscriber for anything that must not be lost, and retained messages only for small last known values. How do you size the session expiry and queue limit in practice so a long outage does not silently turn into lost telemetry?
The persistent-session section is where I’d draw a second boundary: a message reaching a subscriber still leaves the application’s work to finish. What happens if a worker changes the database and then crashes before recording completion? I’d put that case beside this fault-tolerant queue exercise and decide how a retry recognizes work already applied. That makes the choice between telemetry delivery and recoverable jobs much easier to explain.
Great point, Emily. Persistent sessions only cover delivery to the client, not what the app does after. I'd ack only after the DB commit, and expect retries since delivery is at least once. Then store a message ID with a unique constraint in the same transaction as the change, so a retry sees the work is already applied and skips it. If the jobs must be recoverable, I'd use a real queue and keep MQTT for telemetry.
That transaction boundary is exactly the part I wanted to make explicit. I’d put a crash immediately after the commit into the example, then replay the same application message ID and check that the business change still appears once. Thanks for working through the failure case.
The moment engineers realize MQTT has no queues, they invariably abuse QoS 2 to fake one—reinventing half of TCP's state machine in userspace only to watch their broker's memory evaporate on lossy edge links.
Haha, true. Thanks Bohdan! QoS only guarantees delivery per hop, it doesn't make the broker a queue. For queue-like behavior I'd go with a persistent session, QoS 1 and an idempotent consumer. For real durability, bridge to Kafka or RabbitMQ.
the vanishing message is the part that surprises people. no persistent session, no second chance.
Thanks Odinaka! Yes, it surprises a lot of people. A persistent session with QoS 1 or 2 helps, as long as the client subscribed before going offline. Retained messages are also good for "last known value" cases.
That split is useful: persistent session for what a subscriber missed, retained for the last known value a new subscriber needs on arrival.
Great explanation! The distinction between pub/sub and traditional message queues is subtle but important, especially when designing IoT systems.
Thanks! Glad it helped. Picking pub/sub vs a queue early saves a lot of pain later in IoT designs.
The distinction is important because it changes where reliability belongs. QoS describes transport behavior, while the business effect still needs an application-level idempotency key, dedupe window, and observability for delayed or replayed messages.