Data subscriptions
Everything else in this section is about looking at data. A subscription reverses the direction: data comes to you, the moment it arrives, pushed over a standing connection rather than fetched on a schedule.
Delivery is millisecond-scale inside the platform, and that is not a performance brag, it is a category change: a whole set of use cases only exists below the latency of a polling loop.
Push, not poll
Most integrations ask on a schedule: every minute, every five, every fifteen. That puts a floor under how fresh anything downstream can be, on average half the polling interval, no matter how fast the data itself moves. It also scales badly, because every consumer asks again whether anything happened, mostly to hear that nothing did.
A subscription is a standing request. You say once what you want to hear about, and the platform delivers each new datapoint as it lands.
The two kinds
| Live tail | Durable subscription | |
|---|---|---|
| For | Watching, a person or a chart | Delivering, a system that must not miss data |
| Starts from | Now | Where you left off |
| If you disconnect | You miss what happened while away | Nothing is lost: your position is held on the platform, and delivery resumes from it |
| Acknowledgement | None, it is a view | Each delivery is acknowledged, so unconfirmed data is redelivered |
| You meet it in | Insights Live mode | Integrations built by your team |
The distinction matters more than it looks. A wall display can use a live tail, because a missed minute during a reconnect is harmless. A downstream system mirroring data, or an agent acting on it, needs the durable kind, where a dropped connection is an inconvenience rather than a gap in the record.
Two properties of durable subscriptions are worth knowing even if you never build one:
- Reconnects resume anywhere. Your position is held by the platform, not by the server you happened to be talking to, so a reconnect lands on any instance and carries on. No data loss, no special infrastructure.
- Filtering happens platform-side. You subscribe to the series you care about, and only those are delivered. A subscriber is not flooded with everything and left to discard.
What millisecond delivery opens up
The honest way to see the value: list what is impossible at five-minute latency and routine below one second.
A threshold crossing reaches the person on shift while the excursion is still starting, not on the next dashboard refresh. The difference between reacting and reviewing.
A perception agent is only as fast as its data. Fed by subscription, it evaluates every reading as it lands, which is what makes early-warning agents early.
A system that needs your operational data can subscribe to it instead of receiving nightly exports, always current, no batch window, no reconciliation morning.
Watching a machine through a start-up sequence, or an incident as it develops, with the chart moving as the plant moves. This one you can use today with no code at all.
And the one worth flagging for later: closed-loop reactions, where a detected condition triggers an adjustment, are only responsible at subscription latency. That road runs through the guardrails on the agents page, but it starts here, with delivery fast enough to react to. The chain drawn end to end, a detection function firing, the event waking an agent, the agent's draft reaching a person, is on the functions page. Push delivery is also what moves a model from knowing what exists to knowing what is happening, level two of the twin maturity ladder.
Being honest about the milliseconds
"Millisecond latency" means delivery inside the platform: from a datapoint landing to it being on the wire to subscribers. Your own network path adds whatever it adds, and a subscriber that processes slowly adds more. The claim to design against is no schedule in the way, not a specific number on your WAN. Acknowledged redelivery also means a consumer can see the same datapoint twice after a reconnect, so build handlers idempotent.
The other side of the deal: a durable subscriber that stops consuming leaves the platform holding its undelivered data. Build consumers that keep up, and treat a subscription's credentials like any other integration's, a service account with the narrowest access that works.
A worked example: handling a noisy alarm
Subscriptions deliver readings. The first thing most teams want to do with them is not store them, it is to stop reacting to every single one.
The problem. A pump's discharge pressure carries a high limit, and the limit alarm fires on every start-up, every slug of gas and every transient the process throws at it. Four hundred alarms a month, almost all of them clear within seconds. Somebody suppressed it two years ago, and on the one occasion it mattered nobody was watching. Moving the limit does not help: higher and it misses the slow drift, lower and it fires more.
What the control room actually wants is not is it above 4.5 but has it stayed above 4.5, and that is a question about a stretch of time rather than about a reading.
Why the window has to slide
Tumbling windows cut time into fixed slices, which is right for reporting and wrong here. A twenty-five minute excursion straddling two fifteen-minute slices looks like two partial slices, neither of them sustained, so the rule stays quiet through exactly the event it was built for. A sliding window always has the last twenty minutes in view, whatever the clock happens to say.
The design
Filtering happens platform-side, so the consumer receives only what it asked for. The service account needs read access to every series bound to the subscription; a series it cannot read is refused explicitly rather than arriving as silence, which is the failure mode worth knowing about when somebody adds a series later. Service accounts →
A short buffer of readings, trimmed against the newest reading's own time rather than the wall clock. That one choice is what makes the rule behave correctly when a reconnect delivers four hours at once. Why arrival time is the wrong clock →
Raise when the whole window has been above the limit; clear only after ten minutes below it. Raise and clear thresholds must differ, or the rule flaps at the boundary and you have reinvented the nuisance alarm with extra steps.
One event on the pump, carrying the window it was judged over, how long the condition held and the peak value. The next person to open that pump sees the finding and its evidence together, and every later question can count these events.
The whole loop is smaller than the discussion around it:
WINDOW = timedelta(minutes=20)
recent = deque() # (timestamp, value), oldest first
with client.subscriptions.listen(["pump_p101_discharge"]) as listener:
for msg in listener:
# the payload carries the action and the datapoints it affected
for ts, value in datapoints(msg.payload):
if any(t == ts for t, _ in recent):
continue # a redelivery, not a new reading
recent.append((ts, value))
# evict on the carried timestamp, never on the wall clock
while recent and recent[0][0] < recent[-1][0] - WINDOW:
recent.popleft()
evaluate(recent) # raise, clear, or stay quiet
listener.ack([msg.message_id])
The four things that make it survive contact with reality
- A reading can arrive twice. Delivery is at least once, so a reconnect can replay what you already handled. The deduplication above is what stops a redelivery counting twice toward a sustained condition. Writing back is safer than reading: datapoints are keyed by series and timestamp, and events by id, so the platform collapses the duplicates you send it. Your own window is the part nobody protects for you.
- Fill the window before you trust it. On start-up, read the last twenty minutes through the ordinary API rather than waiting twenty minutes to become useful. A rule that is blind for its first twenty minutes is blind precisely when somebody has just restarted it during an incident.
- A window with holes is not a quiet window. If three of the twenty expected readings arrived, the honest output is not "no exceedance", it is that the instrument or the link needs looking at. Raise that as its own quiet finding rather than letting missing data read as good news.
- Acknowledge after you have acted, not before. The ack is a promise that the reading has been durably handled. Acking first turns a crash into a silent gap in the very rule you built to be trustworthy.
What it is worth, and where it goes next
One event a month instead of four hundred alarms, and an alarm the control room has not
switched off. The shape generalises further than it looks:
sustained above, sustained below, rate of change across the window,
the ratio between two series over the window are all the same loop with a different
evaluate.
When no threshold over any window can express the thing you are looking for, because the readings never leave their limits and it is the shape that changed, that is where a sequence model starts earning its keep. And when functions ship, this loop becomes a definition the platform runs rather than a process you keep alive, without changing what it produces.
Where to start
- See it without writing code: open a series in Insights and go Live. That is a live tail over exactly this machinery.
- Build against it: subscriptions are consumed through the API and SDKs, which is developer territory, and they are the live feed behind most applications built on the platform. Developer and SDK documentation →
- Managing subscriptions in the console sits in the Streams area, whose documentation is being reworked alongside the streaming feature itself.
- Stream processing: what to do with the data once it is arriving continuously
- Insights Live mode: the no-code way to watch a subscription
- Events: the discrete occurrences worth subscribing to
- AI agents: what continuous watching makes possible
- Building AI agents: waking an agent on a reading rather than a timer
- Digital twins: the live connection that separates a twin from a picture
- Users and access: credentials for a subscribing system
- Architecture: why reconnects resume on any instance