This cancellation chain is exactly the trap people fall into when they assume abort() is a kill switch. The part I'd add from running agents in production: even when the provider does honor cancellation, the stop point is non-deterministic, so your own accounting has to treat a cancelled request as "cost unknown, upper-bounded by max_tokens" rather than zero. We log the last usage frame we actually received on the stream and reconcile against the provider's reported usage later, because the two disagree often enough to matter on the monthly bill. The other thing worth separating out is side effects already in flight — if the model emitted a tool call before you aborted, the HTTP cancel does nothing for the work that tool kicked off downstream. We ended up making tool execution idempotent and cancellation-aware for that reason. Did you find any providers that actually surface a reliable "stopped at token N" signal, or is it all inference from the usage object?
In my DeepSeek test, no: there was no stop signal at all. An aborted stream delivered no usage frame either, so I could only infer the stop point from billing, and only coarsely. I also looked at OpenAI and Anthropic's docs. OpenAI's Responses API has an explicit cancel endpoint, but only for background: true responses. Its API reference shows a cancelled response with usage: null. For synchronous requests the guide just says to terminate the connection. Anthropic's message_delta usage is cumulative, so you only have what arrived before you disconnected.
So I'd keep three things apart: the provider acknowledging a cancel, generation actually stopping, and final usage. Treating a cancelled request's cost as unknown, bounded by max_tokens plus the prompt, until you can reconcile against reported usage, seems right to me.
And agreed on tool calls. Cancelling the LLM request doesn't undo work a tool already started, so tools need to be idempotent and cancellation-aware.
The opening snippet already carries a useful signal. AbortSignal.timeout() aborts with a TimeoutError DOMException, AbortSignal.any() passes along the reason of whichever source fired first, and fetch rejects with that reason. So checking err.name === 'TimeoutError' separates "the provider was too slow" from "the user closed the tab" without extra state. Those are different stories for your cancellation chain: the timeout is the one where you're more likely paying for tokens nobody read. I'd log the two separately, and keep the disconnect case out of error tracking, or it fills up with things that aren't bugs.
Good point, and it's a cleaner split than anything in the article. Checking err.name === 'TimeoutError' tells you "provider too slow" apart from "user left", with no extra state. The timeout case is the one where you're more likely paying for tokens nobody read, so logging the two separately makes sense. I'd also keep plain disconnects out of error tracking.
One thing that makes the timeout case harder to price: with streaming, providers often send token usage only in the final chunk. OpenAI's stream_options: { include_usage: true } works that way. Abort mid-stream and that chunk never arrives, so your own logs can't say what the cut-off request cost. I'd record the abort reason, the model and the bytes received at that moment, then reconcile against the provider's usage report later. Did any provider in your tests bill noticeably fewer tokens after an abort?
Yes, for the one provider I tried. I only tested DeepSeek, and the balance drop after aborts was close to what the received tokens predicted. It was 0.01 and 0.06 CNY, against about 0.42 CNY for running to max_tokens. No usage frame arrived after the abort, so it's inferred from billing, not counted. I haven't tried other providers, so I can't say whether they behave the same. Your suggestion of logging the abort reason, model and bytes received, then reconciling against the provider's report later, is what I'd do too. It gives you a range, not a number.
Starting the timer at the first content chunk removes a major confounder: a request aborted while queued says little about generation cancellation. For the balance-based experiment, I would add an idle settlement window before each batch and alternate the order of the 3-second and 15-second conditions.
With a 9–13 minute accounting delay, unrelated requests or a previous batch settling late could contaminate the next measurement. Bracketing batches with completed control requests would also reveal price or accounting drift. The resulting conclusion would still be coarse, but the observed 0.06 versus predicted 0.42 would have a cleaner attribution.
Fair points. Only that key was active on the account during the runs, so other traffic isn't a concern. What I didn't do is alternate the 3 s and 15 s order or bracket with control requests, so a late-settling batch of my own could have leaked into the next one. The 0.06 vs 0.42 CNY gap is wide enough that I'd expect the conclusion to hold, but your design would give a cleaner attribution.
Cancellation needs a capability model, not a UI promise. A useful status surface distinguishes “client detached,” “queued work cancelled,” “tool call cancelled,” and “side effect already committed,” with a correlation ID across each boundary. Otherwise Abort becomes a misleading word precisely when the workflow has external authority.
Separating HTTP cancellation, inference cancellation and billing into three questions is the clearest way I have seen this explained. Most teams assume the first one implies the other two.
Max's point that the cost is unknown but bounded by max_tokens is what I would put in front of a client. You cannot promise that an abort saves money, but you can promise the worst case per request, and that is enough to set a budget.
The abort and regenerate case from melonape is the one that quietly doubles spend. A user who hits stop and asks again may be paying for both answers, so the second request should probably carry a lower cap or reuse what already streamed.
At Ascendra Ventures we include a per request cost ceiling in the acceptance criteria for AI features for exactly this reason. Did the DeepSeek test show the billed tokens matching what streamed, or the full completion?
The balance drop was close to the prediction from the characters received (0.01 and 0.06 CNY), and far below the full-run cost (0.42 CNY). No usage frame arrives after an abort, so this is inferred from billing, not counted per request, and the meter has two decimals. It can rule out running to max_tokens, but not a few tokens of overshoot. Your per-request ceiling fits either way.
Both, but at different layers, and only one of them is guaranteed.
On your side, the abort closes the connection and your client stops reading, so the stream ends for your application. That part is certain.
Whether the provider then stops generating depends on the provider. I measured one: DeepSeek, streaming chat requests aborted mid-generation. Billing followed the abort point and stayed far below what running to max_tokens would have cost. That suggests it stopped generating near the abort rather than finishing in the background, but it's indirect evidence. I could only see the bill, not the inference, and a few tokens of overshoot would be below what my meter could resolve.
Other providers can behave differently. OpenAI's background mode, for example, keeps running after the connection drops unless you call its cancel endpoint. So "I aborted" is safe to assume for your own process, and anything about the provider is worth measuring.
A technologist working in automation. Here I write up what I actually learn building tools, engineering AI agents, and keeping systems running, along with the workflows and habits that stick.
The gap between HTTP connection drop and scheduler eviction is where the surprises happen. On hosted inference, a client disconnect sends an RST_STREAM to the reverse proxy, but the worker batch scheduler might keep running the sequence until the next iteration boundary or chunk flush. In production we see cancelled calls log almost full generation tokens because the provider finished the forward passes before the gateway tear-down reached the worker pool. If you run your own vLLM or Triton instances you can tune sequence preemption directly, but with third-party APIs you have to assume any in-flight batch completes and charges accordingly.
That matches the conservative stance for third-party APIs. My result is one provider, and a few tokens of overshoot past the abort would be below what my balance-based meter can resolve. I've added that as an explicit limit.
I write when an idea won't leave me alone 🧠 Building AI agents and the tools to build AI agents. Love connecting AI with other fields and yapping about all of them.
You start the abort timer at the first content chunk, which holds up with thinking disabled. With thinking on, all the reasoning is generated before that first content chunk arrives, so the 3 s and 15 s batches would both carry the full reasoning cost. For a thinking-mode run I would start the timer on the first reasoning chunk instead.
You're right. With thinking on, the reasoning is generated before the first content chunk, so both batches would carry the full reasoning cost. This test had thinking disabled. For that case the timer should start on the first reasoning chunk, as you say
The distinction between stopping a request and stopping the work it initiated is especially important in agentic systems. An agent might trigger multiple downstream operations before the cancellation signal reaches the provider. At IT Path Solutions, we consider cancellation-aware workflows, execution tracking, and reliable recovery mechanisms important parts of building production-ready AI agents, particularly when they interact with external services.
Spot on analysis. The gap between fetch.abort() and actual upstream provider GPU cutoff is one of the most under-documented friction points in agent architectures.
In our multi-agent pipelines, we ran into an even thornier derivative: In-Flight Tool Call Orphans.
When a user or orchestrator triggers an abort signal while an agent is streaming tool-call arguments:
If your framework eagerly dispatches tool execution on partial schema validation, you end up spawning background zombie tasks (CLI child processes, partial file mutations, or untracked webhook calls).
The client disconnected, the provider kept billing until the chunk boundary, and your backend now has uncollected side effects.
We mitigated this by decoupling the agent loop into a strict Two-Phase Commit:
Phase 1 (Ephemeral Buffer): All LLM outputs stream into an isolated memory buffer bound to the request's AbortSignal. If aborted, the entire buffer is discarded immediately—zero I/O permitted.
Phase 2 (Deterministic Gate): Downstream tools only execute once the stream emits a clean finish_reason: "stop" / tool_calls and passes a schema validator.
Pairing that with idempotent, disk-persisted artifact manifests rather than long in-memory chat state has completely eliminated half-applied agent actions during sudden disconnects. Great writeup on the billing lag as well—definitely caught us off guard early on!
The orphan case is real. Cancelling the stream does nothing for a tool the framework has already started. Buffering until a clean finish_reason and schema validation closes that hole for tool-call streams, and the idempotent manifests cover the retry side.
The cost is latency, since tools can't start until the stream completes. It also doesn't help if the abort arrives after a tool is already running.
This cancellation chain is exactly the trap people fall into when they assume
abort()is a kill switch. The part I'd add from running agents in production: even when the provider does honor cancellation, the stop point is non-deterministic, so your own accounting has to treat a cancelled request as "cost unknown, upper-bounded by max_tokens" rather than zero. We log the lastusageframe we actually received on the stream and reconcile against the provider's reported usage later, because the two disagree often enough to matter on the monthly bill. The other thing worth separating out is side effects already in flight — if the model emitted a tool call before you aborted, the HTTP cancel does nothing for the work that tool kicked off downstream. We ended up making tool execution idempotent and cancellation-aware for that reason. Did you find any providers that actually surface a reliable "stopped at token N" signal, or is it all inference from the usage object?Thanks, these are good production points.
In my DeepSeek test, no: there was no stop signal at all. An aborted stream delivered no usage frame either, so I could only infer the stop point from billing, and only coarsely. I also looked at OpenAI and Anthropic's docs. OpenAI's Responses API has an explicit cancel endpoint, but only for background: true responses. Its API reference shows a cancelled response with usage: null. For synchronous requests the guide just says to terminate the connection. Anthropic's message_delta usage is cumulative, so you only have what arrived before you disconnected.
So I'd keep three things apart: the provider acknowledging a cancel, generation actually stopping, and final usage. Treating a cancelled request's cost as unknown, bounded by max_tokens plus the prompt, until you can reconcile against reported usage, seems right to me.
And agreed on tool calls. Cancelling the LLM request doesn't undo work a tool already started, so tools need to be idempotent and cancellation-aware.
The opening snippet already carries a useful signal.
AbortSignal.timeout()aborts with aTimeoutErrorDOMException,AbortSignal.any()passes along the reason of whichever source fired first, and fetch rejects with that reason. So checkingerr.name === 'TimeoutError'separates "the provider was too slow" from "the user closed the tab" without extra state. Those are different stories for your cancellation chain: the timeout is the one where you're more likely paying for tokens nobody read. I'd log the two separately, and keep the disconnect case out of error tracking, or it fills up with things that aren't bugs.Good point, and it's a cleaner split than anything in the article. Checking err.name === 'TimeoutError' tells you "provider too slow" apart from "user left", with no extra state. The timeout case is the one where you're more likely paying for tokens nobody read, so logging the two separately makes sense. I'd also keep plain disconnects out of error tracking.
One thing that makes the timeout case harder to price: with streaming, providers often send token usage only in the final chunk. OpenAI's
stream_options: { include_usage: true }works that way. Abort mid-stream and that chunk never arrives, so your own logs can't say what the cut-off request cost. I'd record the abort reason, the model and the bytes received at that moment, then reconcile against the provider's usage report later. Did any provider in your tests bill noticeably fewer tokens after an abort?Yes, for the one provider I tried. I only tested DeepSeek, and the balance drop after aborts was close to what the received tokens predicted. It was 0.01 and 0.06 CNY, against about 0.42 CNY for running to max_tokens. No usage frame arrived after the abort, so it's inferred from billing, not counted. I haven't tried other providers, so I can't say whether they behave the same. Your suggestion of logging the abort reason, model and bytes received, then reconciling against the provider's report later, is what I'd do too. It gives you a range, not a number.
Starting the timer at the first content chunk removes a major confounder: a request aborted while queued says little about generation cancellation. For the balance-based experiment, I would add an idle settlement window before each batch and alternate the order of the 3-second and 15-second conditions.
With a 9–13 minute accounting delay, unrelated requests or a previous batch settling late could contaminate the next measurement. Bracketing batches with completed control requests would also reveal price or accounting drift. The resulting conclusion would still be coarse, but the observed 0.06 versus predicted 0.42 would have a cleaner attribution.
Fair points. Only that key was active on the account during the runs, so other traffic isn't a concern. What I didn't do is alternate the 3 s and 15 s order or bracket with control requests, so a late-settling batch of my own could have leaked into the next one. The 0.06 vs 0.42 CNY gap is wide enough that I'd expect the conclusion to hold, but your design would give a cleaner attribution.
Cancellation needs a capability model, not a UI promise. A useful status surface distinguishes “client detached,” “queued work cancelled,” “tool call cancelled,” and “side effect already committed,” with a correlation ID across each boundary. Otherwise Abort becomes a misleading word precisely when the workflow has external authority.
Agreed. The next article lists those four outcomes explicitly, with a correlation ID across boundaries. I've also added that to the Part 3 draft.
Separating HTTP cancellation, inference cancellation and billing into three questions is the clearest way I have seen this explained. Most teams assume the first one implies the other two.
Max's point that the cost is unknown but bounded by max_tokens is what I would put in front of a client. You cannot promise that an abort saves money, but you can promise the worst case per request, and that is enough to set a budget.
The abort and regenerate case from melonape is the one that quietly doubles spend. A user who hits stop and asks again may be paying for both answers, so the second request should probably carry a lower cap or reuse what already streamed.
At Ascendra Ventures we include a per request cost ceiling in the acceptance criteria for AI features for exactly this reason. Did the DeepSeek test show the billed tokens matching what streamed, or the full completion?
The balance drop was close to the prediction from the characters received (0.01 and 0.06 CNY), and far below the full-run cost (0.42 CNY). No usage frame arrives after an abort, so this is inferred from billing, not counted per request, and the meter has two decimals. It can rule out running to max_tokens, but not a few tokens of overshoot. Your per-request ceiling fits either way.
Does the abort actually kill the token stream, or just stop waiting for it?
Both, but at different layers, and only one of them is guaranteed.
On your side, the abort closes the connection and your client stops reading, so the stream ends for your application. That part is certain.
Whether the provider then stops generating depends on the provider. I measured one: DeepSeek, streaming chat requests aborted mid-generation. Billing followed the abort point and stayed far below what running to max_tokens would have cost. That suggests it stopped generating near the abort rather than finishing in the background, but it's indirect evidence. I could only see the bill, not the inference, and a few tokens of overshoot would be below what my meter could resolve.
Other providers can behave differently. OpenAI's background mode, for example, keeps running after the connection drops unless you call its cancel endpoint. So "I aborted" is safe to assume for your own process, and anything about the provider is worth measuring.
The bill as a check makes sense to me. For background mode I will treat the cancel call as part of the abort, not an extra.
The gap between HTTP connection drop and scheduler eviction is where the surprises happen. On hosted inference, a client disconnect sends an RST_STREAM to the reverse proxy, but the worker batch scheduler might keep running the sequence until the next iteration boundary or chunk flush. In production we see cancelled calls log almost full generation tokens because the provider finished the forward passes before the gateway tear-down reached the worker pool. If you run your own vLLM or Triton instances you can tune sequence preemption directly, but with third-party APIs you have to assume any in-flight batch completes and charges accordingly.
That matches the conservative stance for third-party APIs. My result is one provider, and a few tokens of overshoot past the abort would be below what my balance-based meter can resolve. I've added that as an explicit limit.
You start the abort timer at the first content chunk, which holds up with thinking disabled. With thinking on, all the reasoning is generated before that first content chunk arrives, so the 3 s and 15 s batches would both carry the full reasoning cost. For a thinking-mode run I would start the timer on the first reasoning chunk instead.
You're right. With thinking on, the reasoning is generated before the first content chunk, so both batches would carry the full reasoning cost. This test had thinking disabled. For that case the timer should start on the first reasoning chunk, as you say
The distinction between stopping a request and stopping the work it initiated is especially important in agentic systems. An agent might trigger multiple downstream operations before the cancellation signal reaches the provider. At IT Path Solutions, we consider cancellation-aware workflows, execution tracking, and reliable recovery mechanisms important parts of building production-ready AI agents, particularly when they interact with external services.
Spot on analysis. The gap between fetch.abort() and actual upstream provider GPU cutoff is one of the most under-documented friction points in agent architectures.
In our multi-agent pipelines, we ran into an even thornier derivative: In-Flight Tool Call Orphans.
When a user or orchestrator triggers an abort signal while an agent is streaming tool-call arguments:
We mitigated this by decoupling the agent loop into a strict Two-Phase Commit:
Pairing that with idempotent, disk-persisted artifact manifests rather than long in-memory chat state has completely eliminated half-applied agent actions during sudden disconnects. Great writeup on the billing lag as well—definitely caught us off guard early on!
The orphan case is real. Cancelling the stream does nothing for a tool the framework has already started. Buffering until a clean finish_reason and schema validation closes that hole for tool-call streams, and the idempotent manifests cover the retry side.
The cost is latency, since tools can't start until the stream completes. It also doesn't help if the abort arrives after a tool is already running.
the nasty combo is abort + regenerate. user hits regenerate, the first one might still be running on the provider side, and now you're paying for two