DEV Community

AI Dev Hub
AI Dev Hub

Posted on

Reading Claude message_stream events without guessing in 2026

Reading Claude message_stream events without guessing in 2026

Log the raw SSE frames, then replay them through a parser that tracks content_block indices. Anthropic's stream is a typed event sequence, so text deltas, input_json_delta fragments for tool calls, and thinking blocks all arrive interleaved on separate indices. Reconstructing the message means grouping by index, not concatenating in arrival order.

Quick disclosure before anything else: the stream event viewer I link to further down is one I built. I'd been pasting SSE dumps into three different generic JSON formatters and a browser devtools panel, and every one of them choked on the fact that a message stream isn't one JSON document, it's a few hundred of them separated by data: lines. None of them knew what a content_block_delta was. Mine is free, runs client-side, has no signup, and uploads nothing. If you already use something better, please tell me what it is.

The bug that cost me a Tuesday afternoon

On March 12, 2026 I shipped a streaming endpoint that proxied Claude responses to a web client. It worked in every test I wrote. It broke in production for roughly 1 in 40 requests, and the failures were always the same shape: the assistant's answer would come through with a chunk of JSON spliced into the middle of a sentence.

My handler was doing the naive thing. For every content_block_delta it grabbed delta.text if present, delta.partial_json otherwise, and appended both to one buffer. That's fine as long as the model produces exactly one content block. The moment it emitted a short text preamble, then a tool_use block, then more text, my buffer became a blender.

I spent about 47 minutes staring at the wrong layer. I assumed the SDK was mis-ordering events, or that my reverse proxy was reassembling chunks badly. Neither. The events arrived in perfect order. I was throwing away the one field that mattered, which is index.

What finally fixed it was dumping the raw stream to a file and reading it end to end. Not the parsed objects my code produced, the actual bytes on the wire. That's a boring debugging move and it's the one I keep forgetting to do first.

What the event sequence actually looks like

A single streamed call from the Messages API is a fixed skeleton with a variable middle. You get message_start once, carrying the message envelope and the initial usage object. Then, for each content block, a content_block_start with an index and a stub of the block, a run of content_block_delta events on that same index, and a content_block_stop. At the end, message_delta carries stop_reason and the final output_tokens, followed by message_stop.

The deltas are typed, and the type tells you which field to read:

  • text_delta has .text
  • input_json_delta has .partial_json (a raw string fragment, only valid JSON once the whole block is concatenated)
  • thinking_delta has .thinking
  • signature_delta closes out an extended-thinking block

The partial_json one bites people. Each fragment is a slice of a JSON string, so {"loc and ation":" and Berlin"} arrive as three separate events. Parse them individually and you get an exception. Concatenate them across the block's entire lifetime and you get valid input for the tool call.

The other thing worth going after is usage accounting. message_start gives you input_tokens, cache_creation_input_tokens, and cache_read_input_tokens. On a cached run I checked last Tuesday, a request reported 218 input tokens and 14,208 cache read tokens. If you only log input_tokens you will look at that call and conclude it was nearly free, which is true, but you'll have no idea why, and no way to tell when your cache starts missing.

How the parser puts a call back together

Here's a small script that takes a saved SSE dump and reconstructs the whole call. It's the same logic the viewer runs, minus the UI. Save your stream by writing every raw line from the response body to a file first.

import json, sys

def parse_stream(path):
    text_parts, tool_json, usage = [], {}, {}
    blocks, stop_reason = {}, None
    with open(path) as fh:
        for line in fh:
            line = line.strip()
            if not line.startswith("data:"):
                continue
            ev = json.loads(line[5:].strip())
            t = ev.get("type")
            if t == "message_start":
                usage.update(ev["message"].get("usage", {}))
            elif t == "content_block_start":
                blocks[ev["index"]] = ev["content_block"]
                if ev["content_block"]["type"] == "tool_use":
                    tool_json[ev["index"]] = ""
            elif t == "content_block_delta":
                d = ev["delta"]
                if d["type"] == "text_delta":
                    text_parts.append(d["text"])
                elif d["type"] == "input_json_delta":
                    tool_json[ev["index"]] += d["partial_json"]
            elif t == "message_delta":
                stop_reason = ev["delta"].get("stop_reason")
                usage.update(ev.get("usage", {}))
    tools = {i: json.loads(s) for i, s in tool_json.items() if s}
    return {"text": "".join(text_parts), "tools": tools,
            "stop_reason": stop_reason, "usage": usage}

if __name__ == "__main__":
    print(json.dumps(parse_stream(sys.argv[1]), indent=2))
Enter fullscreen mode Exit fullscreen mode

Run it with python replay.py stream.txt and you get the assembled text, every tool call's fully-parsed input keyed by block index, the stop reason, and a merged usage object. Fifty lines, no dependencies.

Two details in there that took me longer than they should have. The line[5:] slice assumes data: with no space, which is why the .strip() follows it. And usage.update() on message_delta is deliberate: the final event only carries output_tokens, so updating rather than replacing keeps the input and cache counts from message_start intact. I got that backwards on my first pass and every call reported zero input tokens.

How it compares to what I tried first

I went through four options before writing my own. The comparison below reflects what each one did with a 900-line dump containing two text blocks and one tool call.

Tool Understands typed events Rebuilds tool_use JSON Cache token breakdown Data stays local
Generic JSON formatter No (fails on multi-doc SSE) No No Usually
Browser devtools EventStream tab Shows frames only No No Yes
Hand-rolled jq pipeline With enough effort Manual concat Manual Yes
Anthropic Stream Event Viewer Yes Yes Yes Yes, client-side

The devtools EventStream tab is genuinely useful and I still open it, just for a different job. It shows you that frames arrived and in what order. It won't tell you the tool input was truncated because the model hit max_tokens mid-JSON, which is a real failure mode and shows up as stop_reason: "max_tokens" with an unparseable partial_json accumulation.

The jq route works. I have a 12-line pipeline in a gist that handles text deltas fine. Extending it to group tool JSON by index is where I gave up and wrote actual code.

When you shouldn't bother with this

If you're using the official SDK's client.messages.stream() helper and you never touch tool use, skip all of it. The SDK accumulates the final message for you, exposes .text_stream for the simple case, and handles indices correctly. Reaching for a raw event parser there is work you don't need.

Same if your bottleneck is latency rather than correctness. An event viewer tells you what came back, not how fast. For time-to-first-token you want timestamps recorded at the socket, and no post-hoc replay of a saved dump will give you those.

And if your streams carry customer data under a policy that forbids pasting content into any web page, use the script above locally instead. The viewer runs entirely in your browser and sends nothing anywhere, but "trust me, it's client-side" is not a compliance argument, and I wouldn't expect anyone to accept it as one. Read the network tab or run the 50 lines yourself.

The case where this pays off is the messy middle: you're building on the raw HTTP API, or through a gateway that reshapes events, or debugging why a tool call sometimes arrives with an empty input object. That last one turned out, for me, to be a proxy that buffered and split SSE frames at 4,096 bytes without respecting event boundaries. I would never have found it from parsed objects.

FAQ

Q: Do I need to log the raw stream, or can I feed it the SDK's parsed events?
A: Raw is better. The SDK's accumulated message hides exactly the ordering and index information you're trying to inspect. Write the response body lines to a file before anything parses them.

Q: Why does partial_json fail to parse on its own?
A: Because each delta is an arbitrary byte slice of the tool input JSON, split wherever the token boundary landed. Only the concatenation of every fragment in that block is valid JSON.

Q: What does stop_reason: "tool_use" mean versus "end_turn"?
A: tool_use means the model stopped because it wants a tool result back before continuing. Send the result as a tool_result block in the next user turn. end_turn means it finished on its own.

Q: Where do cache write and cache read counts show up?
A: In the usage object on message_start, as cache_creation_input_tokens and cache_read_input_tokens. They're separate from input_tokens, so summing all three gives you the real prompt size.

Q: Does the viewer work with streams from Bedrock or Vertex?
A: Mostly. The event types match, though the envelope framing differs by platform, so you may need to strip a wrapper before pasting.

Written with AI assistance and human review. Try the tool at aidevhub.io/anthropic-stream-event-viewer.

Top comments (0)