DEV Community

Peyton Green
Peyton Green

Posted on

Why Your LangGraph ToolNode Tests Are Failing (And How to Fix Them)

#python #langgraph #testing #ai

You wrote a clean unit test for your LangGraph tool. It worked fine for months. Then you upgraded your dependencies and got this:

ValueError: Missing required config key 'N/A' for 'tools'.
Enter fullscreen mode Exit fullscreen mode

Or maybe:

TypeError: custom_afunc() got an unexpected keyword argument 'runtime'
Enter fullscreen mode Exit fullscreen mode

Or your tests pass silently while error handling is completely off in production.

You didn't change your code. The framework changed around you.

This is a targeted postmortem of the langgraph-prebuilt 1.0.x breakage cluster — and a practical guide to writing LangGraph agent tests that actually hold up.

What Broke and Why

In October 2025, langgraph-prebuilt 1.0.2 added a required runtime parameter to ToolNode internals. The change was real and necessary for LangGraph's new execution model. The problem: it broke standard unit testing patterns without a migration guide, without a DeprecationWarning, and without being pinned correctly in langgraph==1.0.1.

If you did a clean pip install after October 29, 2025, your ToolNode unit tests silently started failing.

As of langgraph-prebuilt 1.0.8 (February 2026), three of the original four issues remain open. Update, September 2026: they still are — issues #6397 (runtime required on a bare ToolNode.invoke()) and #6486 (handle_tool_errors off by default since 1.0.2) are both open against current LangGraph (verified 1.2.11). The patterns below are still the fix, not a temporary workaround for an old version.

The Four Failure Modes

1. Unit Testing ToolNode Outside Graph Context (Issue #6397 — STILL OPEN)

Symptom:

from langgraph.prebuilt import ToolNode
from langchain_core.messages import HumanMessage

def multiply(a: int, b: int) -> int:
    """Multiply two numbers."""
    return a * b

tool_node = ToolNode(tools=[multiply])

# This crashes
result = tool_node.invoke([HumanMessage(content="what is 3 * 4?")])
Enter fullscreen mode Exit fullscreen mode 
ValueError: Missing required config key 'N/A' for 'tools'.
Enter fullscreen mode Exit fullscreen mode

ToolNode.invoke() outside a compiled graph has no runtime context injected. The node panics.

Fix:

from langgraph.runtime import Runtime

result = tool_node.invoke(
    [HumanMessage(content="what is 3 * 4?")],
    runtime=Runtime()
)
Enter fullscreen mode Exit fullscreen mode

Runtime() with no arguments gives the node the context it needs without wiring up a full graph.

2. Custom Async Tool Functions (Issue #6363 — STILL OPEN)

Symptom:

async def my_afunc(messages):
    return "result"

tool_node = ToolNode(tools=[my_afunc])
await tool_node.ainvoke(messages)
Enter fullscreen mode Exit fullscreen mode 
TypeError: my_afunc() got an unexpected keyword argument 'runtime'
Enter fullscreen mode Exit fullscreen mode

LangGraph passes runtime as a keyword argument to all callables it invokes. If your function signature doesn't accept it, it crashes.

Fix:

async def my_afunc(messages, **kwargs):
    # kwargs absorbs 'runtime' and any future injected params
    return "result"
Enter fullscreen mode Exit fullscreen mode

This is the defensive pattern. Absorbing **kwargs future-proofs against additional injected parameters.

3. Error Handling Regressed to Off (Issue #6486 — STILL OPEN)

Symptom: Your tools silently fail. No exception raised. The agent continues as if nothing happened.

This one is nasty because it doesn't produce a traceback. handle_tool_errors defaulted to True in earlier LangGraph versions. At 1.0.x, it reverted to False.

Any graph that relied on the default — which is all of them — silently lost error handling without a code change.

Fix:

tool_node = ToolNode([my_tool], handle_tool_errors=True)
Enter fullscreen mode Exit fullscreen mode

Never rely on the default. Always pass it explicitly. Add this as a lint rule in your project.

4. CancelledError Not Caught (Issue #6726 — OPEN, January 2026)

In async execution, asyncio.CancelledError propagates through ToolNode uncaught when timeout logic cancels a task. Your tool's cleanup code doesn't run. This surfaces as flaky test behavior under load.

There's no clean workaround until the upstream fix lands. Defensively, wrap tool implementations in try/except asyncio.CancelledError for tools that manage external connections or file handles.

Testing Patterns That Hold Up

Setup: Dependencies and pytest config 

# pyproject.toml
[tool.pytest.ini_options]
asyncio_mode = "auto"

[project.dependencies]
langgraph = ">=1.0.2"          # the runtime= and handle_tool_errors= fixes both need this floor;
                                # no upper bound needed — verified working through 1.2.11
langgraph-prebuilt = ">=1.0.2"
langchain-core = ">=0.3.0"
pytest = ">=8.0"
pytest-asyncio = ">=0.23"
Enter fullscreen mode Exit fullscreen mode

Pin the minor version. This project has a history of breaking changes on minor bumps.

Pattern 1: Unit Testing a Tool in Isolation 

import pytest
from langchain_core.messages import AIMessage, ToolMessage
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
from langgraph.prebuilt import ToolNode
from langgraph.runtime import Runtime


@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    # In real code this calls an API
    return f"The weather in {city} is sunny, 22°C"


@pytest.fixture
def weather_tool_node():
    return ToolNode(tools=[get_weather])


def test_tool_node_unit(weather_tool_node):
    """Test ToolNode directly without a compiled graph."""
    # ToolNode expects an AIMessage with tool_calls
    message = AIMessage(
        content="",
        tool_calls=[{
            "id": "call_001",
            "name": "get_weather",
            "args": {"city": "Berlin"},
        }]
    )

    # Runtime() is the fix for issue #6397
    result = weather_tool_node.invoke([message], runtime=Runtime())

    assert len(result) == 1
    tool_message = result[0]
    assert isinstance(tool_message, ToolMessage)
    assert "Berlin" in tool_message.content
    assert tool_message.tool_call_id == "call_001"
Enter fullscreen mode Exit fullscreen mode

Pattern 2: Testing Async Tool Execution 

import asyncio
import pytest
from langchain_core.messages import AIMessage, ToolMessage
from langchain_core.tools import tool
from langgraph.prebuilt import ToolNode
from langgraph.runtime import Runtime


@tool
async def fetch_price(ticker: str) -> float:
    """Fetch the current price of a stock ticker."""
    await asyncio.sleep(0)  # real implementation: await http_client.get(...)
    prices = {"AAPL": 189.50, "GOOGL": 175.20, "MSFT": 420.00}
    return prices.get(ticker, 0.0)


@pytest.fixture
def price_tool_node():
    return ToolNode(
        tools=[fetch_price],
        handle_tool_errors=True,  # always explicit — #6486
    )


async def test_async_tool_node(price_tool_node):
    """Test async ToolNode execution."""
    message = AIMessage(
        content="",
        tool_calls=[{
            "id": "call_002",
            "name": "fetch_price",
            "args": {"ticker": "AAPL"},
        }]
    )

    result = await price_tool_node.ainvoke([message], runtime=Runtime())

    assert len(result) == 1
    assert isinstance(result[0], ToolMessage)
    assert "189.5" in result[0].content
Enter fullscreen mode Exit fullscreen mode

Pattern 3: Testing Multiple Tool Calls in One Message

LangGraph agents can emit multiple tool calls in a single response. Test this directly.

async def test_multiple_tool_calls(price_tool_node):
    """LangGraph batches tool calls — test both in one invoke."""
    message = AIMessage(
        content="",
        tool_calls=[
            {"id": "call_003", "name": "fetch_price", "args": {"ticker": "AAPL"}},
            {"id": "call_004", "name": "fetch_price", "args": {"ticker": "MSFT"}},
        ]
    )

    results = await price_tool_node.ainvoke([message], runtime=Runtime())

    assert len(results) == 2
    tool_ids = {r.tool_call_id for r in results}
    assert tool_ids == {"call_003", "call_004"}
Enter fullscreen mode Exit fullscreen mode

Pattern 4: Mocking LLM Calls Without API Keys

Testing the full agent loop requires an LLM. For CI, you don't want API keys or network calls.

from unittest.mock import AsyncMock, patch
from langchain_core.messages import AIMessage
import pytest


@pytest.fixture
def mock_llm():
    """Mock LLM that returns a deterministic tool call response."""
    mock = AsyncMock()
    mock.ainvoke.return_value = AIMessage(
        content="",
        tool_calls=[{
            "id": "call_mock_01",
            "name": "get_weather",
            "args": {"city": "Tokyo"},
        }]
    )
    return mock


async def test_agent_with_mock_llm(mock_llm, weather_tool_node):
    """Test agent logic without an API key."""
    # Simulate what the graph does: LLM → tool call → ToolNode
    llm_response = await mock_llm.ainvoke([HumanMessage(content="Weather in Tokyo?")])

    tool_results = await weather_tool_node.ainvoke(
        [llm_response],
        runtime=Runtime()
    )

    assert len(tool_results) == 1
    assert "Tokyo" in tool_results[0].content
    mock_llm.ainvoke.assert_called_once()
Enter fullscreen mode Exit fullscreen mode

For more complex scenarios — multi-turn conversations, stateful mocks, response sequences — use langchain_core.utils.testing.GenericFakeChatModel:

from langchain_core.utils.testing import GenericFakeChatModel

# Pre-program a sequence of responses
responses = [
    AIMessage(
        content="",
        tool_calls=[{"id": "c1", "name": "get_weather", "args": {"city": "Oslo"}}]
    ),
    AIMessage(content="The weather in Oslo is cold and rainy."),
]

fake_llm = GenericFakeChatModel(messages=iter(responses))
Enter fullscreen mode Exit fullscreen mode

Each call to fake_llm.invoke() or fake_llm.ainvoke() pops the next response from the sequence. This lets you test multi-step agent flows deterministically.

Pattern 5: Testing Error Handling

With handle_tool_errors=True, errors get converted to ToolMessage objects instead of propagating. Test that this actually works.

from langchain_core.tools import tool
from langchain_core.messages import ToolMessage
from langgraph.prebuilt import ToolNode
from langgraph.runtime import Runtime


@tool
def fragile_tool(value: int) -> str:
    """A tool that fails on negative input."""
    if value < 0:
        raise ValueError(f"Value must be non-negative, got {value}")
    return f"processed: {value}"


async def test_tool_error_handling():
    """Error handling must be explicit — never rely on the default (#6486)."""
    node = ToolNode(
        tools=[fragile_tool],
        handle_tool_errors=True,  # required — default is False since 1.0.x
    )

    message = AIMessage(
        content="",
        tool_calls=[{
            "id": "call_err",
            "name": "fragile_tool",
            "args": {"value": -5},
        }]
    )

    results = await node.ainvoke([message], runtime=Runtime())

    # With handle_tool_errors=True, errors become ToolMessages
    assert len(results) == 1
    assert isinstance(results[0], ToolMessage)
    assert "ValueError" in results[0].content or "non-negative" in results[0].content


async def test_tool_error_propagates_without_handler():
    """Without handle_tool_errors, errors propagate — tests should verify both modes."""
    node = ToolNode(
        tools=[fragile_tool],
        handle_tool_errors=False,  # explicit
    )

    message = AIMessage(
        content="",
        tool_calls=[{
            "id": "call_err2",
            "name": "fragile_tool",
            "args": {"value": -1},
        }]
    )

    with pytest.raises(ValueError, match="non-negative"):
        await node.ainvoke([message], runtime=Runtime())
Enter fullscreen mode Exit fullscreen mode

Pattern 6: Full Graph Integration Test

Unit tests catch the surface-level failures. But sometimes you need to verify that the graph state machine works correctly — tool calls are routed, messages accumulate, the cycle terminates.

from langgraph.graph import StateGraph, MessagesState, END
from langgraph.prebuilt import ToolNode, tools_condition
from langchain_core.utils.testing import GenericFakeChatModel
from langchain_core.messages import AIMessage, HumanMessage


def build_test_graph(llm, tools):
    """Build a minimal ReAct graph for integration testing."""
    tool_node = ToolNode(tools=tools, handle_tool_errors=True)

    def call_model(state: MessagesState):
        response = llm.invoke(state["messages"])
        return {"messages": [response]}

    graph = StateGraph(MessagesState)
    graph.add_node("agent", call_model)
    graph.add_node("tools", tool_node)
    graph.add_edge("__start__", "agent")
    graph.add_conditional_edges("agent", tools_condition)
    graph.add_edge("tools", "agent")

    return graph.compile()


async def test_full_agent_loop():
    """Integration test: agent calls tool, gets result, produces final answer."""
    @tool
    def add(a: int, b: int) -> int:
        """Add two numbers."""
        return a + b

    # Response sequence: first a tool call, then a final answer
    responses = [
        AIMessage(
            content="",
            tool_calls=[{"id": "c1", "name": "add", "args": {"a": 5, "b": 3}}]
        ),
        AIMessage(content="5 + 3 = 8"),
    ]

    fake_llm = GenericFakeChatModel(messages=iter(responses))
    graph = build_test_graph(fake_llm, [add])

    result = await graph.ainvoke({"messages": [HumanMessage(content="What is 5+3?")]})

    messages = result["messages"]
    # HumanMessage → AIMessage (tool call) → ToolMessage → AIMessage (final)
    assert len(messages) == 4
    assert messages[-1].content == "5 + 3 = 8"
Enter fullscreen mode Exit fullscreen mode

conftest.py Setup

Put the shared fixtures and configuration in conftest.py at the project root:

# conftest.py
import pytest
from langchain_core.tools import tool
from langgraph.prebuilt import ToolNode


@pytest.fixture(scope="session")
def basic_tools():
    """Reusable tools for testing — session scope avoids repeated construction."""
    @tool
    def echo(text: str) -> str:
        """Echo input back."""
        return text

    @tool
    async def async_echo(text: str) -> str:
        """Async echo."""
        return text

    return [echo, async_echo]


@pytest.fixture
def tool_node(basic_tools):
    """Standard ToolNode for tests — always with explicit error handling."""
    return ToolNode(
        tools=basic_tools,
        handle_tool_errors=True,
    )
Enter fullscreen mode Exit fullscreen mode

Summary: The Rules

  1. Always pass runtime=Runtime() when invoking ToolNode outside a compiled graph.
  2. Always pass handle_tool_errors=True explicitly — never rely on the default.
  3. Accept `kwargs** in custom async tool functions — absorbs runtime` and future injected params.
  4. Require langgraph-prebuilt >=1.0.2 — that's the version the runtime=/handle_tool_errors= requirements started at. No upper bound needed: the same patterns hold through at least 1.2.x, since the underlying issues (#6397, #6486) are still open upstream.
  5. Use GenericFakeChatModel for multi-step agent testing without API keys.
  6. Test error handling explicitly — both handle_tool_errors=True and False modes.

The Bigger Pattern

The langgraph-prebuilt package split was a versioning failure during a major restructure — a runtime parameter added to internals without backward compatibility, without clear migration docs, without optional defaulting for external callers.

This pattern repeats. Every fast-moving AI framework has a phase like this. The defense isn't to avoid LangGraph — it's to test at the right level of abstraction, pin your dependencies, and write tests that fail loudly before the bug reaches production.

If you're building production LangGraph agents, the patterns above — Runtime(), **kwargs, explicit error handling flags, GenericFakeChatModel — aren't workarounds. They're the stable interface until the upstream fixes land.

Further Reading

The async patterns in this article are part of the Python Testing Toolkit — async_test_patterns.py has production-ready fixtures for testing async tools, FastAPI endpoints, and LangGraph agents without API keys or live network calls.

Top comments (0)