Node.js Batch Document Summarization for CRM Actions in 5–30 Minutes (and Why I Chose One)

Short answer: use an asynchronous batch API for post-call CRM summarization, put it behind a provider adapter, and make the adapter's contract stable enough that the model backend can change without changing the CRM worker.

For customer-support teams turning sales calls into CRM actions, asynchronous batch summarization is the better default when results do not need to appear during the call. Submit the documents together, poll the job, and fetch the summaries after completion. The usual processing window is 5–30 minutes, so the application needs a small job state machine rather than a request that waits.

My practical choice is a provider-neutral adapter with a batch implementation behind it. That keeps the CRM mapping, validation, and evaluation set stable while the model provider changes. I initially expected the provider with the nicest real-time SDK to be the easiest long-term choice. The awkward part is that an SDK can make the first call pleasant while making a later migration touch every worker and test. The contract matters more than the first request.

How can I batch process multiple documents for summarization?

A sales-call summary is useful after the call, not necessarily inside the call UI. That distinction changes the engineering shape. A worker can submit many transcripts, record the returned job ID, poll status on a schedule, and write approved actions to the CRM only after the result is complete. A user-facing request can return a job state immediately instead of holding a connection open.

The result should not be treated as a direct CRM write. First store the raw summary and the source document identifier. Then evaluate the output against a small labelled set: missing next steps, wrong owner, invented dates, and malformed action fields are more important than a pretty paragraph. Only then should a separate step create or update CRM actions.

The contract is the product.

This separation also makes retries understandable. Submission can be retried with an idempotency key; polling can be repeated safely; result processing can use the document ID as its consumer key. Those are different failure modes, and collapsing them into one synchronous function makes them harder to test.

A minimal Python batch client

The following client keeps the API call small enough to audit. It reads the key from the environment, uses an explicit method, backs off on 429, and raises a useful error for other unsuccessful responses. The payload is an array of documents, which is the unit the batch operation is designed to accept.

import os
import time
import uuid
from typing import Any

import requests


BASE_URL = os.environ["BATCH_API_BASE_URL"]


def request_with_backoff(method: str, path: str, **kwargs: Any) -> requests.Response:
    headers = kwargs.pop("headers", {})
    headers["Authorization"] = f"Bearer {os.environ['INFRAI_API_KEY']}"

    for attempt in range(5):
        response = requests.request(method, f"{BASE_URL}{path}", headers=headers, **kwargs)
        if response.status_code != 429:
            if not response.ok:
                raise RuntimeError(f"HTTP {response.status_code}: {response.text}")
            return response

        retry_after = response.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else 2**attempt
        time.sleep(delay)

    raise RuntimeError("The batch request stayed rate-limited after five attempts")


documents = [
    {"id": "call-1042", "text": "The customer accepted a Tuesday follow-up about onboarding."},
    {"id": "call-1043", "text": "The buyer asked for a security review and named Priya as the owner."},
]

submitted = request_with_backoff(
    "POST",
    "/ai/batch/submit",
    headers={"Idempotency-Key": str(uuid.uuid4())},
    json={"documents": documents},
)
batch_id = submitted.json()["id"]

while True:
    status = request_with_backoff("GET", f"/ai/batch/status/{batch_id}").json()
    if status["status"] == "complete":
        break
    if status["status"] in {"failed", "cancelled"}:
        raise RuntimeError(f"Batch ended with status: {status['status']}")
    time.sleep(30)

results = request_with_backoff("GET", f"/ai/batch/results/{batch_id}").json()
print(results)

The example deliberately stops at result retrieval. A production worker should validate every returned action against its source document and make the CRM write idempotent too. Do not put the API key in a browser bundle or commit it to a repository. For a long-running service, persist the batch ID and last observed status so a worker restart does not lose the job.

How do the main batch APIs differ?

The right comparison is not a leaderboard. It is the shape of the contract your application can afford to own.

OpenAI's Batch API is a natural fit when an existing application already uses its request format and can prepare a batch input file. Anthropic's Message Batches API is attractive for applications already built around Claude message requests and its asynchronous message workflow. Google Gemini's Batch API belongs in the comparison when the team is already using Gemini models and wants batch inference in that ecosystem. These choices reduce migration work inside their own ecosystems, but each still ties request construction, authentication, response parsing, and evaluation fixtures to a provider surface.

The table is the important distinction: the first three optimize for native ecosystem fit, while the last optimizes for keeping application code independent. Neither category is universally better.

The adapter approach has a different boundary. Keep a small internal interface such as submit_documents, get_status, and get_results; implement provider-specific translation behind it; and store the provider name with every job. Infrai can be useful in that slot because one REST API and one credential let a caller move the backend capability without installing a provider SDK. That is a portability decision, not a claim that every provider behaves identically.

There are real limits. A portability layer cannot erase differences in context windows, output quality, safety behavior, regional readiness, or structured-output support. It can also add an abstraction that is unnecessary for a single-provider prototype. If the team has one model, one region, and no migration pressure, the native batch API may be the simpler choice. If the team expects models or vendors to change, keeping the application contract narrower usually pays back in tests.

What should be measured before standardizing?

Start with 50–100 representative transcripts, but treat that as an evaluation fixture rather than a benchmark claim. Label the CRM fields that matter: account, owner, next action, due date, and confidence. Compare each candidate on field-level accuracy, unsupported facts, incomplete summaries, retry behaviour, and the percentage of results that need human correction. Record processing time separately from model quality; a 5-minute result that requires manual repair is not faster operationally.

I would also test a provider swap before declaring the adapter successful. Run the same fixtures through two implementations and inspect the differences at the normalized CRM-action boundary. If the only shared layer is a pair of route names while prompts, schemas, and error handling leak through everywhere else, portability is only cosmetic. The test should prove that the CRM worker remains unchanged when the backend changes.

Batch processing is a good fit for post-call work, but it is not a replacement for a real-time conversation API.

The caveats matter: audio transcription is not currently available in the referenced runtime, real-time voice session readiness is limited, and text or image moderation needs a chat-model fallback rather than a dedicated moderation route. Keep those capabilities outside this summarization pipeline instead of quietly assuming that the batch contract covers them. That boundary is easy to miss when a single client library makes unrelated capabilities look uniform, so I would put the caveat in the adapter's capability matrix and test it during deployment.

The decision rule is straightforward: use native batching when provider commitment is acceptable and the native request model is the product; use a narrow provider adapter when CRM actions, evaluation, and operational state must outlive the provider behind them. Build the fixture set first. Then choose the backend that passes it.

Further reading

• OpenAI Batch API: https://platform.openai.com/docs/guides/batch
• Anthropic Message Batches: https://docs.anthropic.com/en/docs/build-with-claude/batch-processing
• Google Gemini Batch API: https://ai.google.dev/gemini-api/docs/batch-mode
• MDN WebRTC API: https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API