DEV Community

Avery Lin
Avery Lin

Posted on

Label Docs Units Restate or Promise Before Any Model Draft

Generated API documentation fails when a single paragraph mixes restated schema facts with unapproved product promises. A classifier that labels each unit as RESTATE, SYNTHESIZE, PROMISE, or EVALUATE removes that mixture before any draft starts. Only RESTATE and SYNTHESIZE units may receive model prose, while PROMISE and EVALUATE units become stubs with named human owners. The remainder of this article specifies a unit schema, a refusal gate, a test plan, and explicit limits.

Mixed-authority pages leak obligations

Reviewers treat a generated page as one artifact, so a stray compatibility sentence inherits trust from nearby field tables. That trust is unearned when the sentence has no schema node, no contract test, and no owner. Editing after generation is already late, because the claim sits in the same file as restated facts. Classification before drafting keeps promissory language out of the model context window entirely.

Release notes show the same failure in a different costume, because a git diff is restatable while breaking-change language is a promise. Agents that assume versions, default timeouts, or migration cost are not failing at grammar; they are failing at authority. The workflow below does not score fluency or completeness of restated prose. It only answers whether a documentation unit is allowed to exist as model output.

Four authority classes

Treat each heading, table, example, or admonition as one documentation unit with exactly one class. The listings in this article are a worked example and not a production benchmark.

Class Model may draft? Required evidence Typical failure if skipped
RESTATE Yes One cited source path plus a locator Invented field names
SYNTHESIZE Yes Two or more cited sources Hidden policy in connective phrasing
PROMISE No Named human owner Accidental SLA or support language
EVALUATE No Named human owner Uncited fitness or ranking claims

RESTATE units restate a file, schema node, fixture, or test identifier without adding an obligation. SYNTHESIZE units join those citations into a reading order, still without recommending a product policy. PROMISE units include support windows, deprecation dates, compatibility claims, throughput, and any future tense about behavior. EVALUATE units include comparisons, fitness claims, and adjectives that rank the interface for readers.

If a unit cannot present the evidence required for RESTATE or SYNTHESIZE, a fluent model does not promote it. The unit is reclassified as PROMISE or EVALUATE and emitted as a stub that continuous integration can grep.

Workflow

1. Inventory units from a committed outline

Start from an outline in version control, not from a chat transcript that invented extra headings. Each unit needs a stable identifier, a target path, and a class hypothesis before any generation call runs.

# docs/units.yaml — worked example, not live product data
version: 1
units:
  - id: payments.capture.fields
    target: docs/api/payments.md#capture-fields
    class: RESTATE
    sources:
      - path: openapi/payments.yaml
        locator: "#/paths/~1payments~1capture/post/requestBody"
  - id: payments.capture.happy-path
    target: docs/api/payments.md#capture-example
    class: SYNTHESIZE
    sources:
      - path: openapi/payments.yaml
        locator: "#/paths/~1payments~1capture/post"
      - path: tests/contract/test_capture.py
        locator: "test_capture_201"
  - id: payments.capture.support-window
    target: docs/api/payments.md#support
    class: PROMISE
    owner: api-steward
    due: "2026-09-16"
  - id: payments.capture.when-to-use
    target: docs/api/payments.md#guidance
    class: EVALUATE
    owner: api-steward
    due: "2026-09-16"
  - id: payments.capture.changelog
    target: docs/api/payments.md#changelog
    class: RESTATE
    sources:
      - path: CHANGELOG.md
        locator: "## Unreleased"
Enter fullscreen mode Exit fullscreen mode

The inventory is the source of truth for the generator, and models do not invent units during a run. A changelog unit may restate added paths from CHANGELOG.md, but it may not declare that the change is backward compatible.

2. Encode the refusal gate as a pure function

The gate receives a unit record and returns whether a model may draft a body. Keep the function free of network calls so the test suite stays deterministic across laptops and CI images.

# docs_gate/authority.py
from __future__ import annotations

from dataclasses import dataclass
from typing import Literal

Authority = Literal["RESTATE", "SYNTHESIZE", "PROMISE", "EVALUATE"]
PROMISSORY_MARKERS = (
    "we will",
    "supported until",
    "backward compatible",
    "breaking change",
    "sla",
    "guaranteed",
)
EVALUATIVE_MARKERS = (
    "best",
    "simplest",
    "production-ready",
    "recommended for all",
)

@dataclass(frozen=True)
class SourceRef:
    path: str
    locator: str

@dataclass(frozen=True)
class DocsUnit:
    id: str
    target: str
    declared_class: Authority
    sources: tuple[SourceRef, ...] = ()
    owner: str | None = None
    proposed_text: str = ""

def classify(unit: DocsUnit) -> Authority:
    text = unit.proposed_text.lower()
    if any(marker in text for marker in PROMISSORY_MARKERS):
        return "PROMISE"
    if any(marker in text for marker in EVALUATIVE_MARKERS):
        return "EVALUATE"
    if len(unit.sources) >= 2:
        return "SYNTHESIZE"
    if len(unit.sources) == 1:
        return "RESTATE"
    return "EVALUATE"

def allow_model_draft(unit: DocsUnit) -> bool:
    effective = classify(unit)
    if effective != unit.declared_class:
        return False
    if effective in ("PROMISE", "EVALUATE"):
        return False
    if effective == "RESTATE" and len(unit.sources) != 1:
        return False
    if effective == "SYNTHESIZE" and len(unit.sources) < 2:
        return False
    return True
Enter fullscreen mode Exit fullscreen mode

Marker lists are conservative on purpose, because a missed adjective is cheaper than a false RESTATE label. Declared class and effective class must match, so an outline cannot launder a promise by calling it a field table.

3. Extract source slices, then draft only allowed bodies

Send the cited slice, not the entire specification, so the model cannot harvest unrelated promises from later paths. The extractor below is a local command and does not require a model.

python - <<'PY'
from pathlib import Path

path = Path("openapi/payments.yaml")
locator = "#/paths/~1payments~1capture/post/requestBody"
print(f"source={path} bytes={path.stat().st_size} locator={locator}")
# Load only the cited node in the real pipeline; do not ship the whole file.
PY
python -m pytest tests/test_authority.py -q
python -m docs_gate.inventory_check docs/units.yaml
Enter fullscreen mode Exit fullscreen mode

The generator writes Markdown only when allow_model_draft is true. Forbidden units receive a stub that review tools can fail on.

# docs_gate/generate.py
from textwrap import dedent

from docs_gate.authority import DocsUnit, allow_model_draft

STUB = """\
> Human-owned unit `{id}` ({cls}).
> Owner: {owner}. Do not generate prose for this block.
"""

def render_unit(unit: DocsUnit, drafted: str | None) -> str:
    if not allow_model_draft(unit):
        return STUB.format(
            id=unit.id,
            cls=unit.declared_class,
            owner=unit.owner or "UNASSIGNED",
        )
    if not drafted:
        raise ValueError(f"allowed unit {unit.id} produced empty draft")
    sources = ", ".join(f"{item.path} {item.locator}" for item in unit.sources)
    return dedent(f"""\
    <!-- unit:{unit.id} class:{unit.declared_class} -->
    {drafted.strip()}

    _Sources: {sources}_
    """)
Enter fullscreen mode Exit fullscreen mode

When a draft is required for an allowed unit, pass those source slices through a model endpoint the operator already controls. Teams without a standing inference host can run that optional step with MonkeyCode's free model access and free server option; the authority gate stays local and unchanged. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Keep credentials in the environment and never in docs/units.yaml.

4. Prove the refusal with tests on every docs change

# tests/test_authority.py
from docs_gate.authority import DocsUnit, SourceRef, allow_model_draft, classify

SRC = SourceRef("openapi/payments.yaml", "#/paths/~1capture/post")

def test_promise_text_cannot_be_declared_restate():
    unit = DocsUnit(
        id="u1",
        target="docs/a.md#s",
        declared_class="RESTATE",
        sources=(SRC,),
        proposed_text="This endpoint is backward compatible through 2027.",
    )
    assert classify(unit) == "PROMISE"
    assert allow_model_draft(unit) is False

def test_changelog_cannot_declare_a_breaking_change():
    unit = DocsUnit(
        id="u1b",
        target="docs/a.md#changelog",
        declared_class="RESTATE",
        sources=(SourceRef("CHANGELOG.md", "## Unreleased"),),
        proposed_text="This is a breaking change for capture clients.",
    )
    assert classify(unit) == "PROMISE"
    assert allow_model_draft(unit) is False

def test_synthesize_requires_two_sources():
    unit = DocsUnit(
        id="u2",
        target="docs/a.md#ex",
        declared_class="SYNTHESIZE",
        sources=(SRC,),
        proposed_text="Capture returns 201 when the fixture matches.",
    )
    assert allow_model_draft(unit) is False

def test_restatement_with_one_source_is_allowed():
    unit = DocsUnit(
        id="u3",
        target="docs/a.md#fields",
        declared_class="RESTATE",
        sources=(SRC,),
        proposed_text="amount is an integer in minor units.",
    )
    assert allow_model_draft(unit) is True
Enter fullscreen mode Exit fullscreen mode

Run the tests in CI on outline-only pull requests as well as generated pages. A green gate means no PROMISE or EVALUATE unit received a model body; it does not mean the restated prose is complete.

# .github/workflows/docs-authority.yml — proposed wiring
name: docs-authority
on:
  pull_request:
    paths: ["docs/**", "docs_gate/**", "openapi/**", "tests/**"]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: python -m pytest tests/test_authority.py -q
      - run: python -m docs_gate.inventory_check docs/units.yaml
Enter fullscreen mode Exit fullscreen mode

Decision table for reviewers

Observation Action Merge?
Declared RESTATE, classifier returns PROMISE Reclassify, assign owner, emit stub No
Declared SYNTHESIZE, only one source Add a second citation or drop the unit No
Stub owner is UNASSIGNED Block until a human name exists No
Restated body adds a default timeout Treat as PROMISE and delete the sentence No
Restated body quotes the schema locator Allow the draft; still review wording Yes, after review

The last row is intentional: allowed drafts are not auto-merged into the default branch. Humans still review wording, but they no longer hunt for smuggled support statements inside field descriptions.

Limitations

Marker-based classification is incomplete and will miss carefully worded obligations. A promise can read as "callers may assume the same payload next quarter" without matching the list above. SYNTHESIZE units can smuggle policy through connective tissue such as "so retries are expected." Owners can rubber-stamp EVALUATE stubs under release pressure, which reintroduces mixed authority by process rather than by model.

The workflow assumes a committed schema, changelog, or contract test exists as evidence. Teams whose only source is a slide deck cannot produce RESTATE units without inventing locators. Marketing pages, pricing pages, and legal terms are PROMISE or EVALUATE by nature and should not enter this generator at all.

Do not use this approach when the deliverable is a persuasive narrative, an uncited benchmark claim, or an incident report that requires judgment. Do not treat any model endpoint as an owner of record for a stub. Free model access and a free server option are an operator-supplied convenience for running optional drafts; they are not a quota, hardware profile, or durability guarantee.

What this does not replace

Contract tests still decide whether a restatement is true against the live interface. Release managers still decide deprecation dates and support windows in human-owned units. Security reviewers still decide whether examples leak production shapes or credentials. The classifier only removes the class of error where a model writes a promise because the surrounding page looked like a reference.

If you adapt the gate, keep the four classes intact and commit docs/units.yaml beside the pages it controls.

Top comments (0)