Marketplace Signup Friction (CAPTCHA First, Risk Scoring After Behavioral Signals)

Short answer: put a challenge and coarse rate controls before durable account creation, then apply risk scoring only after useful behavioral signals exist; keep new marketplace accounts capability-limited until that second stage has enough evidence.

CAPTCHA and risk scoring solve different timing problems. The challenge makes automated creation more expensive at the one moment when the service has almost no history. The later score can use events that do not exist yet: sign-in outcomes, recovery attempts, device changes, and marketplace actions. Treating either control as the whole defense creates a bad choice between an easy bot funnel and a signup form that punishes legitimate people.

The forgot-password path belongs in the same design. An account that passed signup but has no trustworthy history should not gain a less-protected route back to full access through recovery. For an audit, I want one event trail that can explain the creation decision, later capability changes, and recovery decision without storing raw secrets or pretending a score is proof of identity.

How should marketplace signup place CAPTCHA before creation and score risk after signals arrive?

Use a two-stage state machine. Stage one decides whether the service may create a durable account at all. It sees only request-time evidence, so its controls should stay narrow: a completed challenge, request throttling, a generic response, and server-side validation. OWASP describes CAPTCHA as defense in depth rather than a complete prevention control, and recommends generic authentication responses plus protections against automated attacks. That is the right mental model here.

Stage two starts after creation, but creation should not mean unrestricted access. A fresh marketplace account can verify its contact channel, browse, and build history while higher-impact capabilities remain pending. As events arrive, a policy evaluates them and either preserves the limited state, asks for stronger verification, or enables a capability. The score informs a policy decision; it doesn't authenticate a person by itself.

Creation is not trust.

This distinction is small on a diagram and huge in production.

The data flow is plain: normalize and validate the signup input, check request limits, verify the challenge, emit a creation decision, and only then write the account. Subsequent security-relevant events feed an append-only audit stream. A policy reads a bounded set of those events and records both its input snapshot and outcome. Forgot-password requests use a generic outward response, while the internal policy decides whether recovery can proceed normally or needs additional verification. Every transition gets a correlation ID so an investigator can reconstruct the decision without exposing the response to an attacker.

Here is a runnable, deliberately small policy model. The weights and thresholds are illustrative configuration, not security constants. In a real service, the eval harness must set them from labeled abuse outcomes and false-positive constraints.

from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
import hashlib
import hmac
import json
from typing import Any
from uuid import uuid4


class Access(Enum):
    LIMITED = "limited"
    REVIEW = "review"
    STANDARD = "standard"


@dataclass
class Account:
    account_id: str
    email_ref: str
    access: Access = Access.LIMITED
    signals: list[dict[str, Any]] = field(default_factory=list)


class SignupPolicy:
    def __init__(self, audit_key: bytes):
        self.audit_key = audit_key
        self.accounts: dict[str, Account] = {}
        self.audit: list[dict[str, Any]] = []

    def _email_ref(self, email: str) -> str:
        normalized = email.strip().casefold().encode()
        return hmac.new(self.audit_key, normalized, hashlib.sha256).hexdigest()

    def _record(self, event: str, decision_id: str, **details: Any) -> None:
        self.audit.append({
            "time": datetime.now(timezone.utc).isoformat(),
            "event": event,
            "decision_id": decision_id,
            **details,
        })

    def create(self, email: str, challenge_passed: bool,
               request_allowed: bool) -> dict[str, str]:
        decision_id = str(uuid4())
        allowed = challenge_passed and request_allowed
        self._record(
            "signup_gate",
            decision_id,
            challenge_passed=challenge_passed,
            request_allowed=request_allowed,
            outcome="create" if allowed else "deny",
        )
        if not allowed:
            return {"status": "request_received", "decision_id": decision_id}

        account = Account(str(uuid4()), self._email_ref(email))
        self.accounts[account.account_id] = account
        self._record(
            "account_created",
            decision_id,
            account_id=account.account_id,
            access=account.access.value,
        )
        return {"status": "request_received", "decision_id": decision_id}

    def add_signal(self, account_id: str, kind: str, value: bool) -> None:
        account = self.accounts[account_id]
        account.signals.append({"kind": kind, "value": value})

    def evaluate(self, account_id: str) -> Access:
        account = self.accounts[account_id]
        values = {item["kind"]: item["value"] for item in account.signals}
        score = sum((
            45 if values.get("recovery_burst") else 0,
            35 if values.get("device_changed") else 0,
            30 if values.get("failed_signins") else 0,
            -25 if values.get("contact_verified") else 0,
        ))
        account.access = (
            Access.REVIEW if score >= 50
            else Access.STANDARD if values.get("contact_verified") else Access.LIMITED
        )
        self._record(
            "risk_evaluated",
            str(uuid4()),
            account_id=account_id,
            signal_names=sorted(values),
            score=score,
            outcome=account.access.value,
            policy_version="marketplace-signup-1",
        )
        return account.access


policy = SignupPolicy(b"replace-with-a-secret-from-your-runtime")
policy.create("[email protected]", challenge_passed=True, request_allowed=True)
account_id = next(iter(policy.accounts))
policy.add_signal(account_id, "contact_verified", True)
policy.add_signal(account_id, "device_changed", False)
print(json.dumps({"access": policy.evaluate(account_id).value}))

The example returns the same public signup status on allow and deny. Internally, the audit event keeps the actual outcome. That split matters because a detailed public error can become an account-discovery or policy-probing oracle, while an internal reviewer still needs to know why no record was created. Production code would also keep the challenge verifier, limiter, account store, and audit sink behind separate interfaces; combining them here makes the state transition visible rather than prescribing a deployment topology.

The failure mode is granting trust too early

The tempting implementation is challenge passed -> active account. It feels complete in a notebook because the happy-path test turns green, but it gives a one-time challenge more authority than it deserves. A human-assisted solver or a patient automated actor can pass that gate and immediately use the most valuable marketplace action. A delayed score then becomes incident commentary instead of enforcement.

Make the post-creation boundary concrete. For a buyer, limited access might exclude unusually sensitive account changes. For a seller, it might hold listing publication or payout-related changes until contact verification and enough consistent behavior exist. Those examples are policy choices, not universal rules; the important property is that account existence and capability authorization are separate transitions. Record which policy version authorized each transition.

Recovery is where teams often reconnect them accidentally. The external response to a forgot-password request should remain generic so it doesn't reveal whether an account exists, as OWASP recommends for authentication-related flows. Internally, evaluate the request against the account's current state and available signals. A recently changed device combined with repeated recovery attempts may justify stronger verification, while the absence of history should remain “unknown,” not “safe.” Don't silently turn missing data into a zero-risk score.

Consider the full recovery trace for a newly created seller. The signup gate records a passed challenge and creates the account as limited; contact verification later permits an ordinary capability, but no long behavioral history exists yet. A forgot-password request then arrives from a changed device after repeated recovery activity. The public handler still acknowledges the request with the same generic wording it would use for any address. Behind that response, the recovery service resolves the pseudonymous account reference, reads the exact policy version, marks the device and recovery signals as present, and asks the policy for a transition. The policy returns review, so no higher-impact capability is granted. The audit sink records the correlation ID, bounded input names, policy version, and review outcome. Support can now inspect one coherent chain without seeing a raw recovery token, and an attacker learns nothing from the acknowledgement. If the device signal had been unavailable rather than false, the trace would preserve that distinction instead of quietly lowering the score. That last branch is the kind of case an eval suite needs to pin down before deployment.

Keep it boring.

An audit record should explain the decision without becoming a shadow profile. Store a pseudonymous account reference, event time, policy version, names of signals consulted, decision, and correlation ID. Retention, access, and deletion rules need explicit ownership. Raw challenge tokens, recovery tokens, passwords, and unnecessary device attributes do not belong in that decision log. I'm not sure one universal retention period can fit every marketplace; contractual, regulatory, and incident-response requirements would have to resolve that locally.

One more subtle failure appears when an evaluator recomputes history with today's policy. That produces a plausible answer, but not necessarily the answer the service made at the time. Preserve the policy version and the bounded input snapshot used for the transition. Otherwise, a threshold change can rewrite the apparent reason for an old decision.

Evaluate friction and abuse as separate outcomes

An eval-driven rollout needs two scorecards. The abuse scorecard asks whether unwanted creation and harmful downstream actions were contained. The user scorecard asks where legitimate people abandoned, repeated a challenge, waited for verification, or entered recovery. Combining those into one “conversion versus fraud” number hides the exact control that needs adjustment.

Start with replayable policy tests built from synthetic cases, then use carefully governed labeled outcomes from the service. Include sparse-history accounts, accessibility paths, shared networks, changed devices, repeated recovery requests, and delayed verification. The suite should assert transitions and audit explanations, not only numeric scores. For example, a case can assert that a passed signup challenge creates a limited account, no behavioral history cannot produce standard access, and a recovery burst never changes the public response text.

The synthetic code AUTH-RISK-042 in an internal test report can identify “recovery burst plus changed device requires review.” It should never leak to the public response. A stable case ID is useful because a prompt, model, rule, or weight change can be compared against the same expected decision without claiming that a notebook metric represents production behavior.

Unknown is not safe.

For an AI-assisted risk component, pin its model or prompt version in the audit metadata, constrain its output to a small decision schema, and test cost alongside decision quality. Free-form explanations are poor authorization inputs. They are harder to compare, can expose sensitive attributes in logs, and consume tokens without necessarily improving the policy. A conventional rule should handle facts that are already deterministic; reserve a model for signals whose ambiguity actually warrants it. Your mileage may vary because label quality and attacker behavior differ, so shadow evaluation should precede any authority-changing rollout.

Measure each gate independently. A spike in challenge failure says something different from a spike in post-creation review. Likewise, a fall in successful recovery could indicate stronger abuse resistance or could mean legitimate users are stuck. The event model must let operators split those populations before adjusting a threshold.

Where should the architecture draw its trust boundaries?

The challenge verifier should issue a short-lived, single-purpose server-side result to the signup gate. The client must not be able to declare that a challenge passed. The account writer accepts only a gate decision, and the capability service accepts only an explicit policy transition. This keeps a UI shortcut from becoming an authorization shortcut.

Risk computation also needs a strict input contract. Use named, versioned signals with clear provenance; distinguish false from unknown; reject stale evidence according to the policy; and avoid feeding raw unbounded event text into an evaluator. The output should be a small enum such as limited, review, or standard, accompanied by a reason category suitable for restricted audit access. The service applying that output still owns the final authorization check.

The catch is that delayed scoring adds states, queues, policy versions, and support work. It is not suitable when every new account must receive the same low-impact capability and there are no meaningful later signals; in that narrow case, a pre-creation challenge plus rate controls may be enough. A high-risk marketplace action should take the opposite path: keep staged capabilities and add stronger verification rather than trusting a score. Stick with deterministic rules when the available evidence is small and well-defined. Add a model only when an eval shows that it improves a decision that matters enough to justify its latency, token cost, monitoring, and review burden.

This is also why the risk evaluator should fail closed for high-impact capability changes without blocking basic account recovery messaging. The public request can still receive its generic acknowledgement, while the internal transition remains pending until an authoritative decision exists. No hidden default should promote an unknown account.

Ship the controls as one auditable workflow

Before deployment, trace one legitimate signup and one denied signup from challenge verification through the account store, confirming that only the allowed path creates a durable record and both public responses remain generic. Then trace a limited account through contact verification, a policy evaluation, and a capability change; the audit view should show timestamps, correlation IDs, policy versions, bounded signal names, and outcomes. Repeat the exercise for forgot-password requests with an existing account and a nonexistent address, checking that the outward timing and wording are handled consistently while internal decisions remain distinguishable.

Run the synthetic eval suite on every rule, prompt, model, or threshold change. Shadow new scoring logic before it can change access. Alert on shifts at each stage rather than one aggregate funnel, budget evaluator latency and token use, restrict access to the audit stream, and test retention deletion. Finally, rehearse the support path for a legitimate user held in review. If an operator cannot explain and safely resolve that state, the workflow isn't ready for production.

The durable design decision is simple: spend a little friction before creation, accumulate evidence afterward, and never confuse an account record with earned trust. CAPTCHA narrows the automated funnel. Versioned policy transitions govern what the account can do. A restrained, correlated audit trail connects signup to recovery without advertising internal decisions to an attacker.

References

• https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html