From Protocol to Working System: Building and Testing an MTP Evaluator is

Part 6 of the Innovation Wormhole Research Series

Cross-domain innovation often begins with an analogy:

“Aviation learns from safety incidents. Could cybersecurity learn the same way?”

But an analogy is not a transfer.

A mechanism that works in one domain may fail in another because its enabling conditions—trust, incentives, regulation, data quality, or verification—do not hold.

This article turns the Mechanism Transfer Protocol (MTP) into a small executable evaluator.

Its goal is not to automate innovation.

Its goal is to make transfer recommendations transparent, inspectable, and testable.

The Problem

Most AI systems answer this question:

“Where has something similar been done before?”

But cross-domain innovation needs a harder question:

“What mechanism made it work, under which conditions, and do those conditions still hold in the target domain?”

A document-retrieval system can find articles about aviation safety and cybersecurity incident response.

It cannot decide whether the underlying learning mechanism can actually be transferred.

What Is an MTP Evaluator?

An MTP evaluator takes a structured Mechanism Transfer Card and decides whether a proposed transfer is ready for a controlled pilot.

It performs three steps:

1. Hard admissibility gates — reject incomplete or unsupported proposals before scoring.
2. Transparent scoring — evaluate structural, functional, causal, evidence, and contextual fit.
3. Pilot-readiness assessment — decide whether a controlled pilot is justified.

The key rule is non-compensatory:

A high similarity score cannot compensate for an unsupported critical assumption.

Example: Aviation Safety to Cybersecurity

Aviation safety programs such as the FAA’s Aviation Safety Action Program encourage structured reporting of safety concerns and use that information to support corrective action.

Cybersecurity incident-response guidance, including NIST SP 800-61 Rev. 3, similarly emphasizes learning from incidents and continuous improvement.

A plausible transfer hypothesis is:

Operational event
      ↓
Structured reporting
      ↓
Systematic analysis
      ↓
Corrective action
      ↓
Verification
      ↓
Organizational learning

The mechanism may transfer.

The institutional design may not.

For example, aviation-style reporting protections cannot simply be copied into cybersecurity.

Confidentiality, legal exposure, employee incentives, data sensitivity, and disclosure consequences all need separate examination.

Minimal Python Implementation

This evaluator uses only the Python standard library.

from dataclasses import dataclass
from typing import Dict, List


@dataclass
class Assumption:
    name: str
    critical: bool
    supported: bool


@dataclass
class MechanismTransferCard:
    source_domain: str
    source_problem: str
    target_domain: str
    target_problem: str
    mechanism: str
    causal_chain: List[str]
    evidence: List[str]
    assumptions: List[Assumption]
    scores: Dict[str, float]
    reviewer_approved: bool


WEIGHTS = {
    "structural": 0.20,
    "functional": 0.20,
    "causal": 0.25,
    "evidence": 0.20,
    "contextual": 0.15,
}

PILOT_THRESHOLD = 70.0
MIN_EVIDENCE_ITEMS = 1


def validate_card(card: MechanismTransferCard) -> List[str]:
    errors = []

    required_fields = {
        "source_domain": card.source_domain,
        "source_problem": card.source_problem,
        "target_domain": card.target_domain,
        "target_problem": card.target_problem,
        "mechanism": card.mechanism,
    }

    for field, value in required_fields.items():
        if not value or not value.strip():
            errors.append(f"Missing required field: {field}")

    if not card.causal_chain:
        errors.append("Causal chain is missing.")

    if not card.evidence:
        errors.append("Supporting evidence is missing.")

    if not card.assumptions:
        errors.append("Enabling assumptions are missing.")

    if set(card.scores) != set(WEIGHTS):
        errors.append("Score dimensions do not match evaluator schema.")
    else:
        for name, value in card.scores.items():
            if not isinstance(value, (int, float)) or not 0 <= value <= 1:
                errors.append(
                    f"{name} score must be numeric between 0 and 1."
                )

    if abs(sum(WEIGHTS.values()) - 1.0) > 1e-9:
        errors.append("Weights must sum to 1.")

    return errors


def check_hard_gates(card: MechanismTransferCard) -> List[str]:
    blockers = validate_card(card)

    for assumption in card.assumptions:
        if assumption.critical and not assumption.supported:
            blockers.append(
                f"Unsupported critical assumption: {assumption.name}"
            )

    if len(card.evidence) < MIN_EVIDENCE_ITEMS:
        blockers.append("Insufficient evidence records.")

    if not card.reviewer_approved:
        blockers.append("Reviewer approval is absent.")

    return blockers


def calculate_tfs(card: MechanismTransferCard) -> float:
    weighted_score = sum(
        WEIGHTS[name] * card.scores[name]
        for name in WEIGHTS
    )

    return round(weighted_score * 100, 2)


def evaluate_mtp(card: MechanismTransferCard) -> Dict:
    blockers = check_hard_gates(card)

    if blockers:
        return {
            "status": "BLOCKED",
            "blockers": blockers,
            "tfs": None,
            "pilot_ready": False,
        }

    tfs = calculate_tfs(card)
    pilot_ready = tfs >= PILOT_THRESHOLD

    return {
        "status": "PILOT_CANDIDATE" if pilot_ready else "REVIEW_REQUIRED",
        "blockers": [],
        "tfs": tfs,
        "pilot_ready": pilot_ready,
    }

Scoring the Aviation-to-Cybersecurity Transfer

The resulting Transfer Feasibility Score is:

TFS = 80.25

With a prototype threshold of 70, the proposal becomes a PILOT_CANDIDATE—provided every mandatory gate passes.

But this score is not a probability.

It does not mean the transfer has an 80.25% chance of success.

It is an index of assessed transfer conditions, based on supplied expert judgments.

The Most Important Test

Change one assumption:

card.assumptions.supported = False[1]

result = evaluate_mtp(card)

assert result["status"] == "BLOCKED"
assert result["tfs"] is None

Even if the transfer scores 80.25, it is blocked.

That is intentional.

If employees cannot safely report incidents, the proposed learning loop may fail regardless of how similar aviation and cybersecurity appear on paper.

Critical uncertainty should not be averaged away.

What This Does Not Prove

This prototype does not:

• Automatically verify sources.
• Prove that a causal mechanism is complete.
• Predict whether a transfer will succeed.
• Replace expert review or empirical testing.
• Authorize production deployment.

It separates three different questions:

A system can pass the first test and still fail the other two.

Next Step: Active Falsification

The next version should not only ask:

“Is this assumption supported?”

It should ask:

“What observation would make us abandon this assumption?”

For the aviation-to-cybersecurity case, useful tests could include:

• A confidential reporting simulation.
• A tabletop incident-response exercise.
• A retrospective analysis of reporting completeness.
• A comparison of corrective-action closure rates.
• An independent check of whether a control reduced the targeted risk.

The goal is not to collect more data.

The goal is to identify the smallest experiment that can change the decision.

Conclusion

The distance between a promising analogy and a defensible innovation is the distance between resemblance and mechanism.

An MTP evaluator does not solve cross-domain innovation.

It makes the claim testable.

It helps a system say:

• This mechanism appears relevant.
• These conditions made it work.
• These conditions are still unverified.
• This is the evidence behind the recommendation.
• This is why the proposal is blocked or deferred.
• This is the smallest informative experiment to run next.

That is a more useful foundation for AI-assisted innovation than generating another plausible idea.
created by Seyed Alireza Alhosseini Almodarresieh
https://almodarresieh.github.io/