DEV Community

payintlab
payintlab

Posted on

PSP Change Drift Watch #2: turning the changelog watch into a runnable monitor

#api #automation #monitoring

What this series does: watch Stripe, PayPal and Adyen for API changes, SDK majors
and ecosystem issues, and write up only the ones that can silently break a live
integration. Issue 1 covered five drift findings by hand. Issue 2 turns the
manual watch into a small, runnable tool.

The problem with doing it by hand

Issue 1 listed five changes worth acting on (PayPal IPN migration, Stripe
billed_until, stripe-node 21, Adyen v72, and a few low-priority notes).
Producing that list meant reading changelogs and release notes across three
platforms, then judging which changes could "return 200 while the business logic
never runs".

Doing that by hand has two failure modes:

  1. You only notice drift after it already broke something. That is the whole point of these bugs: they are silent.
  2. You forget to re-check. A changelog read in September is stale by October.

The six pain classes I keep seeing (duplicate events, field-format drift, lost
callbacks, signature failures, new event types, sandbox/production mismatch)
share one root cause: the platform changed and your system did not follow.
So the useful primitive is "detect the change early", not "read the changelog
occasionally".

What I built

A small monitor, psp-drift-watch, that turns the watch into a repeatable job:

  • sources.json — the watchlist: changelogs, migration guides and SDK release notes, each tagged with a platform and a signal type.
  • fetch.py — fetches every source and fingerprints the content.
  • diff.py — compares against the last snapshot and writes a drift report.
  • a regression hook — optionally runs the payment-qa-framework test suite when something changes.

The fingerprint is a content hash, plus ETag / Last-Modified when the server
provides them, so any edit to a watched page shows up as a drift entry on the
next run.

Honest about what it does and does not do

This is a monitor, not a detector. It tells you "this page changed", not
"this change breaks you". The judgment still lives in the three signals from
Issue 1:

  1. EOL or migration — there is a deadline, but the silent failures live in the migration window.
  2. Field default changes — no error, the value just disappears.
  3. SDK major with a pinned API version — the upgrade itself hides the risk.

A changed source is a prompt to apply that judgment (plus the regression
checklist), not a verdict.

Run it 

git clone https://github.com/PayIntLab/psp-drift-watch.git
cd psp-drift-watch

python3 fetch.py                 # first snapshot
cp snapshot.json baseline.json   # freeze the baseline

./drift.sh                       # fetch + diff + report
./drift.sh --regress             # also trigger the regression suite
Enter fullscreen mode Exit fullscreen mode

The report (drift-report.md) lists changed / new / failed / unchanged sources,
plus the pre-upgrade regression checklist.

Regression checklist (unchanged from Issue 1)

  • Pin the API version; upgrading is an explicit step, not a side effect of a dependency update.
  • Run success, decline, expiry, refund, dispute and duplicate-callback cases.
  • Assert in tests whether a missing field raises or silently continues.
  • Cover subscription changes separately: create, renew, cancel, plan change.
  • Trigger the same event once in sandbox and once in production, and confirm both behave the same.

Over the years I have done both development and testing for cross-border
payment systems: payment integrations and SDK upgrades, test automation
frameworks, payment flow testing, dropped-order triage, and reconciliation
fallbacks. I now focus on payment verification for products going global, and I
also take on integration and automation work. Happy to compare notes.

Repo

https://github.com/PayIntLab/psp-drift-watch

Top comments (0)