I run a small trading-alerts project, and this week I added eight new signal engines to a bot that was already running. The code was the easy part. The hard part was making sure the statistics stayed honest.
The bug you only see once
A signal engine reads recent candles and says "there is a setup". When you deploy a new engine on a live bot, its very first cycle looks at the last few hours of history, finds setups that formed before the engine existed, and happily records them as trades.
Those trades never happened in real time. Nobody could have acted on them. But they now sit in your journal, they count in your win rate, and every comparison you make afterwards is polluted.
The fix: version the engines, skip the first cycle
The bot keeps a small persistent state file. I store an engines version in it:
ENGINES_VERSION = "v3"
def run_cycle(state, engines, bars, journal):
# True only on the first cycle after a new engine set is deployed
warm = state.get("_engines") != ENGINES_VERSION
for engine in engines:
try:
signal = engine.analyze(bars)
except Exception:
log.exception("engine %s failed", engine.id)
continue # one broken engine must not stop the others
if not signal:
continue
if warm:
continue # warm-up: compute, but record nothing
if age_seconds(signal) > engine.max_age:
continue # stale setup: too late to be actionable
journal.record(signal)
state["_engines"] = ENGINES_VERSION # persisted, so a restart does not re-trigger warm-up
Three details matter:
- The marker is persisted. If it only lived in memory, every container restart would silently skip a cycle.
- The marker is written after the loop. A crash in the middle means the next start is still "warm", which is the safe direction.
- Warm-up is not enough on its own. An age guard per engine (a few minutes for fast setups, a bit more for slower ones) also rejects setups that were detected late, for example after a feed hiccup.
After the deploy, I checked the journal: zero trades from the new engines, even though they were already running. That is the whole test, and it is worth automating.
Engines that depend on the clock are hard to test offline
Some engines are tied to a session (an opening range, for instance). They use the current wall-clock time to decide whether the range window has closed. That means you cannot replay last week's candles through them and get a meaningful answer.
What worked for me: test the plumbing with a fake feed in the right format, check that the engine runs without errors and produces no output outside its window, and accept that the first real validation happens on the first real session.
Smaller lessons
- Isolate every engine in its own try/except. A bug in one detector should cost you one detector, not the whole cycle.
- Don't share strategy code between bots. Copying a file feels wasteful until a fix in one bot silently changes another bot's behaviour. Each bot gets its own copy.
- Keep alerts off while comparing engines. Eight new engines firing notifications on day one is noise, and noise hides the signal you are trying to measure.
What this does not tell you
Nothing here says any strategy makes money. It is about measurement hygiene: if your journal contains trades that could not have been taken, your comparisons are meaningless, whatever the strategies are. Trading carries a real risk of loss, and this is engineering, not financial advice.
I build ICT Edge, a trading-alerts tool, which is where this comes from.
Disclosure: I drafted this post with AI assistance and reviewed and edited it myself.
Top comments (2)
One thing the global marker does that you may not want: bumping ENGINES_VERSION to "v4" later puts all eight existing engines back into warm-up too, so a real setup on an old engine during that first cycle is dropped along with the new engine's backfill. Keeping the marker per engine (state["_engines"][engine.id] = version, warm only for the engine whose entry is missing) limits the blind spot to the engine that is actually new.
Two smaller edges. Writing the marker after the loop means a crash halfway re-runs the engines that already recorded, so the journal needs a dedup key such as (engine id, setup bar timestamp) or you get the opposite problem, the same trade twice. And the age guard only protects you if age is measured from the bar that completed the setup, on closed bars; a setup read off a still-forming candle can appear and vanish between cycles, and that is a fake trade the warm-up flag will not catch.
Your check ("zero trades from the new engines after deploy") is a good one to keep as a test. I'd add its mirror: the old engines still recorded a known setup on the deploy cycle.
Thanks, this is a useful review, and sorry for the late reply. The global marker is a real blind spot: bumping the version puts every engine back into warm-up, so a per-engine marker is the better design and I am taking it as a follow-up. The mirror test (old engines still record a known setup on the deploy cycle) is a good addition too. On closed bars: yes, the engines only read completed bars, never a forming candle. On duplicates: a per-engine lock blocks a new signal while a trade is open, but the crash-halfway case you describe needs a proper dedup key such as engine id plus setup bar timestamp, so that goes on the list as well.