Most backtests end with one number. Total return, maybe a drawdown and a win rate. I kept getting stuck at that point: the number told me a rule was bad, but not which part of it to change.
So I built Hawk Backtester around a different question. Not "how much did this rule make", but "why did it win, and why did it lose". This is the first time I'm showing it outside Japan, so here is the whole thing, including what it can't do yet.
Same total, opposite fixes
Two strategies can both end at -2.8% and need completely different changes.
- In one, most losing trades were in profit at some point, then reversed and hit the stop. That's an exit problem.
- In the other, most losers were never in profit at all. That's an entry problem, often entering in the wrong kind of market.
If you only look at the total, you tweak the take profit in both cases, and in one of them you're fixing the part that wasn't broken. Everything in the app is built to make that split visible.
Try it without signing up
The demo runs in your browser with no account. The engine is C++ compiled to WebAssembly, and a 1,500-bar run takes about 15 milliseconds, so it re-runs every time you move a slider.
It uses synthetic USD/JPY-like hourly data where trends, ranges and shocks take turns. Pick a market type and a built-in strategy (MA cross, Donchian breakout, Bollinger, RSI) and move the sliders.
https://app.hawk-backtester.com/en/demo/?utm_source=devto&utm_medium=article&utm_campaign=showdev
What it tells you about a run
Which markets it won and lost in. Trades are grouped by the market regime at entry. The same breakout rule made about +10% in trending data and lost about 6% in ranging data. The total hid that completely.
Whether the entry or the exit is the problem. Every trade records its best open profit (MFE), its worst open loss (MAE) and why it closed: take profit, stop loss or the opposite signal. A few hints are generated from those trades, and clicking one filters the trade list down to the trades behind it.
Whether a change really helped. Re-run after a change and the app shows which trades changed compared to the previous run. When I moved a take profit from 1% to 2%, the trade count stayed at 60, but 20 trades ended differently and about half of those got worse. The total said "slightly better".
Whether a parameter is just luck. Sweep two parameters and you get a heatmap. The average of the best cell's neighbours is shown next to it. If the neighbours are good too, it's a plateau. If they aren't, the best cell was probably noise.
There's also a button that sends the aggregated numbers (not your code, not raw data) to an LLM and gets back what to look at and what to test next. It's told not to give trading advice.
Your own strategy, in Python
pip install -U hawk-bt
hawk-bt run my_strategy.py
from hawk_bt import Strategy, Context, Param
class MaCross(Strategy):
fast = Param(10, min=3, max=50, step=1)
slow = Param(50, min=10, max=200, step=5)
async def step(self, ctx: Context) -> None:
... # read ctx.state, call ctx.engine.place_ticket(...)
Your Python process talks to the browser over a localhost websocket, so the strategy code never leaves your machine. Every Param shows up as a slider, and saving the file re-runs the backtest and shows the diff against the previous run.
How fills work
I found four fill-model bugs in my own engine. Fixing them moved results by up to 4% and changed which strategy ranked first, so I write the assumptions down now:
- TP and SL fill at their level, or at the open if the bar gaps through
- if one bar touches both, the stop is assumed to come first
- spread, fees and slippage are charged on every fill
- you can enter at the signal bar's close or the next bar's open
Data
No market data is bundled. You can upload a CSV, generate synthetic data, or import bars with your own data-provider API key (Twelve Data, for example). The key is stored encrypted and only used for your imports.
What it can't do yet
- one instrument per backtest, no portfolios
- bar data only, no ticks
- the app is built for desktop; on a phone only the demo really works
Pricing
The free plan has unlimited backtests (built-in strategies, Python and watch mode), 5 saved results, sweeps up to 25 runs and 3 AI summaries a month. Paid plans raise those limits.
I also wrote up the ideas behind each view: MFE and MAE, splitting results by regime and avoiding overfitting.
https://app.hawk-backtester.com/en/?utm_source=devto&utm_medium=article&utm_campaign=showdev
Almost nobody uses this yet. If you backtest in Python today, what do you look at after the total return, and what would you need to see before trying this on a real strategy?
Top comments (2)
The entry-versus-exit split rests on "was in profit at some point", and that test is easy to satisfy by accident: with a stop of 1% and noisy bars, a random entry will often show a small positive MFE before it is stopped out, so "most losers were once in profit" can come out high even when the entry has no edge. A sturdier cut is MFE as a multiple of the stop distance (say at least 1R) and, as a control, the same MFE distribution for the same exits applied to random entry times. If the real losers' MFE doesn't beat the random-entry one, it is not an exit problem.
On the regime table (+10% trending, -6% ranging): how is the regime at entry decided? If it comes from a label the generator knows, fine for the demo. On real bars it has to be computed only from data up to the entry bar (a trailing ADX or range ratio, not a trend fitted over the whole series), otherwise the rule looks good in "trending" partly because trending was defined by what happened next.
The stop-first assumption when a bar touches both levels is the pessimistic choice, which is the right default. Do you report how many trades in a run actually hit that case? On hourly bars with 1% levels it can be a meaningful share, and it tells the reader how much the result depends on the assumption.
Thanks, this was useful enough that I changed the tool before answering.
On the split: you're right that "was in profit at some point" is too easy to satisfy. The demo uses a rough cut (MFE above 80% of the final loss and above 0.15%). The MT5 report analyser was stricter (MFE of at least half the loss, with a resampled 90% range), but it had no control. It now has the one you describe: each loser is replayed from random bar opens with the same side, the same loss distance as a stop and the same holding time, MFE is measured as a multiple of that loss distance, and the random losers are classified with the same rule. If the real share doesn't beat the random one (one-sided, about 95%), the tool no longer calls it an exit or entry problem. On the bundled sample, "in profit first" is 34% against 25% for random entries, so that one holds. "Went wrong right away" is 33% against 39%, so the entry finding was noise.
On regimes: in the demo the label comes from the data generator, as you guessed. In the MT5 analyser it's ADX(14) on 1-hour bars, read from the last hour that closed before the entry, so nothing after the entry is used.
On same-bar ties: they weren't reported. Now the exit analysis tab and the demo count stop-outs whose bar also reached the target, and show how much profit would change if the target had filled first. In the demo with 1% levels on hourly bars it's 0 in every bundled market, because an hourly bar there moves far less than the 2% between the levels. With tighter levels on real data it won't be.