You run a backtest, the curve looks good, and then the uncomfortable question lands: is your data actually clean?
Most MT5 CSV exports are dirty — quietly. The terminal hands you a file that looks complete. It is not.
The five failure modes
Gaps. Connection drops, broker outages, re-export windows that stop mid-hour — missing bars that nobody notices until results drift between platforms.
Duplicates. Re-export the same range twice, merge two files, and the same bar appears twice. Inflated volume counts, broken "first bar of the day" conditions.
Zero-volume / zero-price bars. Exchange halts and feed glitches produce bars with no trades. Indicators computed on them are wrong, and the wrongness does not show up on visual inspection.
Timezone ambiguity. MT5 stores bars in broker-local session time. The same CSV read as UTC moves your strategy to a completely different time of day.
Expected vs actual. A known date range + timeframe should contain a known number of bars. If your file has 9,282 bars where 12,276 are expected, something vanished — and you would not know without doing the arithmetic.
What a clean-data check looks like
A useful report answers five numbers per file, in under a minute:
expected bars vs actual bars
missing bars, and how they cluster (one long gap = broker outage; many short = feed drops)
duplicate bars (exact count, not "there might be some")
zero-volume ratio
timezone/session note — the report states its assumption instead of hiding it
Example: 12,276 expected bars, 9,282 present → 2,994 missing, clustered in 2 gaps (2,970 + 16), gap-free ratio 75.6%. That is a file you should not backtest without first understanding why two days vanished.
Healthy baseline: gap-free ≥ 99%, zero duplicates, zero-volume < 1%. If your file fails that, the results above it are an estimate, not a measurement.
Try it — free, no signup
I built a small tool that runs this check in your browser: https://mt5-cleaner.streamlit.app
Drag a CSV in → it detects broker format, timeframe, and range → you get the quality report plus a cleaned CSV. Files never leave your browser and are gone when you close the tab. No account, no API key, free up to 200 MB.
The workflow I use before any backtest:
Export history from MT5 (CSV)
Run it through the checker — read the 5 numbers
If gap-free < 99%, re-export with a wider range or a different session before touching the strategy code
Sixty seconds of checking saves hours of debugging a strategy that was never wrong — the data was.
Source: https://github.com/RedSamerai/quantclean
Questions or weird report shapes — post them in the comments, the clustering heuristics are the fun part.
RedSamerai
Posted on Edited on
I Built a Free In-Browser Tool That Tells You If Your MT5 Data Is Clean
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (4)
Good checklist. Clean data fixes one problem, but there is another one that data checks cannot catch: how many variants you tried before picking the one you are testing. A result that looks good after 20 tries needs a stricter threshold than one that was tried once. Writing down the pass criteria before the first run and counting the variants you tried is a cheap way to keep the clean-data result honest.
this is the half I keep kicking down the road. A cleaner can fix the measurement, but it's blind to how many variants you tried before the one you're keeping — 20 tries needs a stricter bar than 1, obviously. Writing the pass criteria down before run #1 and keeping a variant count is the boring cheap fix, and I think it belongs in the backtesting setup itself, not left as a habit.
The 12,276 expected vs 9,282 present example is a good illustration, but "expected" is the number I'd trust least: it depends on the instrument's session calendar. Index CFDs and futures close daily and on holidays, so a naive bars-per-day count flags those closures as gaps. A gap report is more useful if it separates gaps that fall inside a known closed session from gaps inside an open one.
On the timezone note, one more case: MT5 broker time often follows a fixed offset from New York (a common setup is GMT+2 in winter, GMT+3 in summer), so the broker-time clock jumps against UTC twice a year. A file that spans a DST change has two different UTC mappings in it. A cheap check is to look at the hour of the first bar of each trading day: if it moves by one hour across a US or EU DST date, the offset changed mid-file, and reading the whole CSV with one offset shifts half of it.
On duplicates, do you distinguish exact repeats from same-timestamp bars with different OHLC (what you get when two exports of the same range come from different server snapshots)? Dropping the second one silently picks a price.
honestly the expected count is the one I trust least — it's just bars/day × days, so anything that doesn't trade every single day (index CFDs, futures, your local holidays) shows up in the gap list as a "problem" that isn't one. splitting it into gaps-inside-closed-sessions vs gaps-during-open-hours is the actual fix, and I'll do that so only the second half needs your attention.
The DST thing I'd have missed. Fixed offset from NY (GMT+2/3) is basically the default broker config, and a file crossing the switch is carrying two UTC mappings in it. Your first-bar-hour-per-day check is the cheap detector, that's what I'm adding.
Dupes: you're right, I'm not distinguishing exact repeats from same-timestamp-different-OHLC. And that second case is the one where picking is a real decision, not a cleanup — I don't want to silently keep whichever export happened to be first. They should surface as "ambiguous, you pick."