DEV Community

y4u
y4u

Posted on

I Kept Testing "Large File" Tools: PilotEdit Crashed on a 3GB Japanese Replace, lnav Crashed Indexing 48GB

After benchmarking EmEditor, klogg, UltraEdit, 010 Editor and a Wasm viewer on real data, I asked several AIs which tools I still owed a test. Two names kept coming up: PilotEdit (Windows) and lnav (Mac/Linux, free OSS).

Short version: neither survived the 48GB real-world file (OpenStreetMap Japan XML, 892 million lines — the same file my whole series uses).

PilotEdit: opens fine, can't fix anything

Quick Mode (its dedicated large-file mode), Windows notebook with 16GB RAM:

  • Open: 3GB = 15.4s / 10GB = 44s / 48GB = 4m32s — roughly on par with EmEditor (4m06s). The indexing works
  • But the UI blocks completely while loading: the window goes dark for 4.5 minutes, not even the first line visible
  • Replace-all Tokyo東京 (5 bytes → 6 bytes, an utterly ordinary multi-byte replace): 3GB crashed after 2m10s; 10GB and 48GB hung 5+ minutes, never completed

For scale: 3GB / 100M lines fits entirely in RAM, and on the same file EmEditor does that replace in 0.9s; my own viewer (UwView Pro + Edit Upgrade) in 6.3s for two.

lnav 0.14.1: 3GB and 10GB complete, 48GB crashes at 1:48

lnav went in as CLI culture's representative for interactive log investigation (Mac M4, 32GB RAM). 3GB and 10GB (each ~100M lines) complete: full read-through in 35.5s and 1m38.8s, interpreted as logfmt logs, displayed down to the last line at 100%.

Search works there too: 2–3s at 3GB; Osaka 9.95s and Tokyo 10.1s at 10GB (klogg does ~2–3s, my viewer 1–2s on the same 10GB file).

At 48GB, first paint took 1.4 seconds — among the fastest I've measured. Then, at 1m48s, it crashed, apparently while building its index — at 48GB I never reached the search tests. lnav's wall sits somewhere between 10GB and 48GB.

In fairness: lnav is built for log files; a 48GB XML treated as plain text is outside its main lane. And for both tools — this could be a configuration issue on my side; if you know better settings, please tell me and I'll correct this post.

Three tiers of "right after opening"

  1. Full block (PilotEdit): dark screen until loading ends — ~4.5 min at 48GB
  2. Head only (klogg, EmEditor): top visible instantly, but no jumping to the end until the full scan finishes — ~2–9 min at 48GB
  3. Whole file immediately (the UwView family, excluding the Wasm version): background indexing (54–140s) while you jump anywhere from second one

lnav feels like tier 3 for 1.4 seconds — then it's gone.

48GB real-data scoreboard so far

Tool Open Search Replace
UwView Pro + Edit Upgrade immediate (index 54–140s) 12.5–23.9s 47.0s (two replaces)
EmEditor 103–301s 144–245s 11+ min, 31-min saves (16GB RAM)
klogg (viewer) head only, 110–545s 120–585s
grep / ripgrep 64.64s / 71.49s
PilotEdit 4m32s (dark screen) failed
lnav 0.14.1 1.4s first paint → crash at 1m48s (3GB/10GB: completes) — (never reached)
010 Editor 56.7s replace 12m45.3s, save crashed
Log Voyager stops ~5GB did not work

Old reviews saying "PilotEdit handles huge files comfortably" usually mean tens of MB to ~1GB, judged by "it opened and found an English word" — sometimes on artificial data that's mostly newlines. Real data packed with tags and multi-byte text, plus a length-changing replace, is a different load entirely.

One takeaway: among GUI apps there's hardly anything left to compare against, so from here my comparisons go mainly against CLI tools. My viewer's drill-down search has already gained grep's -v and -w as checkboxes, plus a uniq -c-style frequency tally (all shipped in v1.5.0); the next feature is being selected now.

All numbers are single-environment measurements by the developer of one of the compared tools — apply the discount. Full conditions and tables:

https://uvp.y42u.net/en/blog/uvp-pilotedit-lnav-48gb-en/

UwView Pro is on sale (Windows, macOS, Linux — one license covers all three; 14-day free trial — editing features included).

Top comments (0)