AI Photo Restoration Keeps Changing Faces — Here's the Pipeline That Stops It

If you've run old family photos through an AI restorer, you've probably hit the failure mode nobody advertises: it changes faces. Heavy restoration on a low-resolution scan doesn't recover a face — it re-draws a plausible one. You end up with a stranger wearing your grandmother's expression.

Why faces drift

Face restoration models are trained to produce plausible faces. With a high-signal source, "plausible" and "correct" converge. Below a certain signal level — small scan, compression, damage sitting on the features — the model stops restoring and starts re-drawing. It's a fidelity-versus-strength trade-off baked into every tool of this class.

The pipeline

1. Scan at 600 dpi, in color — color even for black-and-white originals, because the stain and tone information helps separate damage from image. A flat daylight photo of the print beats nothing, but a real scan beats everything.
2. Repair damage first, enhance faces second, upscale third — separate passes, not one mega-pass. Scratches and fading act like added noise, and face restoration running on top of damage is where the drift starts.
3. Bias toward fidelity, not strength — the "beautiful" results that swap identity are usually high strength on low-res input.
4. Verify at 100% zoom on eyes, smile, hairline — if any of the three changed, reject the pass and re-run with less strength. That's the identity check.
5. Colorize last — colorization is a guess about the past, not a record of it. Keep the black-and-white original as the archival master.

The hard ceiling

Past roughly half of the facial detail missing, no model can recover identity — anything it produces is invention. That's manual-restoration territory. Below that threshold, the order of operations above is the difference between a sharper version of the same person and a stranger.

What I built with this

I packaged this workflow into AI Photo Editor: free local adjustments (crop, tone, contrast — nothing uploaded) plus a few free AI edits a day for enhance / restore / background removal. The local layer handles the prep steps without spending an AI credit, and the AI pass only ever sees the cleaned scan.

Honest limits: it won't fix a photo beyond the reconstruction threshold above, and keeping the period look is the goal by design — if you want a "modernized" face, this is the wrong tool.