Google dropped two-thirds of our site from its index in a single day. We found the culprit — except it wasn't him

Hi! I want to tell you a story about two weeks we spent figuring out who tanked our site in Google. We found someone, we were pleased with ourselves, and then it turned out we'd got the wrong guy.

A bit of context first. We build Toolevia, which is eighty-four file tools that run right in your browser: merge PDFs, strip the background off a photo, work out how much a mortgage really costs, that sort of thing. The file never leaves your machine, because there isn't a server in the picture to send it to. We spent three months writing it, and in August we shipped the whole thing over three weeks: 84 tools in 58 languages, which multiplies out to 5,104 pages.

And, to our mild surprise, it took off. On August 8 Google had 494 of our pages in its index; ten days later it had 2,218, and on the 20th the site showed up in search results 1,269 times in one day. For a domain that was two weeks old this felt genuinely good, and I'll admit we got a little pleased with ourselves.

Then on August 21 we had 1,134 impressions. On the 22nd, 216. On the 23rd, three. Not three thousand, not three hundred — three, for the whole day. And that's roughly where it stayed, somewhere between one and seven a day, right through the end of the month, as if someone had turned the dial all the way down and gone home.

Bars are how many pages Google reported as indexed in its own snapshots; the line is daily impressions. The red dashed line is the day in question.

First thought: we must have broken something

Which is honestly the most comforting theory there is, because anything you broke yourself you can also fix yourself.

So we went into Search Console. Manual actions: "No issues detected." Security: "No issues detected." Server errors: zero, ever. But 1,337 pages had left the index, and another 1,406 were sitting in "Crawled – currently not indexed," which, if you think about it, is the most insulting status Google offers. It downloaded the page, read it, and decided it didn't want it. Not "couldn't" — didn't want to.

Then we went over the site properly, all of it rather than a sample, after first checking that production was serving byte-for-byte what we had in the build directory. Here's how that went.

The only things that changed in that window were translation dictionaries and three UI components.

You know what's worse than a broken line in robots.txt? Not having one. A broken line you fix in a minute, hit "request indexing," and go get a coffee. Here there was nothing to fix.

We did find two mistakes. Just not the right ones

Since we were in there anyway, we found our own sins, and I'll be honest about them because they're instructive.

Sin number one: we changed the URLs of pages that were already published. Everyone knows you don't do that, we knew you don't do that, and we did it anyway — at scale. We renamed 159 URLs in a single day when three tools got better names in all 57 languages at once. That left 208 dead URLs, 122 of which Google had already filed under 404.

Sin number two is dumber. Our sitemap generator, if a language didn't have its translation yet, would fall back to the English URL and put that in the sitemap instead. Between "English text is live" and "the other 57 translations have landed" there were usually a few hours, and during those hours the sitemap contained ghosts like /bn/multiple-pages-per-sheet-pdf/ that died the moment the Bengali arrived. Google barely caught any of them — seven in a sample of a thousand — but the mechanism fired on every single tool.

We've closed both: the dead URLs now 301 to their replacements, and slug changes and the sitemap are guarded by checks that won't let it happen again. But neither explains the drop. 208 out of 5,104 is four percent, and we lost two-thirds.

Suspect number one: us, and our last batch

Here's where I have to confess. This theory was mine, and I bought it immediately and completely.

Look how nicely it fits. Waves 7 and 8, our last seventeen tools, nearly a thousand pages, shipped between August 22 and 29. The drop started on August 22. A young domain gets five thousand pages dumped on it in three weeks, the same text multiplied across 58 languages, and Google — obviously — decides it's looking at textbook scaled content abuse and cuts the whole thing off. The dates line up, there's a motive, the picture is clear.

Clear enough that I didn't want to test it. Which is exactly why we tested it in a way that could make it fail, rather than a way that would confirm it.

We split every page into two groups: group A had existed before August 22, group B appeared on the 22nd or later. The birth date of each of the 4,872 language-plus-tool pairs came from git, where it's recorded to the minute. The logic is simple: if Google tanked the index because of the new batch, the new batch had to at least exist at the time of the drop, and Google had to have read it.

The suspect turned out to have three alibis, any one of which would have been enough.

First, he hadn't arrived yet. On the day of the drop, waves 7 and 8 amounted to nine pages out of 4,127 — nine, two-tenths of a percent of the corpus. Ninety-three percent of their pages went live after the drop had already finished.

Second, the clock. The first tool of wave 7, remove-exif, was committed on August 22 at 22:58, with English text only and not a single translation. By then, August 22 had already delivered its 216 impressions instead of 1,134. A cause can't come after its effect, however much you'd like it to.

Third, nobody had read him. You can only judge what you've downloaded, and the wave 7–8 pages have zero impressions in the site's entire life; in the samples of non-indexed pages, out of their ninety-two, Google had fetched exactly one.

The theory died, and it immediately became obvious what had actually dropped: group A. Of the 800 pages that had ever appeared in search, 359 are sitting in "Crawled – currently not indexed," and all 359 existed before August 22. Adjusting for the fact that Search Console doesn't export the whole table, that's roughly two-thirds. These weren't new pages that hadn't made it in yet. They were old pages that had been in the index and got thrown out.

Suspect number two: the core update

On August 26 Google started rolling out the August core update, and the temptation to blame it was enormous — it's convenient, and it means you don't have to change anything.

Except the dates don't work, and not by a little. We hit bottom on August 23, three days before the update began, and three impressions a day is bottom in the literal sense; there's nowhere further down. There's no second step in the data. Maybe the update piled something on top, but you couldn't see it if it did, because by the time it arrived there was nothing left to drop.

So who, then?

What remained was to look at how Google chose what to drop. If it was judging content, there should be some gradient: short texts falling out more than long ones, weak pages more than strong ones, something along those lines.

There is no gradient. None at all. By text length, the share of dropped pages is 41–48% whether the page is under 300 words or over 500. By prior visibility it's 44–50%, and the most visible pages, the ones with 30+ impressions, actually dropped less, at 33%. By ranking position it's 41–50% whether you were in the top ten or past sixty.

Flat shelves. So Google wasn't weeding out weak pages; it made a decision about the site as a whole and removed roughly every second page without much regard for which one. As far as we can tell, that's the signature of a host-level decision, not page-by-page filtering.

There is one lead. What Google had managed to read before the drop was waves 1 through 6: the corpus grew from 880 to 4,118 URLs between August 7 and 19, and 63% of the dropped pages were last fetched between August 16 and 19. But that's a correlation, and we're not going to name it as the culprit, because we've already been burned once on evidence like that.

Case still open

We're left with two theories, and with data from a single site there's no way to tell them apart.

One more thing from the same export, and it also points to a site-wide decision: not one of the 58 languages dropped out entirely. Google keeps some of every language, just very unevenly. Hebrew gets 23% of its pages in, Latin-script and Cyrillic languages around 20%, and the eleven Indic languages, from Hindi to Tamil, get 7%. That's probably a post of its own, but the chart is too good not to show.

Share of pages with at least one impression, grouped by writing system. Red is the group Google keeps the least of.

One: an algorithmic reassessment of the site as duplicated content. 84 texts in 58 languages is 4,872 pages carrying the same value, and we naturally believe that translating for different audiences is nothing like 4,872 clones aimed at one audience. Google, it seems, is worse at telling those apart than we'd like.

Two, and I like this one just as much: the domain is young, Google gave the pages a chance to show, measured what happened, and pulled them. Plenty of people describe this. The only thing that bothers me is the shape — it's usually described as a gentle slide, and ours went from over a thousand impressions to three in two days, which isn't a slide, it's a cliff.

What we know for certain is what it wasn't: not a manual action, not a technical fault, and not the last batch.

What we're doing about it

We're not shipping new tools for now, because at 15.6% indexed another ten pages fix nothing, and if theory one is right they'd just repeat the hit. Construction is frozen for the duration of the investigation.

Our own two mistakes we've closed with mechanisms rather than memory, which as it turns out is unreliable. Redirects are now generated from a registry of published URLs, and the registry itself is a gate that won't let anyone change a published slug or ship a tool that isn't translated into every language. I'd like to think we won't step on that particular rake a third time.

We also started this blog, and that's part of the plan too. It's in two languages only and doesn't get multiplied by translation — if Google really is reacting to sameness, then writing that exists nowhere else is about the only thing we have to offer it.

And then we wait. On October 1 we'll pull the same set of Search Console reports and compare. The baseline is written down: 15.6% indexed, 798 pages with impressions. If anything moves, I'll write it up — I'm curious myself.

If you've seen something similar — young domain, big multilingual launch, a cliff-shaped drop with no manual action — tell us in the comments how it ended. We genuinely want to know, and not only because of our own site.

The rule I wrote down for myself from all this: the most convincing theory is the one you should test first, precisely because it's the one you don't feel like testing.