📝 Originally published (in Japanese) at forge.workstyle.tech.
TL;DR A Japanese blog was translated and automatically posted to dev.to, but its title and description remained in Japanese. The English body and generated devto.md made the post look ready, so the check for total translation failure did not stop it. A pre-post check now inspects the title, description, body, and tags, while people still review meaning and numbers.
When the Body Is in English, It Slips Through
If translation failed, no file for posting was created. That safeguard existed. But it only stopped the process when the entire translation failed and no file was produced.
In a case like this one, the body was in English while only the title and description remained in Japanese. The English body had been generated, and so had devto.md. From the perspective of the downstream crosspost.mjs, it looked like a post-ready article was available.
The rule “don’t create the file if English translation fails” assumed that failure would affect the article as a whole. In reality, each field could succeed or fail independently. The body could be translated while the title and description were left behind. There was no check to catch that partial success.
It wasn’t just the title. The article also had a Japanese table and timeline inside a code block. For example, this section with the numbers:
YouTube Shorts
Impressions Approx. 2,900
Views 279 (continued viewing: 9.5%)
Average view duration 0:13 (about 85% of a 15.3-second video)
Even when the body paragraphs had been translated into English, blocks like these weren’t necessarily in English too. Code blocks aren’t only for programs; they can also contain logs, tables of numbers, and processing timelines. Reading just some of the visible paragraphs wasn’t enough to know the state of the article as a whole.
The English Translation Had Other Errors, Too
The articles I reviewed around the same time also contained mistranslations in the body text.
The output token count, “134万” (1.34 million), had become 134,000 in English. “二次関数的に膨らむ” (grow quadratically) had been translated as exponentially, and “制限にはかかっていません” (it is not subject to the limit) as it is not being used. Even when the meanings seem close, they do not describe the same state or value.
The code block language tags were wrong, too. Every log or text block was tagged python. Checking only whether the text had become English could not tell us whether the numbers were preserved, the meanings were accurate, or the block types were correct.
There were limits to checking translation success with a single indicator. The fact that a file had been created did not mean that every part of the article had been translated correctly.
Natural-Sounding English Errors Can’t Be Found by Checking the Characters Alone
On October 2, 2026, a person spotted and corrected another article’s title before it was published.
Comparing Japanese Speech Recognition with 1,000 Utterances
The article was about classifying utterances, not speech recognition. The English phrasing wasn’t unnatural. That’s exactly why checking only for leftover Japanese characters can’t catch this kind of error. The tags also contained the same value twice: "llm, llm".
A person caught the title before it went live. That made one thing clear: if we rely on pre-publication review, we could miss another error. We need to catch leftover Japanese, but we also need to block anything we can validate mechanically at the entry point to the publishing process.
Still, character checks can’t catch every mistranslation. Speech Recognition is natural English. The number 134,000 also looks like a natural English number. Checking only the character types won’t reveal a mismatch with the original meaning.
Inspect the Article Itself Right Before Posting
On October 3, 2026, I added a pre-post check to crosspost.mjs. It checks the title, description, body, and tags in the actual devto.md being sent—not whether the translation process succeeded.
If the title or description contains hiragana or katakana, the post is held. If any Japanese remains at the point of posting, it is not sent as is. For the body, the check excludes code blocks and inline code, then holds the post if three or more lines contain kana or katakana.
The body threshold is not zero. It allows one or two lines of Japanese, such as in data examples or quotations. Code blocks and inline code are also excluded from the body check. That way, Japanese characters or values there alone are not treated as a translation failure across the entire body.
The check also holds the post if tags are duplicated. When a post is held, the reason is logged and a notification is sent to the Mac. If the content has been reviewed, it can still be posted with --force. After automatically stopping a post, there’s still a way to review the content and let it through.
Results from Checking 94 Articles
After adding the checks, I reviewed 94 existing articles. None had Japanese text left in their titles or descriptions.
This means no Japanese text was found in the titles and descriptions of the 94 articles reviewed. It does not mean there were no mistranslations in the body text or that all the numbers were correct. We need to distinguish what the checks can verify from what still requires a person to read and review.
This setup can catch hiragana or katakana in titles and descriptions, substantial amounts of hiragana or katakana left in the body text, and duplicate tags. It cannot identify mistranslations that read naturally in English or numbers that have been transcribed incorrectly. A person still needs to read the articles to check for those issues.
This check also cannot detect text embedded in PNG diagrams, so the text in those diagrams remains in Japanese. If we create diagrams in SVG or HTML, we can replace the text and export English versions. We have decided to take that approach, but the process for creating English diagrams has not yet been implemented.
Key takeaways
- Check the title, description, body, and tags in the output immediately before posting.
- Hold posts when the title or description contains hiragana or katakana, or when three or more body lines contain them outside code.
- Hold duplicate tags, log the reason, and send a Mac notification; reviewed posts can be sent with
--force. - Human review is still needed for mistranslations and number changes that look natural in English.
- The check does not inspect text embedded in PNG diagrams; the English diagram workflow is not implemented yet.
Top comments (0)