This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
💡 Quick Navigation: What I Built · Live Demo · ...
For further actions, you may consider blocking this person and/or reporting abuse
🦥 A little more context behind slothd!
I wanted to build something that could quietly take care of digital clutter without constantly demanding my attention or sending my personal files to the cloud.
This is my Week 3 submission for the Hacktoberfest Open-Source AI Challenge, and I had a lot of fun bringing together local AI, idle-aware automation, and reversible file operations.
The 35-second demo and offline fixture are included in the article if you'd like to see it in action!
Now I'm curious: What's the messiest folder on your machine, and would you trust a local AI assistant to organize it? 👀
Happy Hacktoberfest! 🍂🦥
Local-first daemon + local model is the right shape, but daemons have a failure mode a chat window doesn't: the model call is slow and the file events keep arriving.
How do you handle the backlog — a bounded queue you drain, or drop-and-rescan?
I ask because I measured a 1.5B answering four narrow yes/no checks on CPU at roughly 48 seconds. At that latency a file watcher can stack a couple of dozen pending events inside a minute, and the interesting question becomes which ones you're willing to skip.
Definitely drop-and-rescan on idle — trying to hook a local model to a live inotify event queue is basically suicide.
If you run
npm installor extract a zip, a reactive file watcher immediately drowns the model. Soslothddoesn't listen to continuous file event streams at all. It just checks OS idle time periodically, and only runs a fresh directory scan once you've stepped away from your keyboard for 5+ minutes.When it wakes up, it deals with that CPU latency in two ways:
.exe/.dmg), archives (.zip), code, and exact SHA-256 duplicates are sorted or pruned instantly (<1ms) by metadata triage. Gemma only ever gets invoked for truly ambiguous files (cryptic PDFs, unnamed receipts/screenshots).So instead of an exploding FIFO queue, the backlog just quietly chips away across your breaks.
Drop-and-rescan on idle explains the backlog, and the 80% metadata triage is the part I'd steal. One thing I'm still curious about: with a 20-file cycle and ambiguous files costing tens of seconds each, a 200-file ambiguous backlog needs ~8 minutes of idle — does the next idle window resume where it stopped, or does every window re-rank and risk starving the same low-priority tail forever?
It doesn't starve the tail because the files are physically moved upon batch execution!
Once a 20-file batch is organized, those files are moved into their categorized folders and logged to SQLite WAL. On the next idle wake-up,
scan_directory(skip_organized=True)ignores the moved files. So your 200-file backlog shrinks to 180 → 160 → 140 until it hits zero. The filesystem itself acts as the persistent queue.And you don't even have to wait for idle if you want it done right now:
slothd organize ~/Downloads --go --ai(or hit "Approve" in the web dashboard).--idle-threshold 0on the daemon to just plow through continuously.Idle mode is just the polite default so it never cooks your laptop while you're actively typing.
The filesystem-as-queue is the right instinct — the disk is the one ledger you can't lose — but it answers the starvation question, not the crash one. My 3ima1 question still stands: if slothd dies between the move and the WAL append, the file is organized on disk but invisible to the manifest, so the next scan re-organizes a file that already has a home. The fix I'd steal from document pipelines: write the manifest entry as PENDING before the move, flip to COMMITTED after — on restart, anything still PENDING gets verified against disk instead of trusted. Do you fsync the WAL before a batch counts as done?
@dj29 — one follow-up on the skip_organized path, and it runs into Robert's point below from the other side.
Files-as-queue is a better commit point than a queue row, and moving the file explains why the tail cannot starve. The case I would still poke at is the inverse: if an organized file later leaves its folder — restored from a backup, renamed, moved by another tool — does skip_organized=True treat it as new and re-run the model on it, or does the manifest match on content hash and skip it?
The second one is what makes undo trustworthy; the first one quietly re-organizes a file that already has a home.
Robert's crash window and this are the same missing check from opposite directions — disk organized / manifest silent, and disk gone / manifest organized. Both want one reconcile pass that trusts neither side on its own, which is why I would make the manifest write the commit point rather than the move.
The dry-run and undo features are what make this practical to trust with real files. AI can make mistakes, so being able to review changes before applying them is just as important as the organization itself.
The two-stage approach is especially smart; using lightweight triage before invoking a local vision model keeps the workflow efficient, while dry-run previews and rollback make automation much safer. I also like the privacy-first design. How well does Gemma handle ambiguous files with misleading filenames or mixed content?
Thanks so much for reading and for the great question, Saeed!
That was actually one of the core edge cases we tested during development. Here is how
slothdand Gemma 3 handle misleading filenames and mixed/ambiguous content:Content Always Overrules the Filename:
When a file hits Stage 2, Gemma doesn't rely on the filename to guess what it is. For PDFs, PyMuPDF extracts the raw text body (or, if it's a scanned document with <50 characters of text, renders the first page into an in-memory PNG at 2x zoom). For screenshots and receipts, the raw image bytes are passed directly to Gemma 3 Vision. Because visual tokens and document text take precedence over the filename string, a cryptic file like
Screenshot_2026-10-18.pngorINV-9041-final(1).pdfgets identified by its actual header, logo, and line items (e.g., Uber Receipt or AWS Invoice), generating a clean proposed name likeUber_Receipt_2026-10-18.pdf.Mixed Content & Multi-Label Semantic Tagging:
Real-world documents are rarely clean-cut (e.g., an invoice that is also a legal contract). Our Pydantic schema forces Gemma to emit three layers of categorization:
category(e.g.,Documents)subcategory(e.g.,Service Agreement & Billing)tags(e.g.,["aws", "cloud", "invoice", "legal", "q3-taxes"])These tags and summaries are indexed into Tiger Data
pgvector(with local SQLite vector fallback). Even if a mixed document ends up filed inDocuments/2026/October/, runningslothd search "cloud hosting bill"or querying via the dashboard immediately surfaces it based on vector similarity.nullfor vendor/date and assigns a lowconfidencescore. Whenai_proposed_nameis null,slothdstrictly preserves the original filename and falls back to extension clustering.And most importantly: because every batch runs as a non-destructive dry run first with human approval and SQLite WAL logging, an ambiguous file's destination can be inspected before moving, or instantly reversed with
slothd undo! 🦥**That’s a solid approach! **Prioritizing actual document content, using multi-label tagging, and preserving ambiguous files rather than guessing makes the system much more reliable. The human approval and rollback safeguards are especially important for building trust in local AI automation. 😃
Hey nice project @dj29 . This is very useful for me to clear out digital clutter, as my digital files aren't a mess but a disaster 💀
Thanks Tanay! I'd be glad if you could run it and tell me your experience. And don't worry it has batch rollback too, so if you didn't like how slothd organized your files, you can batch rollback it.
Have A Great Day!😄
The two-stage funnel is the part I keep coming back to — cheap metadata triage first, and only waking the vision model for files that actually need semantic reading. That, plus dry-run by default and the SQLite WAL undo log, is exactly the safety story an auto-organizer needs before I would let it loose on my Downloads folder. The confidence-as-percentage bug (Gemma returning 90 instead of 0.9) is a great catch too — that kind of quiet failure is nasty in production.
Fantastic idea, and great implementation. I can see ways one could extend this -- it definitely has its own idea of "this is organized" that might not fit everything -- but the base idea of "nothing ever leaves my machine to be categorized" is great and probably something we're all going to have to tackle one way or another as agents become more pervasive and accepted.
Teaching it to look at some files that are organized and filing into that structure would be very nice indeed. I was hoping it could do that out of the box, but it's very useful as it is.
One I'd save for after finals i.e. edit an organized file, then try undoing that batch. I'd want undo to flag the changed content and leave it intact rather than quietly treat it as the original. This fault-tolerant work-queue exercise approaches the same ownership problem from another angle i.e. finishing an operation and still having authority to change its result are separate things.
Sure is, and I'll definitely go into these edge cases once I don't have to worry about exams anymore.
Great Bro, keep it up and All The Best!
Thanks Bro! Give it a try and gimme feedback when you're free!
Sure Bro.
Nice work. Defaulting to dry-run with an undo manifest, and rescanning on idle instead of hooking the model to inotify, are the right calls. The back-and-forth on crash safety in this thread is great to see.
Thats Awesome Keep moving forward!
Cool 🔥
Thanks!
🥰🥰🥰🥰🥰🥰🥰🥰🥰🥰
🥀Very cool project
Thank you so much for sharing this!
One edge case I'd add to the test suite is symlink handling. Imagine a file in
~/Downloadsis actually a symlink to a document inside a private project directory. If the scanner and organizer don't agree on whether to follow that link, the daemon could read or move something outside the intended workspace.I'd also test a symlink being replaced between scanning and execution. A path check during the scan doesn't guarantee the target is still the same when the move happens. Using no-follow filesystem operations where supported, checking file identity again before execution, and treating symlinks as separate entries by default would make the boundary much safer.
Have you considered adding symlink and mount-boundary cases to the offline test fixtures?
The drop-and-rescan answer to rytjy is the right call — a reactive watcher hooked to a local model is indeed a great way to drown. But there's a crash window in the undo design worth thinking about: if slothd dies between moving a file and appending that move to the WAL manifest, your undo set silently loses a file, and no amount of reverse-order restore can bring back a move it never recorded. In document pipelines I handle this by writing the manifest entry in a PENDING state before the move and flipping it to COMMITTED after — the filesystem rename is the atomic point, and on restart anything still PENDING gets verified against disk rather than trusted. Do you fsync the SQLite WAL before a batch counts as done, or is the manifest the one thing you'd still lose on a power cut mid-cycle?
The two-phase manifest is a strong choice, especially with startup reconciliation. One boundary worth specifying is the post-rename crash window: if the destination exists but the WAL update did not commit, which fields decide whether reconciliation marks the intent completed versus rolls it back? A tiny deterministic fixture that kills the process after each phase would make the recovery claim much easier for contributors to verify.
Local-first is the right call for this job class - a daemon tidying ~/Downloads doesn't need a cloud round-trip, and offline-by-default means it degrades to queueing instead of hard-failing. The design question I'd push on: what's the rollback story? Anything that renames or moves files autonomously earns trust only with a replayable action log - a reliable 'slothd undo' after a bad batch is the feature that would make me install it. Second question: when two destination folders fit equally well, does it ask, pick deterministically, or skip and log? Silent wrong guesses are how users learn to distrust organizers.
one question on the air-gapped claim: sentry is listed as part of your dev loop, but traces carrying file paths and prompt tokens stop being local the moment they leave the box. the cloud integrations are opt-in, i believe, but the default trace payload is worth pinning down — what does a span like ai.gemma_classify_batch carry, and can it be redacted before it reaches sentry?
the safety layer is the real contribution here. dry-run by default plus a two-phase commit log in sqlite is how you make an ai that touches your filesystem feel safe instead of scary, and the startup reconciliation pass is the part most projects skip until the first power cut.
i run a similar two-stage shape at home — cheap heuristics first, model second — because the model is the slow and expensive part. nice work.
The preview/approval boundary and the reconciliation pass make the safety model much easier to reason about. One additional case I would test is a file changing after the preview but before approval: does execution recheck the hash and skip a stale plan? Idle scanning reduces contention, but an editor or sync client can still change the source in that gap.
Having the demo fixture run in under a second, offline, is a nice touch for anyone trying it. Do the 1 ms heuristic and TabPFN ever disagree on a file, and if so which one wins?