Find My Thingy: A Local-First AI Memory Assistant Powered by Gemma 3 4B
Have you ever remembered saving something important in a document, but couldn't remember which file it was in?
Maybe it was a piece of code, a project note, an explanation, or an idea you wrote down weeks ago.
Searching through multiple files just to find one small detail can be frustrating.
That's the problem behind Find My Thingy.
What is Find My Thingy?
Find My Thingy is a local-first personal memory assistant that helps you find information stored in your documents using natural-language questions.
Instead of manually opening files and searching for keywords, you can add your documents to the application and ask questions about their contents.
The application retrieves relevant passages and uses a locally running AI model to generate an answer with source citations.
The idea is simple: your information stays on your device, and you can ask questions about it whenever you need it.
How does it work?
The workflow has four main steps:
- Add documents: Import PDFs, TXT files, and Markdown notes.
- Extract and index: The application extracts text, divides it into chunks, and creates embeddings.
- Retrieve relevant information: When you ask a question, the system searches for relevant document passages.
- Generate an answer: Gemma 3 4B, running through Ollama, generates an answer grounded in the retrieved information.
Source citations help you trace an answer back to the document it came from.
Tech Stack
Here are the technologies used to build the project:
- Frontend: React, Vite, TypeScript, Tailwind CSS
- Backend: Python, FastAPI
- Database: SQLite
- Vector database: ChromaDB
-
Embeddings: Sentence Transformers (
all-MiniLM-L6-v2) - Local AI: Gemma 3 4B through Ollama
- PDF processing: PyMuPDF
Why local-first?
I wanted the application to work without depending on a hosted AI API for everyday document retrieval and question answering.
Running Gemma locally through Ollama means the application can process questions on the user's own machine once the required models and dependencies are installed.
There is an initial setup requirement: the application needs its dependencies and models downloaded before it can operate fully offline.
Key features
- Import PDFs, TXT files, and Markdown documents.
- Search document contents using natural-language questions.
- Generate answers grounded in retrieved passages.
- Display source citations.
- Store personal memories separately from document content.
- Use a locally running language model.
- Manage documents and saved memories through a dashboard.
Challenges during development
Building a local AI application involves more than connecting a model to a chat interface.
Some of the important challenges include:
- Extracting useful text from different document formats.
- Creating meaningful chunks for retrieval.
- Avoiding answers when the available context does not support them.
- Connecting the frontend, backend, vector database, and local model.
- Managing model availability and local dependencies.
- Making the setup process understandable for Windows users.
One important lesson was that a convincing AI response is not enough. The application also needs to show where the information came from and avoid making unsupported claims.
Current limitations
This is still a work in progress. Some limitations include:
- Scanned PDFs are not currently processed using OCR.
- Indexing is synchronous.
- Chat history is session-only.
- Memory extraction is best-effort.
- The application is designed for a trusted local device rather than multi-user deployment.
What's next?
Some areas I would like to improve:
- Better semantic retrieval for personal memories.
- More robust document processing.
- A smoother one-click Windows installation and launch experience.
- Improved retrieval evaluation and testing.
- Better support for larger document libraries.
Try it out
GitHub Repository: https://github.com/Drishya-code/Find-My-Thingy
The repository contains the source code and setup instructions.
If you explore the project, I'd especially appreciate feedback on the retrieval quality, local setup experience, and ways to make the assistant more useful.
The goal of Find My Thingy is straightforward: stop searching through everything you saved and start asking for what you remember.
Thanks for reading!
Top comments (16)
Source citations on top of a local Gemma 3 4B pipeline is the detail I like most, since that is what makes a personal RAG trustworthy instead of a confident guesser. The hard part I keep hitting is evaluating retrieval quality once it is all on-device with all-MiniLM-L6-v2; I wrote up how I test that for LlamaIndex here: dev.to/kartik-nvjk/llamaindex-make... . How are you checking that the retrieved chunk actually supports the answer it cites?
That's something I'm still improving. Right now, I mainly check this by looking at the retrieved chunks and making sure the cited chunk actually contains the information used in the answer, rather than just being semantically similar.
I haven't implemented a full automated faithfulness evaluation yet. My next step would be to build a small evaluation set with expected supporting chunks and measure things like Recall@k, context precision, and claim-level faithfulness separately. That would make it much easier to tell whether a bad answer came from retrieval or from Gemma itself.
Really like how you've approached this problem. Instead of building another generic chatbot, you've focused on personal knowledge management. That's a nice direction!
thankyou !!
How is the personal memory library different from the document-based knowledge base? Are they stored and retrieved separately?
Yes, they're kept separate. The document knowledge base is for information coming from files I upload, while the Memory Library is for personal facts I explicitly want Find My Thingy to remember.
So basically, documents are things I give it to read, while memories are things I tell it to remember. This separation also makes retrieval more predictable instead of mixing everything together.
Very good project. Good thinking
thankyou!
Could you explain how you are using ChromaDB and Sentence Transformers together for document retrieval?
Yeah! I use Sentence Transformers to turn the document chunks into embeddings, and then store those embeddings in ChromaDB. When I ask something, the query is embedded in the same way and ChromaDB finds the most relevant chunks. Those chunks are then passed to Gemma as context for the answer.
So the basic flow is: document → chunks → embeddings → ChromaDB → relevant chunks → Gemma
Thankyou!!...yeah planning to add that too
Okay
yes!
This is such a cool take on personal knowledge management! Love the local-first approach and the fact that answers are backed by source citations. Great work! 👏
Thankyou 🥰