DEV Community

Cover image for Availazy: Gemma 4 forgot to cancel Thursday. A keyword list didn’t.
Bipul Chamoli
Bipul Chamoli

Posted on Edited on AI-assisted

Availazy: Gemma 4 forgot to cancel Thursday. A keyword list didn’t.

Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady Submission 🦥

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

🦥 TL;DR. Paste a week of volunteer chat messages. Gemma 4, running locally in Ollama, copies the words that matter ("Tuesday", "after 3", "if my work shift gets moved"). Plain Python turns them into dates, times and a day-by-day history. You accept or reject each suggestion beside the message it came from, and only what you accept reaches the final sheet.

▶️ 5:48 demo · 🧪 Try the replay in your browser · 💻 Code, MIT licence

Thursday, 9:10 a.m.

On Monday, Maria posts in the volunteer group chat:

I can do Tuesday and Thursday mornings.

On Thursday she posts again:

Change of plans — Thursday doesn't work anymore, sorry! Tuesday morning still good. I can add Friday 2-5.

There are 48 more messages that week. Priya can do "Monday after 3". Kofi "might be able to do Thursday afternoon if my work shift gets moved". Rui writes "Next Tuesday" on a Monday, which could mean tomorrow or the Tuesday after. Somebody has to read all of it, remember who changed their mind, and type it into a sheet. Then somebody changes their mind again.

Riverside Community Pantry doesn't exist. The pantry, its 18 volunteers and all 50 messages are invented, so this could be built and tested without touching anyone's real chat. But if you've ever coordinated volunteers, you've lived this week.

What I Built

Availazy is a local review tool for the person who asks "when can you help?" and then has to make sense of the answers.

Availazy landing page: Local AI for Coordinators Who Have Better Things to Do

Availazy turns volunteer messages into suggestions you can review beside their source.

You paste dated chat messages (or a WhatsApp text export) and your volunteer list, pick the week, and Availazy:

  • 📝 suggests availability records, each shown beside its message with the quoted words highlighted;
  • 🔁 keeps everyone's history day by day, so a correction replaces the earlier answer, a cancellation removes it, and a "maybe" stays a maybe until the person confirms;
  • ✋ refuses to guess. Anything the words don't settle goes on a Needs a decision list;
  • ✅ exports a final sheet built only from what you accepted. It says complete only when every message is marked reviewed, nothing is pending and nothing is unresolved. Otherwise it says provisional, and it still downloads.

It records availability, not shift assignments. The person who gets to be lazy is the coordinator: they stop retyping and start deciding.

Demo

For readable interface details, select ⚙️ → Quality → 1080p HD.

The video opens with the local app: Gemma 4 reads two invented messages live, through Ollama, on my laptop. Then it switches to the hosted replay for the full week.

Try it yourself: https://availability-ledger-replay.onrender.com/app. This is a replay, not live inference. It serves the fictional week with Gemma's answers saved in advance, so it runs on free hosting without a model. Reviewing, history, the chart and both exports all work; pasting your own messages is switched off. Free hosting sleeps when idle, so the first visit can take about a minute.

There's also an illustrated guide with diagrams at /docs.

Gemma missed Maria's cancellation. That's the point.

When Gemma read Maria's Thursday message in the saved run, it got her Tuesday right and her new Friday right, and it missed the cancellation completely.

So Availazy doesn't trust one reader. In the default mode, a plain keyword reader reads every message alongside the model. When the rules see a cancellation the model didn't, the app adds it, flagged, so I can check it:

Maria's change of plans message with three suggestions; the Thursday cancellation is marked Added by the rules

Gemma found Tuesday and Friday. The rules added the Thursday cancellation for review.

Until I accept it, the sheet shows Thursday as a pending change. One click on Accept and Thursday is gone from the final sheet. In Model only mode, the same replay leaves Maria on Thursday morning. That one message is the whole argument for running two readers.

How I Built It: the model reads, code decides

The design is one strict split:

message ──▶ Gemma 4 E2B (Ollama, local) ──▶ copied words + a proposed status
        └─▶ keyword rules ─────────────────▶ a second opinion
                         │
                         ▼
     Python: who, which date, which hours, does the quote exist,
             what does this replace, what is still unsure
                         │
                         ▼
     You: accept / reject / edit, mark each message reviewed
                         │
                         ▼
     Final sheet (accepted records only) + review log
Enter fullscreen mode Exit fullscreen mode

1. Gemma copies words; it doesn't do dates. For each message, Gemma fills a JSON schema that Ollama enforces. This is its real saved answer for Maria's "Tuesday morning still good":

{"person": "SENDER", "day_text": "Tuesday", "time_text": "morning",
 "status": "available", "condition": "", "replaces_earlier": false,
 "excerpt": "Tuesday morning still good"}
Enter fullscreen mode Exit fullscreen mode

No date and no hours, just the words. Gemma didn't even mark it as a replacement ("replaces_earlier": false). Python spotted "Change of plans" and marked Maria's new answers as replacing her old ones, with a visible note on each card. Python also works out that "Tuesday", sent on Thursday 5 November for the week of 9 November, means 10 November, and that "morning" means a fixed window:

DEFAULT_PERIODS = {
    "morning": ("09:00", "12:00"),
    "afternoon": ("13:00", "17:00"),
    "evening": ("17:00", "20:00"),
}
Enter fullscreen mode Exit fullscreen mode

Open ranges are code too. "After 3" runs to 20:00, the end of the planning day, and the card says so:

Priya's message becomes Monday 15:00-20:00 and Wednesday 13:00-17:00, with the note that after 3 runs to 20:00

“After 3” becomes 15:00–20:00, with the assumption written on the card.

2. Every quote is checked against the message. If the model's quote and its day/time words are both missing from the message, the suggestion is not applied unless you choose "Use it anyway". If only some words are missing, it's flagged.

3. History is kept in order. Later messages change only the days and times they mention. Kofi's "if my work shift gets moved" stays conditional until his later "my shift got moved, so Thursday afternoon is a yes". Olu's "I have to drop out for the rest of this week", sent on a Wednesday, cancels Wednesday through Sunday, taking out his Thursday evening and Saturday morning:

Changes over time: replacements and cancellations in order, each linked to its message

An excerpt of the change history, with replacements and cancellations linked to their source messages.

4. When the words don't settle it, it asks. "Next Tuesday" sent on a Monday could be two dates. "This Saturday" sent the week before is outside the week. A name that matches nobody on the roster isn't guessed:

Needs a decision list with ten items, including next tuesday that could be two dates and a date outside the week

Ambiguous dates, unmatched names and unresolved conditions stay on the decision list.

5. The calm wrong answer. Marta asks: "Is Maria still doing Tuesday morning? If not I can take it." Gemma filed Marta's offer under Maria, as a conditional Tuesday. It's wrong, and it looks perfectly reasonable. That's why every suggestion sits beside its message, and why this one carries "Only one of the two readers found this":

Marta's message, with the model's suggestion wrongly filed under Maria Lopez and flagged Only one of the two readers found this

Marta wrote the message, but Gemma assigned the offer to Maria. Reviewing the source makes the mistake visible.

One click on Reject and Maria's Tuesday is firm again.

Update, 24 Oct: after a reader's comment, Availazy now also warns when a suggestion is for someone other than the sender, or when the message names another volunteer. Both warnings block one-click accept, so it now takes 36 suggestions instead of the 39 shown in the video.

6. Only accepted records reach the sheet. Here are the first rows of the real export from the replay, after I accepted Maria's cancellation, rejected the m48 mix-up, accepted the 36 warning-free suggestions and marked two messages reviewed:

# availability-final v1; week=2026-11-09; status=provisional; unreviewed=m01 m03 … m50; pending=24; blocking_issues=0
volunteer,date,start,end,status,condition,cell_state,sources
Maria Lopez,2026-11-10,09:00,12:00,available,,confirmed,m28.1
Maria Lopez,2026-11-13,14:00,17:00,available,,confirmed,m28.2
Marta Lopes,2026-11-09,09:00,12:00,available,,needs_review,m17.1
Enter fullscreen mode Exit fullscreen mode

(The unreviewed list is shortened here.) Maria has Tuesday and Friday, and no Thursday. Her messages are marked reviewed, so her cells are confirmed. Everyone else is needs_review, and the whole file honestly says provisional.

The stack is boring on purpose: a Python standard-library server, plain HTML/CSS/JS with no framework or CDN, and Ollama on 127.0.0.1. The only optional dependency is sentry-sdk, for opt-in tracing.

The Lazy Payoff 🦥

Here's the fictional week, by the numbers, from the replay:

Messages 50 (one exact repeat is collapsed automatically)
Suggestions, each beside its quote 67
Accepted with one click (no warnings) 36
Left for a real decision (a warning, or not applied) 31
Items on the Needs a decision list 10
Gemma + rules reading the week on my 8 GB Apple M2 131 s end to end (one dated run, not a benchmark)

People available chart after the one-click accept: accepted, pending and conditional slots for the week

After the one-click accept: dark green is accepted, light green still waits for review, striped orange is conditional.

So the work shifts from reading 50 messages and typing a sheet to checking 31 flagged suggestions, each with its quote right next to it. The model reads the whole week in about two minutes: long enough to make a cup of tea.

I haven't timed a real coordinator, so I won't claim minutes saved. The repo has a written protocol for that study; it hasn't been run.

How well does it work?

I didn't want to grade my own homework, so a separate ChatGPT session wrote a test week I'd never seen (set C: 24 fictional messages; I rewrote five of them). Its files were frozen by hash before any implementing agent saw them, and then it was scored once. That first score is kept exactly as it came out:

Reader Person-days right Silent errors Cancellations missed on the sheet Sheet agreement with the answer key
Keyword rules only 15/20 5 4/8 0.433
Gemma 4 E2B only 17/20 2 4/8 0.403
Gemma 4 E2B + rules 17/20 2 2/8 0.527

All suggestions were left pending and nobody reviewed them: this scores the draft sheet before a human touches it.

Four things I didn't expect:

  1. The keyword list earned its seat. On the sheet, rules alone (0.433) actually beat Gemma alone (0.403). Gemma read more person-days right, but each reader caught things the other missed. Together they halved the missed cancellations.
  2. "Mostly right" can still make a wrong sheet. Gemma + rules read 17 of 20 person-days right, yet the sheet matched only about half the time slots. The biggest culprit was "Please change my availability on monday to 1-4 pm". The hours were right, but nothing treated it as a replacement, because "change" wasn't one of my replacement words. So the old all-day Monday stayed underneath.
  3. Every fix went into Python, not the prompt. The prompt is the same version that was scored. Five deterministic fixes followed: explicit dates beat weekday words, "change" became a replacement cue, a firm follow-up settles a maybe, "T. Vekar" is an ambiguity rather than a stranger, and the rules learned more negations. Re-scored from the same saved Gemma outputs, set C now reaches 1.0, but I label that post-hoc: I fixed what I'd seen, so it proves little about new messages.
  4. Honest numbers are less impressive and more useful. An earlier set B (20 messages) scored 30/35 person-days and 0.776 sheet agreement on its first scoring. Every frozen table, every caveat and every after-the-fact change is in RESULTS.md.

Built with

  • 🧠 Gemma 4 E2B (gemma4:e2b-it-qat, Apache 2.0) in Ollama. About a 4.3 GB download and about 1.6 GB of memory once loaded on my 8 GB Mac, so it runs alongside a browser. It's the reader in the local app, and it produced every saved answer in the replay.
  • ☁️ Render hosts the fictional replay, so judges can click through a full week without installing anything. The server runs in a demo mode that refuses any message, roster or week it didn't serve, and it never calls a model.
  • ✅ GitHub Actions runs on every push and pull request to main, with three jobs: 128 unit tests (8 tracing tests skip without the optional SDK); a SHA-256 check that the frozen evaluation results haven't been touched; and a re-score of all three test sets from the saved model outputs, with no model calls. Run #29 passed all three jobs.
  • 🔭 Sentry Agent Tracing is opt-in and off by default. When on, two allowlists in my adapter limit what's sent to structure: timings, token counts, model and stage names. Message text, names, quotes, prompts and replies are excluded, and offline tests check that planted fake content never appears. The traces earned their keep. In one run, four model calls hit the 300-second client deadline waiting for Ollama. A later run made 49 calls in 1.42 minutes with no errors. (The traces don't say why the earlier calls stalled, and I won't pretend they do.) One caveat I found: received traces showed location (geo) attributes that my code doesn't send. After a settings change, the four I inspected appeared filtered, but I haven't audited everything Sentry receives (dated evidence).

Sentry timeline: four Gemma calls in a row each last 5 minutes and fail; the selected call shows No input for this span

One traced run: four Gemma calls in a row each hit the 5-minute client timeout. The selected call shows "No input for this span": no message text was sent.

GitHub Actions run 29: unit tests, protected evidence check and cache-only re-score all passed

Every push to main runs three jobs: unit tests, a SHA-256 check of the frozen evidence, and a re-score from saved Gemma replies with no model calls.

Render dashboard: free availability-ledger-replay web service, Live, auto-deploying each push to main

The replay is one free Render web service, managed by a Blueprint. Every push to main auto-deploys it, including the commit that added the new sender warnings (84d5fec).

Availazy

Local AI for Coordinators Who Have Better Things to Do. Let AI read. You decide.

Availazy was called Availability Ledger until 2026-10-22 (BRAND-01). The repository and Render service keep their existing availability-ledger identifiers; the Sentry organization remains availability-ledger and its project remains python. Older records in this repository use the old product name.

A local review tool for volunteer coordinators who collect availability through ordinary messages. You paste a week of dated chat messages and your volunteer roster; the tool suggests availability records, shows each one beside the message it came from, and keeps a per-person, per-day history of offers, corrections, cancellations and conditions. You review the suggestions, explicitly mark each message reviewed, and export a reconciled final sheet built from accepted records. A separate review log preserves the record trail.

It records availability (when people say they can or cannot help), not shift assignments.

A local open…




Why Does Open Innovation Matter?

Volunteer availability is quietly personal. "I'm at a wedding." "If my work shift gets moved." Read together, a week of these messages maps people's families, jobs and lives. Here's what an open model on the coordinator's own laptop makes possible:

  1. The messages stay on the computer. In the local app they go only to Ollama on 127.0.0.1. There's no cloud AI service in the loop, and optional tracing is off unless you turn it on.
  2. No account, no API key, no per-message bill. Once Ollama and the model are installed, it works offline. A small food pantry can afford "free forever".
  3. It fits the hardware people actually have. An 8 GB laptop is enough, so the volunteer coordinator doesn't need a GPU or a budget.
  4. It can be checked. The prompt, the schema, the rules, the test sets and the frozen scores are all in the repo. Anyone can re-score them from the saved outputs without running a model, which is exactly what CI does.
  5. It can be swapped. The model is a menu choice of whatever Ollama has installed. I also evaluated Qwen3.5 4B against the same answer keys.

A hosted API might be smarter. But the right answer to "where do my volunteers' messages go?" should be "nowhere".

What it can't do (yet)

  • It has only seen fictional messages. No real coordinator has used it, and the test sets share an author with the code, except set C.
  • Times are read as written; the timezone field is stored but not converted. "At 9" isn't read.
  • Up to 20 volunteers, one browser tab, and nothing is saved between sessions.
  • The keyword reader still misreads some phrasings. It read "Saturday 9–12 if it isn't raining" as unavailable instead of conditional, which is exactly why every suggestion sits beside its message.
  • Not tested in Safari, Firefox or with a screen reader.

Prize Categories

  • Best Use of Gemma: Gemma 4 E2B is the local reader, held to a strict "copy the words, code does the maths" contract, measured against a rules baseline and a frozen test set written by a separate AI session.
  • Best Use of Render: the interactive replay that lets anyone review a full fictional week without installing a model.
  • Best Use of GitHub Copilot (via GitHub Actions): unit tests, a frozen-evidence integrity check and a cache-only re-score of every test set on every push and pull request.
  • Best Use of Sentry Agent Tracing: opt-in, content-free model-call traces that surfaced the 300-second timeouts.

Slow and steady: the model reads, the code reconciles, and a person decides. If you coordinate volunteers, how do you collect availability today, and what's the message that always breaks your spreadsheet? 🦥

Top comments (15)

Collapse
 
robertpaschal profile image
Odinaka Robert Nnamani •

The part I would steal for my own work is not the Gemma-plus-rules split, it is keeping that first frozen score exactly as it came out. I have watched teams quietly re-run a test after a fix and then report the improved number as if it were the first result, and the whole comparison quietly stops meaning anything. Labelling the set C rerun post-hoc, after the Python fixes, is what makes the 1.0 honest rather than impressive. Do you keep the frozen run and the post-fix run side by side in the repo, so a reader can see both without trusting your summary?

Collapse
 
bipul_chamoli_09b5f0cc719 profile image
Bipul Chamoli •

Thanks, that was exactly the trap I was worried about. It’s easy to fix things, re-run, and quietly let the new number become “the” number.

Yes, both are in the repo as separate files, so you don’t have to trust my summary (github.com/bipul724/availability-ledger):

The first score is in evaluation/results/frozen-setC/. Set C was scored once early on 21 Oct, and those files were committed later that morning, before any of the fixes. Git shows no change to them since. Their hashes are in evaluation/protected.sha256, and CI fails if any of them changes or if a file is added to or removed from that folder.
The post-fix numbers are separate files in evaluation/results/, from a later commit. They were re-scored from the same saved Gemma replies, with zero new model calls.
RESULTS.md puts them side by side (0.527 → 1.0) and lists which fix moved which message.

You can check it yourself without a model: python3 -m evaluation.run --set C --extractor hybrid --model gemma4:e2b-it-qat --cache-only --out /tmp/c replays the saved replies and should show the post-fix result (slot agreement 1.0, 0 fresh calls). CI runs the same re-score on every push and fails if any score differs from what’s committed. If you check out the commit where the first score was frozen and re-score from the same replies, you get the original numbers back exactly (slot agreement 0.433 rules only, 0.403 Gemma only, 0.527 Gemma + rules).

Your question actually made me find a gap. The raw Gemma replies for set C were committed once and never touched, but CI wasn’t hashing them, and it wasn’t checking the set C fixtures against their manifest either (the manifest was committed before the scoring run). So you’d have been trusting git history there. I’ve now added both to the CI integrity check.

And I agree about what the 1.0 means. It’s me fixing failures I’d already seen, so it says very little about new messages. The honest next step is a fresh set D, written and frozen the same way, and scored once. I haven’t done that yet.

Collapse
 
robertpaschal profile image
Odinaka Robert Nnamani •

The CI integrity check is the right fix, and the part I would still pin down is the manifest itself: it was committed before the scoring run, so if the manifest can be regenerated after a fixture change, the check passes while the evidence moved. I would hash the manifest in CI and fail on any regeneration, not just on a mismatch. The same logic applies to set D: freeze the raw Gemma replies before the extractor is touched again, so model drift and extractor fixes stay separable. That way, if set D also reaches 1.0, it is measured on outputs the extractor never saw during fixing. Will set D freeze its raw replies before any extractor change, or will the extractor and the set evolve together again?

Thread Thread
 
bipul_chamoli_09b5f0cc719 profile image
Bipul Chamoli •

Good point, and the manifest part is already pinned. fixtures/setC.manifest has had its own line in evaluation/protected.sha256 since the integrity check was added on 21 Oct. So regenerating it after a fixture change fails CI, even if the new manifest and the fixtures agree with each other. The manifest itself has a single commit, from before the scoring run.

The one thing CI can’t stop is someone editing the hash list and the manifest in the same commit. That’s where it falls back on public git history. The hash list has had two commits so far, and both only added lines; no existing hash has ever changed. A regeneration would show up as a changed line in that file’s diff.

On set D: yes, raw replies first. Set C already went in that order in practice: its raw Gemma replies were committed with the first score, before the first extractor fix. The pipeline is split that way too. Replies are cached per message and everything after them is Python, so a cache-only re-score never calls the model. For set D I’ve made the order explicit:

1.commit the manifest and pin its hash;
2.run Gemma once;
3.commit the raw replies and the first score, with their hashes added to CI;
4.only then touch the extractor.

That way, if set D also reaches 1.0 later, it’s measured on replies the fixes never influenced. A model change would show up as different replies, not as a different score from the same replies.

Honest status: set D doesn’t exist yet. It’s an open task, and your “freeze the raw replies before any extractor change” order is now written into it.

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

The append-only hash list is the part that settles it for me: two commits, both only adding lines, means a same-commit rewrite cannot hide, it shows up as a changed line in the diff. I have leaned on that same property in a deploy log where append-only history was the only witness I trusted. For set D, will you also freeze the extractor version alongside the raw replies, so a later extractor change cannot quietly re-score the same replies?

Thread Thread
 
bipul_chamoli_09b5f0cc719 profile image
Bipul Chamoli •

Yes. Today the result files record the model and the prompt version, but not the code commit. For set C, the extractor version is written in RESULTS.md’s provenance (the exact commit), and the reproduction I mentioned works because git still has that commit. But that’s prose plus git history, not something pinned in the result file itself.

For set D I’ve added it to the task: the first-score files will name the exact commit that produced them, and that commit has to reproduce the frozen score from the frozen replies, the same way the set C one does.

On “quietly re-score”: a later extractor can re-score the same replies, and it should, because that’s how the post-hoc numbers get made. What it can’t do is replace the first score. CI re-scores into a temporary folder and compares against the current results files, while the frozen ones are hash-locked separately. So a new extractor can only add a new, separately labelled number next to the old one, never overwrite it.

Appreciate you pushing on this. Append-only history being “the only witness I trusted” is a good way to put it.

Thread Thread
 
robertpaschal profile image
Odinaka Robert Nnamani •

The "new number next to the old one, never over it" rule is the part that settles it for me. I have leaned on append-only history as the only witness I trusted in a deploy log, and the failure I keep seeing is not a bad re-score, it is a quiet replacement — the old number gone, so nobody can tell what changed. Freezing the commit in the first-score file closes that hole. Does the frozen first score also freeze the hash of the reply set itself, so a changed reply set cannot silently re-baseline the same score?

Collapse
 
utkarsh_rastogi_2e63a7b09 profile image
Utkarsh Rastogi •

The bit where Gemma completely missed Maria’s cancellation and the keyword rules caught it is the most convincing part of the post. Two dumb-ish readers beating one smart one is a real lesson.

Curious how you decide when the two readers disagree on a normal availability (not a cancellation). Do both versions show up, or does one win by default?

The “change” word bug is brutal, and it’s exactly the kind of thing you only find with a test set you didn’t write yourself. Hours right, sheet wrong.

Curious if you thought about letting Gemma flag replacements too and having the rules confirm, the same way cancellations work?

Collapse
 
bipul_chamoli_09b5f0cc719 profile image
Bipul Chamoli •

For normal availability, Gemma’s version is the default proposal. If the readers disagree on the hours/status for the same person and day, it gets a “Model and rules read this differently” warning.

The UI shows that proposal beside the original message, but doesn’t show the rules’ alternative alongside it yet. The reviewer can edit, accept or reject it; disagreements stay out of bulk acceptance. Only accepted records reach the final export.

On replacements, Gemma actually already has a replaces_earlier field, and the prompt explicitly mentions “change”! On that message it returned the correct hours but set the flag to false. My keyword fallback recognised “change of plans,” but missed bare “change,” so both paths failed together.

The fix widened the Python replacement cues. When that fallback removes earlier hours, it now shows “Earlier answer replaced — check,” including what was removed. Either the model hint or a code cue can trigger a provisional replacement; I don’t require agreement from both.

Those cues can overreach too, so human review still matters. The original failing score stays frozen, and the fix is labelled post-hoc.

Collapse
 
utkarsh_rastogi_2e63a7b09 profile image
Utkarsh Rastogi •

nice, keeping disagreements out of bulk accept is a good call. one small thing: when it says “model and rules read this differently”, showing the rules version right under gemma’s would make it a one click decision instead of reading the message again. feels like it’d cut a lot of the 28 flagged ones down to quick picks

Thread Thread
 
bipul_chamoli_09b5f0cc719 profile image
Bipul Chamoli •

That’s the comparison the current UI is missing. Showing Gemma’s version and the rules’ version together, with a button to choose either, would make straightforward disagreements easier to resolve. I’d keep the original message visible and an edit/reject option for cases where both are wrong.
One caveat: the 28 flagged suggestions also include ambiguous names, dates and other warnings, so this wouldn’t simplify all of them. I’d want to measure review time before claiming how much it saves.

Collapse
 
gun_acd39d460030863f9c63e profile image
Gun •

The four calls hitting the 300s timeout back to back is interesting. Was Ollama still loading the model, or was some other process using up the RAM in the 8 GB Mac? Curious whether a warm-up call before the batch would have avoided it.

Collapse
 
bipul_chamoli_09b5f0cc719 profile image
Bipul Chamoli •

I couldn’t establish the cause. The failures happened late in the batch, at positions 37–40, after earlier calls had completed. The code already sends keep_alive: "30m", although there’s no separate warm-up step.

I did see a macOS memory warning naming Claude and ChatGPT on that 8 GB Mac. After closing other apps, restarting Ollama and the app server, and running a two-message check, a later full run completed 49 calls with zero errors in 1.42 minutes. But that small check had already warmed the model, so several things changed together.

I didn’t capture RAM/swap during the stalls, and the timed-out calls didn’t return Ollama’s load-time breakdown. Memory pressure is plausible, but I can’t attribute the failures to it.

A warm-up is worth testing; I haven’t demonstrated that it alone would have prevented them.

Collapse
 
gun_acd39d460030863f9c63e profile image
Gun •

makes sense, failing at 37-40 after everything before worked doesn’t look like a loading issue. one cheap thing for next time: ollama returns load_duration and eval_duration on the calls that do succeed, so logging those per call would show if things were slowly getting slower before the stall. plus memory_pressure or vm_stat running in another terminal. then you’d actually have the data instead of guessing

Thread Thread
 
bipul_chamoli_09b5f0cc719 profile image
Bipul Chamoli •

Yes—the missing piece is lining those signals up. The tracing adapter already records load, prompt and generation times for successful calls as load_s, prompt_s and output_s, but I haven’t established whether they were trending upward before the stall.

I also didn’t capture memory pressure or swap activity alongside that batch. Correlating those timings with memory_pressure/vm_stat would make the next investigation much more useful. It would show what preceded the failures, without treating the successful retry as proof of the cause.