I own two laptops. A 2020 Intel MacBook Air, 8GB RAM, no unified memory, gifted by my sister. And a Windows machine: Intel i5 11th Gen, integrated ...
For further actions, you may consider blocking this person and/or reporting abuse
Hey, this article appears to have been generated with the assistance of ChatGPT or possibly some other AI tool.
We allow our community members to use AI assistance when writing articles as long as they abide by our guidelines. Please review the guidelines and edit your post to add a disclaimer.
Failure to follow these guidelines could result in DEV admin lowering the score of your post, making it less visible to the rest of the community. Or, if upon review we find this post to be particularly harmful, we may decide to unpublish it completely.
We hope you understand and take care to follow our guidelines going forward!
🤣🤣🤣
Sloan are rated E for everyone :)
It is bordering on personal vendetta at this point. Francis has a personal vendetta against me.🤣🤣
To be honest, we can’t jump to quick conclusions about who’s using this sloth account. Still, it really brings back memories of our interactions back when we were dealing with this situation together. I miss those moments, hahaha.🤣
Yeah I remember vividly like it was yesterday.it was mad sick.🤣🤣
The thing is, you rent cloud, but you still need your interface. A laptop running an IDE, using a cloud model, still uses power. The real question is the delta, how much do you save per hour, on average, over a month, vs running local inference. But in all honesty, the conversation goes out the window when you consider Qoder's Lite model is pretty damn impressive and at $20 a month for a sub, you get 2000 credits (for eg. qwen, minimax, kimi, etc.) and access to Lite, for free... And I've pushed millions of LOC through it, I dont even wanna know how many tokens it ran through, all I know is it was likely over 30m+, especially cuz I ran multi-agents and have each agent spawn sub-agents, across 2 PCs and 2 laptops at once... At any given time, you'll see atleast 16+ agents running at once... You cant get that in hardware for $20 a month, nor can you find that anywhere else on cloud?
You're right about the delta. My laptop still draws power, even with cloud inference. I didn't run that math in the piece.
Qoder's price is impressive. $20 a month for that much throughput beats anything I could build locally.
My real point was different. It's about the days the network or the power fails, not the price per hour. Cloud wins on cost almost every time. It doesn't help when the connection drops mid-task. That's the gap local fills for me.
There's also the model choice itself. Some local models skip the usual guardrails. No cloud subscription gives you that, no matter the price.
give qoder's a try, it's pretty solid for stepping over guardrails, I mean I had it apply for jobs for me, using resume helper to generate the CV, it browsed the web to find jobs and while it blocked the actual submission on my behalf on principle, it's confirmation and non-blocking, so dunno bout your exact use cases, but for me, I found that anything that blocked it can be overridden.
True on the power cut, but how long does a laptop's battery last you running local inference? Accounting for battery life degradation, the total downtime is closer, especially when you compare vs having a fallback network, like a LTE router?
Fair question on battery. Sustained local inference drains a laptop fast, maybe an hour or two on battery, not a full day. Degradation makes that worse over time.
An LTE router as backup is real infrastructure though, not free. Data costs money here, and coverage still drops sometimes. It's a fallback, not a guarantee.
So the honest comparison isn't local vs cloud. It's which failure mode you'd rather plan around. I'd still rather own the failure mode than depend on someone else's uptime.
True, in my case I was looking at local/portable inference, so I can work when in the middle of nowhere without internet, a bit different than working without power.
That being said, have you heard of Strix Halo and RTX Spark? AMD and Nvidia's take on 'ai inference laptops'
Fair enough, no internet and no power are different problems.
Hadn't heard of either before this. Strix Halo gets close to 100GB of usable memory on a laptop chip, at 45-80W instead of the 450W a desktop GPU pulls. Real room for big models, on battery, off-grid.
Spark leans more toward raw compute and CUDA.
Strix Halo might beat the Mac swap for what you're describing here.
Good article and honesty it's a rare perspective that I like to see!
Cloud dependency creates a single point of failure. I am not against cloud as such.
API pricing requires guaranteed network stability.
Local inference provides a mandatory structural fallback.
True enterprise architecture plans for degradation.
If you do not own the failure mode, you do not control the uptime.
Please do continue to write so wonderful articles !
"If you do not own the failure mode, you do not control the uptime" is a better version of the whole point than anything I wrote. Stealing that line, credit to you.
Appreciate the read, thank you.
Steel away.
I got so much value out of reading your content it's the biggest robbery there ever was.
Quiting Carli Anchelotti from Real Madrid when they got Jude Bellingham for "very cheap"
It was like 103 Million without Bonuses and 133 Million with Bonuses.
All I am saying is : I got my money's worth a long time ago from picking your brain 🧠 😃
Ali, Bellingham at 103M looks cheap now but so does most business before the market catches up. Value shows up late, not on day one. Real Madrid found that out.
Appreciate you sticking around long enough to make that comparison because my comment section's still waiting on its Bellingham moment.
Glad you got yours early though even if it cost you nothing but scroll time. 😁😁
I’m currently at a municipal‑level public library in my local urban area. I get free access to books and free public Wi‑Fi, with virtually no power outages or network drops. Comparing this setup to yours, I really don’t know what to say.
I own two laptops locally. One is equipped with a GTX 4070 Super, 8 GB VRAM and 32 GB RAM — I originally bought it purely for 3D gaming. The other carries an AMD 880M integrated GPU with 32 GB RAM. There was a period when I was deeply into locally deploying all kinds of 4B, 7B, 9B, and even MoE models, running all sorts of stress‑limit benchmarks. Right now, the only model that strikes a decent balance between speed and output quality is gemma‑4‑e4b‑qat.
Major local AI players here are quite competitive: Traework, Workbuddy, Qoder, QwenWork, Doubao and so on. You basically get a large pool of credits just for downloading and signing up, plus bonus credits from daily check‑ins. Besides their official large models, these platforms support hooking up your own local LLMs. I’ve connected gemma‑4‑e4b into that workflow, and it works reasonably well.
I’m the sort of person who takes advantage of free resources whenever possible. I’ve assigned all these AI agents the same working directory, and clearly defined what they can and cannot do. They are only allowed to offer suggestions and solution drafts. Final decision‑making rests first with my Herems agent Daoma, and then with me personally.
That contrast is real and it's the whole point. Stable power and free wifi at a public library is not something I can assume here. That gap is exactly what the piece is about, not a hardware preference.
Gemma-4-e4b-qat is a good call, I hadn't tested that one for the speed/quality balance yet. Will look into it.
The agent structure you describe is smart. Suggestions and drafts only, final call goes through you. That's a good default for any local setup, honestly, not just a resource-constrained one.
Exactly. Right now my workflow is still semi‑automated. I have to manually switch between the web interfaces of various top‑tier large models and keep copying‑and‑pasting content back and forth. I’d really like to automate this part too, but then I wonder what would be left for me to contribute. So for now I’m sticking with this manual hand‑off setup. It’s already great being able to leverage the free compute from web‑based services, hahaha.🤣
Fair enough. Automating the whole loop can remove the thinking part too not just the copy-paste. Manual hand-off keeps you in the decision.
Free compute plus your own judgment in the loop is a good trade not a compromise tbh.
Couldn’t agree more. Human‑in‑the‑loop preserves that hands‑on intuition, even with free web‑based compute 👍
you see my lenovo laptop which used CPU not GPU for my project in AI. I know the feeling
CPU-only local AI is its own kind of patience. What are you running on it?
yeah! I use google colab now. It is better because the notebook is in the cloud base application.
Colab makes sense, free GPU without the CPU wait. Did it replace the local runs entirely or still use both?
I am using a GPU on Google Colab now that I’ve upgraded my plan. The upgrade gives me access to more powerful hardware options, including different GPU models I can choose from depending on my workload.
Interesting!
Hadn't heard about RunPod and Vast.ai, but recently I came across NVIDIA NIM - have you heard about it? I haven't tried it yet, but it might be useful/usable in some scenarios ...
Hadn't heard of NIM before this, just looked it up.
Different layer than RunPod/Vast.ai. Those give you the GPU itself. NIM packages a model with an optimized inference engine and a standard API, but you still need a GPU under it.
There's a free dev tier now, up to 16 GPUs, no paid license needed to test it.
Ah right, I missed that little nuance about "you still need a GPU under it" :P ...
So I read a bit about it - I think what's needed is to sign up to their "NVIDIA Developer Program", and the associated FAQ mentions those 16 GPUs ...
What exactly does that mean - is 1 GPU not enough already, why 16 ? (probably this clearly gives away what a n00b I am, lol)
Not a n00b question, it confused me too at first.
It's not a minimum, it's a ceiling. The free dev tier lets you scale your testing up to 16 GPUs without paying for the enterprise license. 1 GPU is fine to start, even 0 if you're just hitting their hosted API to try it out.
16 is there for people testing larger setups, like running several models at once or simulating a bigger deployment, without NVIDIA charging them for the privilege of testing.
Ah right, thanks for explaining it! I'm pretty eager to try it out, if only to understand how that all works, and when exactly I'd be "hitting" those GPUs, and how many ... sounds like cool stuff to play around with - for free!
That's the right way to learn it, honestly. Spin up the free tier, run a small model, watch what actually happens. You'll understand the GPU count question faster by hitting it once than by reading docs about it.
Let me know what you find, I'm curious too now.
Who say pre M silicon Mac are good at Local inference? Reading your article is just a wasted of time.
This is more like a "you" issue then a general issue
Useless post to be honest
Useless post got you commenting though 😞