Skip to content

DEV Community

Edge AI in Practice Series' Articles

Back to GINIGEN AI's Series

Cover image for Running a 180B Model on a Laptop With No GPU: How 4-bit GGUF Keeps Full Accuracy

GINIGEN AI

Oct 6

Running a 180B Model on a Laptop With No GPU: How 4-bit GGUF Keeps Full Accuracy

#ai #llm #edgeai #quantization

6 min read

Cover image for Building an AI That Refuses to Say "Safe": Reliability Engineering for an Offline Disaster App

GINIGEN AI

Oct 6

Building an AI That Refuses to Say "Safe": Reliability Engineering for an Offline Disaster App

#ai #edgeai #llm #android

7 min read

GINIGEN AI

Oct 6

4-bit GGUF Quality for MoE Models: Why Only 3B of 180B Params Fire, and How to Prove Parity

#ai #llm #quantization #edgeai

7 min read

GINIGEN AI

Oct 11

One Self-Improving Model, Eleven Number-One Titles: What That Takes

#ai #machinelearning #llm #opensource

2 min read

GINIGEN AI

Oct 12

Zero-Token Decisions in the Browser: On-Device Multimodal Classification with EmbeddingGemma 2 and WebGPU

#ai #machinelearning #webgpu #opensource

3 min read

Cover image for LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget

GINIGEN AI

Oct 19

LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget

#ai #machinelearning #edgeai #quantization

11 min read

Cover image for 0.1 Bits Per Weight Is Not 0.1 Bits Per Weight: Where the Bytes Go in Sub-2-Bit LLMs

GINIGEN AI

Oct 21

0.1 Bits Per Weight Is Not 0.1 Bits Per Weight: Where the Bytes Go in Sub-2-Bit LLMs

#ai #machinelearning #edgeai #quantization

14 min read

Cover image for The Inference Hardware You Already Own Might Train Models Too: A 325M Pretraining Run on an NPU

GINIGEN AI

Oct 26

The Inference Hardware You Already Own Might Train Models Too: A 325M Pretraining Run on an NPU

#ai #machinelearning #edgeai #hardware

3 min read