Skip to content

DEV Community

InferenceKey

Open models on your own GPUs: experiments, measurements and the exact commands behind them. Full write-ups at inferencekey.com

inferencekey.com Joined on Oct 14, 2026

Oct 20

Running DeepSeek-V4.1-Flash (763B) on one 96 GB GPU: the three settings that decide your speed

#ai #llm #gpu #opensource

3 min read

Oct 19

Serving Iris-3B Turbo: the 42 GB repo, the 21 GB of VRAM and the 8 MB payload limit

#ai #machinelearning #python #opensource

5 min read

Oct 17

Serving a 125B coding model to a team from one RTX 5090: what broke and what we tuned

#ai #llm #gpu #opensource

3 min read

Oct 17

Kandinsky 6.0 in a pipeline: offload, the untimed default and what to measure

#ai #machinelearning #opensource #gpu

4 min read

Oct 16

A 763B MoE on two 96 GB GPUs: put the idle experts in host RAM, then measure per stream

#llm #gpu #machinelearning #selfhosted

4 min read

Oct 15

Same config.json, different serve command: what to check before serving AREX-2

#ai #llm #machinelearning #gpu

5 min read

Oct 14

Three things that break when you point Claude Code at your own model

#ai #llm #selfhosted #opensource

4 min read

loading...