DEV Community

AI Tech News profile picture

AI Tech News

404 bio not found

Joined Joined on  github website twitter website
One Fleet for Serving and Training? What a 325M Pretraining Run on an Inference NPU Means for AI Infrastructure
wolfsea2357 profile

One Fleet for Serving and Training? What a 325M Pretraining Run on an Inference NPU Means for AI Infrastructure

#ai #machinelearning #infrastructure #hardware
Comments
2 min read

Want to connect with AI Tech News?

Create an account to connect with AI Tech News. You can also sign in below to proceed if you already have an account.

Create Account
Already have an account? Sign in
AI-automated attacks are forcing application security to automate defense
wolfsea2357 profile

AI-automated attacks are forcing application security to automate defense

#security #ai #appsec #devops
Comments
1 min read
KV Cache Offloading to NVMe: The Break-Even Bandwidth Math and Where It Does Not Pay Off
wolfsea2357 profile

KV Cache Offloading to NVMe: The Break-Even Bandwidth Math and Where It Does Not Pay Off

#ai #machinelearning #vllm #llm
Comments
14 min read
Speculative Decoding at High Batch Size: Where the 3x Speedup Turns Into a Tax
wolfsea2357 profile

Speculative Decoding at High Batch Size: Where the 3x Speedup Turns Into a Tax

#ai #machinelearning #vllm #performance
Comments
12 min read
Prefill/Decode Disaggregation in vLLM: The KV Transfer Math That Decides If It Pays Off
wolfsea2357 profile

Prefill/Decode Disaggregation in vLLM: The KV Transfer Math That Decides If It Pays Off

#ai #machinelearning #llm #vllm
Comments
11 min read
On-Device LLM Inference: The Tradeoffs Nobody Puts on the Spec Sheet
wolfsea2357 profile

On-Device LLM Inference: The Tradeoffs Nobody Puts on the Spec Sheet

#ai #machinelearning #llm #edgeai
Comments
3 min read
KV Cache Quantization in LLM Serving: FP8 and INT8 Tradeoffs, the Silent config.json Trap, and How to Measure It Fairly
wolfsea2357 profile

KV Cache Quantization in LLM Serving: FP8 and INT8 Tradeoffs, the Silent config.json Trap, and How to Measure It Fairly

#llm #vllm #performance #devops
Comments 1
7 min read
What Does It Actually Cost to Self-Host an LLM? The Batching Math Nobody Shows You
wolfsea2357 profile

What Does It Actually Cost to Self-Host an LLM? The Batching Math Nobody Shows You

#llm #vllm #performance #devops
Comments
6 min read
Five Ways Your LLM Serving Benchmark Is Lying to You (and How to Catch Each One)
wolfsea2357 profile

Five Ways Your LLM Serving Benchmark Is Lying to You (and How to Catch Each One)

#llm #vllm #performance #devops
Comments 1
15 min read
A 4B Model Just Beat 8B — We Tested 18 Small LLMs and the Results Are Wild
wolfsea2357 profile

A 4B Model Just Beat 8B — We Tested 18 Small LLMs and the Results Are Wild

#ai #machinelearning #opensource #benchmark
Comments
6 min read
MARL: Runtime Middleware That Reduces LLM Hallucination Without Fine-Tuning
wolfsea2357 profile

MARL: Runtime Middleware That Reduces LLM Hallucination Without Fine-Tuning

#ai #llm #python #opensource
Comments 2
4 min read
loading...