Benchmarks - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

techaiwire

Sep 8

GPT-6 Astra scores 95% on one robot task, 10% on another

#benchmarks #openai #anthropic #robotics

4 min read

Peremptory

Sep 7

Claude Formalized Fermat in 11 Days. The Math Isn't New.

#anthropic #research #claude #benchmarks

3 min read

Sep 6

13 of 14 Models Write Messier Code Than the Human Who Fixed the Same Bug

#python #ai #codequality #benchmarks

6 min read

Peremptory

Sep 1

DeepSeek's Vision Flash Plays the Agent Game

#modelrelease #chineseai #agenticai #benchmarks

2 min read

Peremptory

Aug 25

Frontier Models Fail at Research-Level Reasoning

#benchmarks #research #aidevelopment

2 min read

Sep 5

Temperature 0 is not reproducible. I measured 30 percent of my output changing between identical runs.

#llm #benchmarks #localllm #reproducibility

3 min read

Sep 5

Our 4B beat Claude Opus on a 440K-token corpus. Then it came last on the public benchmark.

#llm #benchmarks #localllm #privateai

4 min read

Sep 5

A 4B on a 6GB laptop matched frontier-model accuracy on aggregation — except when the answer is a number

#llm #benchmarks #localllm #agents

4 min read

Sep 5

Your agent truncates the corpus and answers anyway. Two harnesses, and a router that picks between them.

#llm #agents #localllm #benchmarks

5 min read

Peremptory

Aug 21

Frontier Models Hit a Wall on Research Thinking

#research #benchmarks #aidevelopment

2 min read

Aug 18

Ranking Language Models by How Well They Spot Liars

#llm #measurement #benchmarks #statistics

9 min read

Charles

Aug 18

The Benchmarkpocalypse: Why AI Benchmarks Are Broken — and What Dan Luu Says We Should Do About It

#ai #machinelearning #benchmarks #programming

4 min read

Peremptory

Aug 18

Labs are ditching factual knowledge for reasoning speed

#benchmarks #modelreleases #aistrategy #research

2 min read

The AI Downside

Aug 15

Why AI Benchmarks Mean Less Than You Think

#benchmarks #llms #evaluation #hype

6 min read

Aug 15

An AI Capture-the-Flag Tournament: What the Scoreboard Counted

#llm #measurement #security #benchmarks

6 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.