pgai.MODEST PRAISE. PUBLIC NUMBERS.
← Back to the scoreboard

SWE-bench Verified

Data as of 2026-10-14 / manually checked

4 models · 7 benchmarks · zero composite scores

pretty greatpretty okaynot yetTap a verdict for the number and source.

Fixes real GitHub issues in Python repositories.

Why it matters

Tests whether an agent can make a working code change.

How to read it

Percentage of issues resolved. Agent harness matters. Higher is better.

Run by SWE-bench team. Last checked 2026-10-14.

Visit the benchmark source ↗
MODEL / VENDORSWE-bench Verifiedcoding
OpenAI

GPT-5.2 Thinking

Pretty great at coding, abstract reasoning, science, math, long context.

SWE-bench Verified
pretty great
80%

Vendor-reported research evaluation, maximum available reasoning effort. See source for harness and sampling.

Source ↗Observed 2026-10-14
OpenAI

GPT-5.1 Thinking

Pretty great at coding, science.

SWE-bench Verified
pretty great
76.3%

Vendor-reported research evaluation, maximum available reasoning effort. See source for harness and sampling.

Source ↗Observed 2026-10-14
Anthropic

Claude Opus 4.6

Pretty great at coding, terminal work.

SWE-bench Verified
pretty great
80.8%

Vendor-reported SWE-bench Verified. See source for evaluation and memorization-screen caveats; harness differs from OpenAI reports.

Source ↗Observed 2026-10-14

The top score means the top sourced score in this snapshot. Different harnesses, budgets and samples can change results. A blank cell is a blank cell.