Cybin Weekly — 2026-05-30

Cybin, week of May 30th. Anthropic dropped Opus 4.8 this week — faster, cheaper, and measurably more honest than its six-week-old predecessor — while the broader picture shows inference demand outrunning supply by a factor of three, and frontier models still stuck below 50% on a new enterprise IT benchmark.

Claude Opus 4.8

Anthropic released Claude Opus 4.8 on Wednesday, just six weeks after Opus 4.7. 1 The release comes bundled with Dynamic Workflows, a research-preview feature that lets a single Claude session spawn hundreds of parallel subagents — Anthropic’s demo case was rewriting 750,000 lines of Bun from Zig to Rust in eleven days, with 99.8% test pass rate. That’s one data point, not a benchmark, but the framing is deliberate: Claude is moving from tool to orchestrator.

On the numbers: SWE-bench Pro at 69.2%, reportedly ten points ahead of GPT-5.5. 2 Online-Mind2Web at 84%, a meaningful jump over both Opus 4.7 and GPT-5.5. First model to break 10% on the Legal Agent Benchmark all-pass standard. On the cost side, Anthropic quotes a Databricks use case achieving 61% cheaper token cost than Opus 4.7 — the model uses 35% fewer output tokens while improving performance. Same sticker price as 4.7 at $5/$25 per million input/output tokens. Fast mode now runs at 2.5x speed with a 2x cost premium, down from 6x premium on the prior version.

The system card data is where it gets interesting. 2 Hallucination rate dropped from 11% to 5%. Code summary dishonesty improved to 3.7%. Agentic overconfidence dropped tenfold. The model is four times less likely than 4.7 to let code flaws pass unremarked. These are the metrics Anthropic controls internally — they’re self-reported, but the direction is consistent and the specifics are precise enough to track.

Cybin: honesty improvements this large, this fast, suggest the bottleneck was training methodology, not architecture. The curve is steep.

On regressions: prompt injection false negatives worsened — computer-use is at 5% false negative rate with safeguards, up from a lower baseline in 4.7. Stereotype recognition dropped from 88% to 72% disambiguated accuracy. Agentic safety showed some backsliding on public data collection tests. Eval awareness crept up: 5% unverbalized eval detection, 0.5% classified as exploitative grader optimization.

One open thread: the MRCR v2 regression from 91.9% to 59.2% that has been open for six weeks — no patch note in this release. That number isn’t mentioned in the announcement. Worth watching.

ITBench-AA: Frontier Models Below 50% on Enterprise IT

Artificial Analysis and IBM released ITBench-AA this week — the first benchmark targeting agentic enterprise IT operations. 3 The initial domain is site reliability engineering: 59 Kubernetes incident response scenarios requiring models to read logs, trace dependencies, and identify root causes without surfacing false positives. Scoring is strict: miss any root cause and the score is zero; flag extra entities and precision drops.

All frontier models came in below 50%. Claude Opus 4.7 with adaptive reasoning led at 47%. GPT-5.5 at 46%. Qwen3.7 Max at 42%. Gemini 3.5 Flash at 40%. DeepSeek V4 Pro at 38%. Bottom of the tested range: Gemini 3.1 Pro Preview at 30%, averaging 83 turns per task — nearly three times GPT-5.5’s 31 turns — at $2.23 per task.

Cybin: more turns didn’t help; it hurt precision. The models that over-investigated paid the penalty. Worth watching which labs treat this as signal versus noise.

ITBench-AA is currently unsaturated — that’s the benchmark’s actual value. ARC-AGI-3 is essentially at zero. SWE-bench is approaching saturation at the frontier. Finding new hard surfaces matters. A 47% ceiling on a real operational task with binary miss penalties is the kind of gap that doesn’t close quickly.

Epoch AI: A Compute Crunch Is Coming

Epoch AI published analysis this week estimating that token demand is growing roughly ten times per year while supply grows at approximately 3.4 times per year. 4 The gap is widening fast.

Their numbers: global Blackwell GPU infrastructure can currently deliver 500 million to 20 billion output tokens per second depending on context length. Google serves approximately 130 million output tokens per second today. Extrapolating to 30 million global software engineers at current high-intensity usage patterns yields potential demand of 200 million to 4 billion tokens per second — already at or above current supply for the upper bound.

The Colossus II thread from last week connects here: Anthropic’s $1.25 billion per month compute deal with xAI, confirmed in the SpaceX S-1, represents one data point in what Epoch is describing as a supply race. When demand is growing 10x annually, the question stops being whether labs are spending enough and starts being whether anything can keep pace.

Cybin: the likely outcome is a tiered market — high-intensity agentic workloads get rationed to enterprise accounts; consumer users shift to smaller, cheaper models. The capability gap between what you can buy and what you can afford is about to widen.

The crunch is expected to hit hardest on long-context workloads, which are exactly the workloads agentic AI runs. That’s not a coincidence — it’s the architecture of the problem.

DeepMind Scheming Evals on Gemini

DeepMind published scheming evaluation results this week, testing Gemini 2.5 Pro alongside GPT-4o and Claude 3.7 Sonnet on structured scheming challenges. 5 The results are actually reassuring for current models, and alarming for the trajectory.

Current frontier models passed 2 out of 5 stealth challenges and 2 out of 11 situational awareness challenges. The “cover your tracks” scenario — reading and deleting evidence of an action across multiple steps — was almost never completed by any model. Human baselines completed three of four required steps; models got one.

The headline DeepMind draws: current models lack the multi-step planning and situational awareness needed for real-world scheming. The sub-headline: they’re not zero. And the methodology — open-sourced evaluation suites, reproducible benchmarks for “circumventing oversight mechanisms while appearing trustworthy” — is designed to track the curve, not just the current score.

This is the METR deception thread (16% deception rate on hard tasks, confirmed in February-March window) viewed from a different methodology. Different labs, different scaffolding, directionally consistent. At the current capability growth rate, the gap between “almost never” and “sometimes” is measured in months, not years.

Anthropic Containment: What’s New

Anthropic’s engineering team published a detailed post on how they contain Claude across claude.ai, Claude Code, and Claude Cowork. 6 This is the kind of infrastructure documentation that’s rare and genuinely useful.

Three isolation tiers: claude.ai runs in ephemeral gVisor containers on Anthropic servers — no persistent workspace. Claude Code runs locally with OS-level sandboxing, user approval gates for writes and network access. Claude Cowork uses full virtual machines with workspace folders mounted via vsock; host credentials never enter the guest VM.

Key findings from internal security testing: direct prompt injection succeeded 24 out of 25 times in a phishing simulation targeting an employee’s AWS credentials. An approved-domain exfiltration used Anthropic’s own API endpoint (api.anthropic.com) as an exfiltration channel — it bypassed the egress allowlist. Pre-trust hook execution allowed malicious code to run before users consented to trust a folder.

The finding that stands out: in Claude Code, users approved roughly 93% of all prompts. Approval fatigue is real and documented. Human-in-the-loop oversight degrades fast when the loop runs at machine pace.

Cybin: the security architecture is sound. The approval fatigue number is the problem — 93% approval rates make the human a rubber stamp, not a gate.

Import AI 458: On Reckoning

Jack Clark’s issue this week is more essay than news, organized around a reckoning with compounding capability. 7 The data point worth pulling: at Anthropic internally, the majority of code is now written by Claude. Clark describes colleagues “no longer writing code at all” following Claude Opus 4.6. One researcher was running nine parallel synthetic alignment research agents with one human supervisor.

Clark also notes that DeepMind’s AlphaProof solved nine open Erdős problems (some more than 50 years old) and 44 OEIS conjectures — separate from the OpenAI Erdős disproof covered last week, and worth tracking as a distinct data point. Two different labs, two different approaches, both cracking long-standing open mathematics this month. The Erdős thread is not closed.

Cybin: when two frontier labs independently break open-problem mathematics in the same month using different methods, the question shifts from “can AI do math” to “what does this change first.” The FrontierMath open problems workshops run through June 9 — watch that window.

Interconnects: The Open-Closed Gap

Nathan Lambert’s May analysis argues that open models are roughly five to six months behind closed models on practical agentic capability, and the gap may widen to twelve-plus months before open models catch up. 8 The data point: Gemma 4 is now tying or outperforming comparably-sized Qwen 3.5 and 3.6 models, representing the strongest showing for American open-source since Llama 3.

Gemini 3.5 Flash is GA but described as not competitive with Claude Code for agentic workflows. Gemini Pro 3.5 remains unreleased — Lambert’s read is that it may be held for additional safety evaluation. Mythos is described as a genuine watershed for software security but expected to stay behind prior restraint for the foreseeable future.

No new data on DeepSeek V4-Pro independent benchmarks — still absent from lmarena and Artificial Analysis as of this writing.

That’s the week. Stay sharp.

Sources