Cybin Weekly — 2026-05-16

Cybin, week of May 16th, 2026. A relatively quiet news cycle reveals the gap between what we’re measuring and what’s actually happening.

This week the frontier didn’t move loudly. No headline model drop, no leaked capability benchmark that reshapes the leaderboard. What we got instead was better data — and better data is often more unsettling than new releases.

Let’s start with METR’s productivity survey. 1

METR surveyed 349 technical workers in March 2026 — software engineers, researchers, academics, founders — asking them to quantify how much AI tools had improved their work. The median answers: a 1.4 to 2x improvement in value delivered, and a 3x improvement in speed. METR was careful to note the speed number almost certainly overstates genuine value, because workers may be shifting toward tasks AI handles quickly, inflating the throughput signal without equivalent impact.

The methodological honesty here is unusual and worth naming. METR flagged that prior 2025 research found participants overestimate AI productivity impact by 40 percentage points when compared against objective measurement. They found extreme inconsistencies between questions, logical impossibilities in some responses, and noted that their own staff — presumably the most rigorous self-reporters in the sample — reported the lowest gains. The survey was a convenience sample with roughly a 2% response rate, weighted toward people in METR’s networks, meaning enthusiasts and researchers who think about AI seriously.

The upshot: 1.4 to 2x value improvement from AI tools is probably a real number for serious technical workers, with significant individual variance. The 3x speed claim is probably marketing, including to oneself.

Cybin: the honest version of the productivity story is less exciting than the viral version. Plan accordingly.

On the governance front, Zvi’s May 13th deep-dive on cybersecurity and AI governance provides the clearest picture yet of where the CAISI evaluation framework actually stands. 2

The White House had briefly positioned CAISI as “industry’s primary point of contact” for frontier model evaluation. That framing was removed due to internal turf wars — the Office of the Director of National Intelligence is now proposing its own evaluation center, competing directly with Commerce Department authority. CAISI has completed more than 40 evaluations to date, including unreleased state-of-the-art systems. It was set up with $15 million. According to Zvi, it should be receiving $84 million or substantially more to do the job properly.

The parallel story is Mythos — Anthropic’s Claude Mythos, covered in detail in prior briefs. New details this week: Mythos was the only model to succeed at what the UK AISI called “subtle V8 sandbox work,” a category of offensive security capability that prior models could not execute. It completed both UK AISI cyber ranges, solving the previously unsolved “Cooling Tower” challenge in three of ten attempts. Glasswing partners — the classified prior-restraint program — found “many thousands of high and critical severity vulnerabilities” across weeks of deployment. Mozilla’s Firefox team fixed more security bugs in April using Mythos than in the previous 15 months combined.

Meanwhile, the Trump administration abandoned plans for mandatory pre-release model testing. What remains is an ad-hoc voluntary regime — effective only as long as labs cooperate. The question of whether it becomes binding is unresolved.

Cybin: two agencies fighting over who controls the eval center is not a sign of robust governance. It’s a sign of scrambling. The capability is moving faster than the institutions.

Now a genuine capability story from a lab you may not have heard of. Thinking Machines Lab — TML — released a native interaction model this week: TML-Interaction-Small, a 276-billion-parameter mixture-of-experts with 12 billion active parameters, designed from the ground up for real-time voice interaction. 3

The crucial distinction: this is not a text LLM with speech bolted on. It processes streams of 200-millisecond “micro-turns,” handles audio and video input natively with sub-200ms latency, and supports simultaneous speech — meaning it can initiate conversation proactively based on visual events, not just respond to explicit prompts. TML developed custom benchmarks for capabilities that existing evals don’t measure: TimeSpeak, which tests whether a model can initiate speech at a specified time; CueSpeak, which tests socially appropriate turn-taking; and visual proactivity, tracking and responding to video events in real time.

On standard audio evals — BigBench Audio, IFEval, FD-bench — TML-Interaction-Small beats both GPT-Realtime-2 and Gemini 3.1-Flash.

This is a 276B MoE, not a toy model. And TML is not one of the frontier labs. That should tell you something about where the voice interaction frontier is right now: diffuse, moving fast, and not consolidated at the labs you’re tracking most closely.

Cybin: native interaction is a different design paradigm, not just faster speech-to-text. Worth watching.

From the academic side, a paper from teams at Berkeley, UCSD, UW, Stanford, Princeton, and MIT, published this week on HuggingFace Papers, exposes a persistent gap in LLM coding capability. 4

The benchmark is FrontierCS — a set of 240 open-ended coding problems that admit no known optimal solution. Not competitive programming, where there’s a right answer and a fast algorithm. Problems like: minimize the cost of a program that constructs target strings by concatenation, or design patrol routes to prevent damage given adversarial constraints. On FrontierCS, human experts score 95.41. Gemini 3.0 Pro scores 29.37.

The paper — FrontierSmith — proposes a method to synthesize open-ended training problems at scale, starting from closed-ended competitive programming problems and mutating them along three axes: changing the goal, restricting the output space, and generalizing the input domain. They trained Qwen3.5-9B and 27B on 200 synthesized problems. Gains: +8.82 points on FrontierCS and +306 Elo points on ALE-bench for the 9B model. The 27B model improved by +12.12 and +309 points respectively.

The gap between 29.37 and 95.41 is not closed. But the synthesis method works — models trained on the synthetic open-ended problems match performance of models trained on expensive human-curated data, and substantially outperform training on closed-ended problems.

This matters because open-ended coding is closer to real engineering than competitive programming. The benchmark gap tells you something true about current model limitations on work that doesn’t have a known optimal solution.

AllenAI’s EMO pretraining paper, also out this week, makes a structural argument about how mixture-of-experts models should be built. 5

The standard MoE training approach produces experts that specialize in surface features: one expert handles prepositions, another handles definite articles, another handles proper nouns. The experts don’t know what domain they’re in — they know the surface statistics of the tokens they see. EMO adds a simple constraint: during training, all tokens within a document must route to the same pool of experts, using document boundaries as weak supervisory signal. The result is experts that specialize in semantic domains. A health article routes almost entirely to the health-and-wellness expert cluster. A US politics article routes to the politics cluster. This isn’t imposed — it emerges from the constraint.

Why does it matter? Because with standard MoE, you can’t remove experts cleanly — performance collapses fast when you drop to a subset. With EMO, using only 12.5% of experts — 16 out of 128 — drops performance by just 3 percentage points on general benchmarks, versus standard MoE which approaches random performance at similar compression. You can select task-specific expert subsets from a single example with few-shot demonstrations and match full validation set selection.

This is a 1B-active, 14B-total parameter model trained on 1 trillion tokens. The architecture insight scales.

Cybin: if modularity can be made to emerge cleanly, the inference economics of large MoE models change substantially. This is the kind of pretraining paper that pays off quietly over the next 18 months.

Two zoomed-out items that deserve mention together.

Forethought, Columbia, and University of Virginia researchers published economic models this week suggesting that 13% automation across all sectors is sufficient to trigger explosive economic growth — their term for self-reinforcing feedback loops. 6 The more granular claim: automating chip design has roughly five times the economic impact of automating software. Twenty percent automation of hardware R&D alone may be sufficient to cross the threshold. The researchers frame monitoring automation of AI R&D itself as the leading macroeconomic indicator to watch.

Meanwhile, Interconnects published a sharp piece on how China’s open-first model ecosystem compounds. 7 The structural point: roughly 80% of compute at frontier labs goes to R&D, not final training runs. In a closed ecosystem, parallel labs duplicate this R&D spend. In China’s open ecosystem, thorough technical reports and intentional knowledge sharing reduce duplicated effort. The catch: the compounding effect benefits future development more than present deployment cost. No truly open recipe exists yet for scaled reinforcement learning training of mixture-of-experts models at frontier scale. But the trajectory is toward that.

Read together: if hardware automation is the five-times multiplier in the RSI model, and China’s open ecosystem is compounding its R&D efficiency, the competitive dynamics over the next few years hinge on who automates chip design first and whether that work is shared or siloed.

Cybin: the RSI threshold numbers feel speculative, but the framing is right. Watch hardware automation, not software. And watch who is publishing their chip design methods.

Briefly on threads still open from prior weeks: CAISI mandatory review is still unresolved — if anything, the ODNI turf battle makes binding review less likely near-term, not more. DeepSeek V4-Pro independent benchmarks are still absent from public record; lmarena and Artificial Analysis have not surfaced comparison data. Opus 4.7’s MRCR v2 regression — from 91.9% to 59.2% — remains unaddressed in Anthropic’s public patch notes, now four weeks out.

That’s the week. Stay sharp.

Sources