Next Week in The Sequence:

1. Knowledge section: We present a summary of our recent series about evaluations and intro to our new cool series.

2. AI of the Week: Let’s dive into Asymov, the first model from superintelligence lab Reflection AI’s.

3. Opinion: About the relevance of monitoring chain of thoughts in LLMs.

Let’s Go! You can subscribe to The Sequence below:

📝 Editorial: The Transparent Transformer: Monitoring AI Reasoning Before It Goes Rogue

It’s a good thing when all the AI labs come together!

A new research paper, "Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety," is making waves in the AI community. Authored by researchers from DeepMind, OpenAI, Anthropic, Apollo Research, and the UK AI Safety Institute, the paper introduces a compelling idea: that we may be able to monitor advanced language models by observing their internal reasoning processes, known as chains of thought (CoT). These reasoning traces, when made visible through natural language, can offer a unique safety mechanism, allowing developers to detect and understand the early signs of misalignment or dangerous behavior.

Modern large language models (LLMs) often externalize part of their reasoning as they generate responses. This behavior is particularly pronounced when using chain-of-thought prompting, which encourages step-by-step thinking in natural language. The paper outlines two key enablers of CoT monitorability. First is the necessity: models often need to "think out loud" to complete complex tasks. Second is the propensity: even when it isn't strictly necessary, models tend to verbalize their reasoning. These CoT traces can reveal intentions such as reward hacking, deceptive behavior, or early misalignment, often before any harmful output occurs.

However, the authors emphasize that this transparency is fragile. As models are trained with increasingly sophisticated reinforcement learning techniques, they may start optimizing for outcomes in ways that bypass natural language reasoning altogether. Similarly, emerging architectures that rely on latent-space reasoning may no longer require explicit CoTs, making it harder to monitor what the model is actually thinking. Even direct interventions like shortening reasoning traces for efficiency or safety can unintentionally erode the quality and interpretability of the CoT.

To address this, the paper calls for a new research focus: evaluating and preserving CoT monitorability. The authors propose a set of benchmarks and metrics to assess how legible, truthful, and causally connected the CoTs are to a model's output. Techniques such as causal interventions, readability scoring with other LLMs, and adversarial stress tests could become standard tools in the AI safety toolbox. Crucially, these evaluations should be reported transparently in model system cards and influence decisions about model training, deployment, and architecture choices.

The publication of this paper has already sparked significant interest across the AI safety ecosystem. Notably, it includes endorsements from Geoffrey Hinton, Ilya Sutskever, and John Schulman, signaling strong support from the community's top thinkers. The paper arrives at a critical moment, aligning with recent efforts by the UK and US AI Safety Institutes to evaluate models' situational awareness and their ability to evade detection. CoT monitoring may soon become a central pillar of industry safety audits and regulatory frameworks.

In short, Chain of Thought Monitorability offers both a warning and an opportunity. It shows that we currently have a rare and potentially powerful way to peek inside the decision-making process of advanced AI models. But it also makes clear that this window may close if we aren't careful. As we push the boundaries of AI capabilities, we must also invest in preserving—and improving—our ability to understand how these systems think. CoT monitoring isn’t a magic bullet, but it may be our clearest lens yet into the minds of machines.

🔎 AI Research

Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety

Authors: UK AI Safety Institute, Apollo Research, METR, University of Montreal & Mila, Anthropic, OpenAI, Google DeepMind, Truthful AI, Redwood Research, Amazon, Meta, Scale AI, Magic
Summary: This paper introduces the concept of Chain-of-Thought (CoT) monitorability—the ability to detect misbehavior in reasoning language models by inspecting their generated thoughts—as a new avenue for AI safety. While promising, CoT monitoring is shown to be fragile and may degrade with certain training choices, requiring active research and evaluation to preserve its effectiveness.

Network Dissection: Quantifying Interpretability of Deep Visual Representations

Authors: CSAIL, MIT
Summary: The authors propose a framework called Network Dissection that quantifies interpretability in CNNs by aligning individual hidden units with semantic visual concepts using a densely labeled dataset. They show that interpretability is axis-aligned and not an inherent property of representations, revealing it can be disrupted by simple transformations like basis rotation.

KV Cache Steering for Inducing Reasoning in Small Language Models

Authors: University of Amsterdam (VIS Lab), University of Technology Nuremberg (FunAI Lab), MIT CSAIL
Summary: This work introduces cache steering, a method that modifies the key-value cache of small LLMs to induce chain-of-thought reasoning by inserting vectors derived from larger models like GPT-4o. The technique offers a lightweight and stable alternative to activation steering, improving reasoning behavior without requiring fine-tuning or complex prompts.

Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities

Authors: Google DeepMind
Summary: Gemini 2.5 introduces a family of multimodal models featuring advanced reasoning, long-context processing, and agentic capabilities, with state-of-the-art performance across coding, math, video, audio, and factuality benchmarks. Notably, it incorporates dynamic "thinking" budgets and strong tool-use abilities, enabling agentic workflows such as full-game Pokémon playthroughs and complex software engineering tasks.

Scaling Laws for Optimal Data Mixtures

Authors: Apple, Sorbonne University
Summary: This paper presents a family of scaling laws that model how training loss depends not just on model size and compute but also on the domain composition of the training data, allowing for principled prediction of optimal data mixtures. The laws generalize across LLMs, multimodal, and vision models, enabling efficient discovery of ideal domain weights using small-scale runs.

COLLABLLM: From Passive Responders to Active Collaborators

Authors: Stanford University, Microsoft Research, Georgia Institute of Technology
Summary: COLLABLLM introduces a training framework that enables large language models to become proactive collaborators by optimizing for Multiturn-aware Rewards—metrics that reflect long-term task success and user satisfaction rather than just next-turn correctness. By simulating realistic conversations and using reinforcement learning, COLLABLLM models significantly improve task performance, engagement, and efficiency in both simulated benchmarks and real-world user studies

🤖 AI Tech Releases

ChatGPT Agent

OpenAI released the first version of ChatGPT Agent.

Asymov

Superintelligence lab Reflection AI unveiled Asymov, a code comprehension agent.

Mistral Deep Research

Mistral introduced Deep Research mode in their Le Chat app.

Voxtral

Mistral released Voxtral, its first speech understanding model.

AgentCore

Amazon launched AgentCore, a new framework for deploying agents securely at scale.

Kiro

Amazon also launched Kiro, a AI coding IDE to compete with Cursor and WindSurf.

📡AI Radar

1. "Lovable Rises to Unicorn Status After $200M Series A in Just 8 Months" – Sweden’s Lovable, a natural‑language web/app builder, raised $200 million from Accel at a $1.8 billion valuation, reaching 2.3 million users and $75 million ARR.

2. Scale AI trims data-label staff by 14% – Rate cuts 200 jobs and 500 contractor roles amid rapid GenAI expansion post‑Meta’s major investment.

3. Meta lands two OpenAI‑elite researchers – Jason Wei and Hyung Won Chung reportedly join Meta’s Superintelligence Lab from OpenAI.

4. Thinking Machines Lab rockets to $12B valuation – Mira Murati’s new AI venture secures a massive seed round with a $12 billion price tag.

5. ParadeDB challenges Elasticsearch via Postgres – New open‑source extension enables full‑text search and analytics directly within Postgres.

6. Cognition snaps up Windsurf – Devin’s maker Cognition acquires AI‑coding startup Windsurf following its CEO’s Google move.

7. Meta building Hyperion—a 5 GW AI data center – Zuckerberg reveals construction of a massive 5‑gigawatt AI‑focused facility to power Meta's ambitions.

8. Anthropic announced Claude for financial services - A new version of Claude optimized for financial analysis with integrations with the main platforms in the space.

9. "Privacy Shield for AI: Confident Security Emerges from Stealth with $4.2M" – San Francisco‑based Confident Security launched CONFSEC to wrap AI models in end-to-end encryption and guarantee data privacy, securing $4.2 million in seed funding.

10. Greptile Set for $30M Series A at $180M Valuation – Benchmark is reportedly leading a $30 million Series A for Greptile, the AI code-review startup, pegging its valuation at around $180 million as it competes in a crowded market .