Wednesday / AI 101 series: The state of RL

Friday / AI Interview: Ben Goodger: why OpenAI needs its own browser?

Introducing Anthropic Interviewer: What 1,250 professionals told us about working with AI by Anthropic

State of AI: An Empirical 100 Trillion Token Study by OpenRouter and a16z

AI Engineering Code Summit Report by Will Schenk

From Code Foundation Models to Agents and Applications: A Survey and Practical Guide to Code Intelligence by ByteDance

🌟 DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models – Presents an open frontier model family that combines sparse attention, scaled RL, and large-scale agentic task synthesis to rival proprietary systems on reasoning and tool-use benchmarks →read the paper

🌟 Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction – Builds a full-stack ecosystem for training agent models across diverse, hierarchical environments, tying together complexity, diversity, and real-world fidelity into one scalable training loop →read the paper 

TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models – Constructs a truly unified visual latent space where the same representation supports understanding and generation for images and video, showing that shared encoders can boost both sides when trained jointly →read the paper

Vision Bridge Transformer at Scale – Instantiates large Brownian bridge models as a transformer that directly links input and output latents, enabling efficient, conditional image and video translation without the usual noise-to-data detour →read the paper

TV2TV: A Unified Framework for Interleaved Language and Video Generation – Couples a language tower and a video tower so the model can “think in text” between visual segments, interleaving tokens and frames to achieve better controllability and long-horizon video reasoning →read the paper

🌟 SIMA 2: A Generalist Embodied Agent for Virtual Worlds – Extends a foundation model into an embodied agent that can follow goals, converse, and learn new skills across many 3D environments, exploring what a generalist “game-world worker” looks like in practice →read the paper

WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning – Designs an agent with episodic, semantic, and visual memories that can adaptively retrieve across multiple temporal scales, pushing long-video QA beyond text-only summaries →read the paper

Stabilizing Reinforcement Learning with LLMs: Formulation and Practices – Formalizes when token-level surrogate rewards faithfully approximate sequence-level objectives, explaining why techniques like importance sampling, clipping, and routing replay actually stabilize LLM RL in practice →read the paper

🌟 Guided Self-Evolving LLMs with Minimal Human Supervision – Introduces a challenger–solver loop where small amounts of grounded human data guide large-scale synthetic evolution, showing how to push math and reasoning skills without the usual catastrophic drift →read the paper

ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual Reasoning – Turns reward models into agents that call tools, crop images, and fetch pages to justify their scores, improving multimodal alignment by forcing the reward to actually look at evidence →read the paper

🌟 retrainZero: Reinforcement Active Pretraining – Extends RL from narrow post-training into the pretraining stage, letting “reasoners” actively choose spans from raw corpora to predict and breaking the assumption that reasoning requires pre-verified labels →read the paper

Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach – Treats test-time scaling for embodied VLAs as an anti-exploration problem, selecting action chunks with pseudo-count estimates to stabilize behavior without expensive RL updates →read the paper

🌟 Rectifying LLM Thought from Lens of Optimization – Reinterprets chain-of-thought as an optimization trajectory and defines process-level rewards that favor stable, efficient reasoning, improving RLVR pipelines while reducing overthinking and flailing →read the paper

🌟 On GRPO Collapse in Search-R1: The Lazy Likelihood-Displacement Death Spiral – Diagnoses why GRPO-style search-integrated RL collapses by tracking likelihood drift over time, then proposes a lightweight regularizer that preserves likelihood and rescues tool-augmented training runs →read the paper

SR-GRPO: Stable Rank as an Intrinsic Geometric Reward for Large Language Model Alignment – Uses stable rank of hidden states as an internal, supervision-free signal for quality, then plugs it into GRPO to align models with their own representational geometry instead of external reward models →read the paper

SCALE: Selective Resource Allocation for Overcoming Performance Bottlenecks in Mathematical Test-time Scaling – Decomposes math problems into sub-tasks and routes test-time compute selectively, spending “System 2” effort only on hard bits to improve accuracy while cutting total tokens →read the paper

🌟 Glance: Accelerating Diffusion Models with 1 Sample – Splits denoising into semantic and cleanup phases with separate LoRA experts, showing that a base model plus tiny adapters can deliver big speedups with almost no retraining or quality loss →read the paper

UltraImage: Rethinking Resolution Extrapolation in Image Diffusion Transformers – Analyzes why diffusion transformers fall apart at ultra-high resolutions and introduces frequency corrections plus adaptive attention sharpening to reach up to 6K×6K generation without repeating patterns →read the paper

🌟 Rethinking Prompt Design for Inference-time Scaling in Text-to-Visual Generation – Treats the prompt itself as something to scale at inference, iteratively revising text based on verifier feedback across samples to push alignment beyond just more steps or more seeds →read the paper

🌟 SpeContext: Enabling Efficient Long-context Reasoning with Speculative Context Sparsity in LLMs – Uses a distilled model to predict which attention heads and tokens actually matter, pruning retrieval and overlapping KV-cache movement with compute to push long-context throughput on limited hardware →read the paper

WUSH: Near-Optimal Adaptive Transforms for LLM Quantization – Derives data-aware linear transforms that minimize quantization error for joint weight–activation blocks, turning a Hadamard-like backbone into a provably near-optimal, still-efficient transform →read the paper

🌟 CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning – Lets an RL loop auto-design HGEMM kernels guided purely by runtime rewards, systematically beating heavily hand-optimized baselines like cuBLAS in both offline and server settings →read the paper

🌟 PaperDebugger: A Plugin-Based Multi-Agent System for In-Editor Academic Writing, Review, and Editing – Embeds a multi-agent assistant directly inside LaTeX editors, orchestrating tools for search, review, patching, and diffing so that academic writing workflows become continuous, in-editor agent collaborations →read the paper