Wednesday / AI 101 series: xQuant

Friday / Interview:

Agentic work: MCP-Bench tests whether agents can use servers and tools across multi-step tasks. ReportBench evaluates research agents on survey writing – not trivia, but the labor of scholarship itself.

Domain specificity: CMPhysBench asks if models know condensed matter physics. AetherCode scores them on competitive programming. MovieCORE pushes into cognitive reasoning about film.

Reasoning across modalities: T2I-ReasonBench looks at reasoning in text-to-image generation. SEAM checks semantic equivalence across language and vision. SpotEdit stresses precision in visual editing.

Safety and adaptivity: Mind the Third Eye! measures privacy awareness in smartphone agents. InMind tests whether models can adapt to individual reasoning styles.

Harder frontiers: UQ shifts the field from memorized test sets to unsolved questions, where there are no easy shortcuts.

Scientific reasoning disentangled: SCIREAS (Demystifying Scientific Problem-Solving in LLMs) separates domain knowledge from reasoning ability, probing whether models can truly “think scientifically” rather than just recall facts.

Microsoft AI’s MAI-Voice-1 and MAI-1-preview →read their blog

Gemini Nano Banana →read their blog

MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers →read the paper

ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks →read the paper

CMPhysBench: A Benchmark for Evaluating Large Language Models in Condensed Matter Physics →read the paper

AetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions →read the paper

MovieCORE: COgnitive REasoning in Movies →read the paper

T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation →read the paper

SEAM: Semantically Equivalent Across Modalities Benchmark for Vision-Language Models →read the paper

SpotEdit: Evaluating Visually-Guided Image Editing Methods →read the paper

Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered Smartphone Agents →read the paper

InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles →read the paper

Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning (SCIREAS)→read the paper

UQ: Assessing Language Models on Unsolved Questions →read the paper

Can Machines Think? by Alejandro Piad Morffis

Introducing the 2025 Intelligent Applications 40 by Madrona.com

OLMoASR: A series of open speech recognition models
These six fully open ASR models (39M–1.5B parameters) trained on curated datasets up to 680K hours. Benchmarked on 21 unseen test sets, OLMoASR-medium.en achieved 12.8%/11.0% WER (short/long-form), matching Whisper-medium.en. The largest model cut the WER gap with Whisper-large to 0.4% when trained on equal data. Built from a 3M-hour pool filtered to 1M hours, OLMoASR emphasizes reproducibility, rigorous data curation, and transparency →read their blog

gpt-realtime and Realtime API updates for production voice agents
This speech-to-speech model achieving 82.8% accuracy on Big Bench Audio and 30.5% on MultiChallenge—surpassing previous versions. It supports image inputs, SIP phone calling, and remote MCP servers. Function calling accuracy improved to 66.5%. Two new voices, Marin and Cedar, enhance naturalness. Unlike traditional pipelines, it processes audio in one step, reducing latency. The API now offers EU data residency, reusable prompts, and 20% lower pricing than gpt-4o-realtime-preview →read their blog

InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency
This LLM-based multimodal model family featuring Cascade Reinforcement Learning (offline + online RL) to enhance reasoning, achieving +16.0% gain on tasks like MMMU and MathVista. The Visual Resolution Router (ViR) dynamically adjusts visual token resolution, and Decoupled Vision-Language Deployment (DvD) balances GPU load. InternVL3.5-241B-A28B achieves 4.05× faster inference and state-of-the-art performance across general multimodal and agentic tasks among open-source models →read the paper

Hermes 4 technical report
It’s a hybrid reasoning LLM family built using 5M post-training samples (19B tokens), including 3.5M reasoning-heavy examples with sequences up to 16K tokens. They used DataForge for structured synthetic data generation and Atropos for rejection sampling across task-specific RL environments. Models (14B/70B/405B) achieved 81.9% on AIME’24 and 61.3% on LiveCodeBench, outperforming DeepSeek-R1 while reducing overlong outputs by 78%. All weights and evaluations are public →read the paper

USO: Unified style and subject-driven generation via disentangled and reward learning
This one uses a triplet dataset (content, style, stylized image) and trains via style-alignment and content-style disentanglement objectives. A Style Reward Learning (SRL) module further enhances generation quality. USO outperforms open-source models on USO-Bench, a benchmark jointly evaluating style similarity and subject fidelity, achieving state-of-the-art results in both style consistency and subject preservation →read the paper

rStar2-Agent: Agentic reasoning technical report
This is a 14B parameter math reasoning model trained with agentic RL. It uses GRPO-RoC, an RL strategy that handles noisy code environments, and is trained efficiently using only 64 MI300X GPUs. In just 510 RL steps, it achieves 80.6% on AIME24 and 69.8% on AIME25, outperforming DeepSeek-R1 (671B). The model also generalizes to alignment, scientific reasoning, and agentic tool-use tasks →read the paper

VibeVoice technical report
This is a long-form speech synthesis model using next-token diffusion for continuous data generation. A novel tokenizer compresses speech data by 80× compared to Encodec without quality loss. VibeVoice can generate up to 90 minutes of speech involving four speakers in a 64K token window, delivering high-fidelity, multi-speaker dialogue synthesis that surpasses both open-source and proprietary systems in maintaining conversational coherence and naturalness →read the paper

🌟 Diffusion Language Models Know the Answer Before Decoding
accelerate diffusion language model inference by detecting early convergence and committing tokens before full refinement → read the paper

UltraMemV2: Memory Networks Scaling to 120B Parameters with Superior Long-Context Learning
redesign memory-layer architectures to rival MoE efficiency with better long-context performance and lower memory access → read the paper

🌟 Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inference
optimize large-scale LLM serving with HeteroScale, a coordinated autoscaling framework that balances prefill and decode stages across heterogeneous GPUs, improving utilization by 26.6% and saving hundreds of thousands of GPU-hours daily → read the paper

🌟 StepWiser: Stepwise Generative Judges for Wiser Reasoning
train generative reward models that “meta-reason” about intermediate steps, improving judgment accuracy and inference search → read the paper

🌟 ThinkDial: An Open Recipe for Controlling Reasoning Effort in Large Language Models
implement discrete reasoning modes (high, medium, low) to balance computation cost and performance → read the paper

Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?
examine the faithfulness of chain-of-thought reasoning in soft-reasoning tasks, showing influence and reliability can diverge → read the paper

TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency
model sequence generation as tree search to reduce RL training cost while preserving exploration → read the paper

🌟 Provable Benefits of In-Tool Learning for Large Language Models
prove that tool-augmented models scale factual recall beyond parameter limits, outperforming in-weight memorization → read the paper

🌟 Understanding Tool-Integrated Reasoning
provide the first theoretical proof of tool-augmented reasoning’s benefits and propose ASPO for better tool usage → read the paper

🌟 Neither Valid nor Reliable? Investigating the Use of LLMs as Judges
critically assess the reliability, validity, and assumptions behind using LLMs as evaluators in NLP → read the paper

🌟 Unraveling the cognitive patterns of Large Language Models through module communities
analyze emergent module communities in LLMs via network methods inspired by biology, revealing distributed skill patterns → read the paper

Beyond Transcription: Mechanistic Interpretability in ASR
apply interpretability tools like logit lens and activation patching to speech recognition, uncovering hidden acoustic-semantic dynamics → read the paper

Efficient Code Embeddings from Code Generation Models
build compact autoregressive code embedding models for retrieval, Q&A, and cross-language similarity → read the paper

Autoregressive Universal Video Segmentation Model
unify prompted and unprompted video segmentation into one autoregressive architecture for streaming video → read the paper

🌟 Mixture of Contexts for Long Video Generation
introduce sparse attention routing for diffusion transformers to preserve consistency in long video synthesis → read the paper

Self-Rewarding Vision-Language Model via Reasoning Decomposition
strengthen visual reasoning in VLMs by decomposing perception and reasoning, rewarding self-contained perceptions → read the paper

🌟Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning
stabilize text-to-image reinforcement learning with pairwise preference rewards and a unified benchmark → read the paper

OmniHuman-1.5: Instilling an active mind in avatars via cognitive simulation
generate semantically expressive avatar animations by using LLM-structured conditions and a Multimodal DiT with Pseudo Last Frame for lip-sync, motion naturalness, and semantic alignment across single/multi-person and non-human scenes →read the paper

Spacer: Towards Engineered Scientific Inspiration
generate creative, grounded scientific hypotheses by recombining keyword graphs and refining them into concepts → read the paper

CODA: Coordinating the Cerebrum and Cerebellum for a Dual-Brain Computer Use Agent
combine a generalist planner and specialist executor with decoupled RL for scientific computing GUIs → read the paper

AWorld: Orchestrating the Training Recipe for Agentic AI
scale reinforcement learning for agentic AI with distributed interaction environments, enabling faster experience generation → read the paper

UItron: Foundational GUI agent with advanced perception and planning
train a large-scale mobile/PC GUI agent with SFT + curriculum RL over 1M+ steps to improve perception, grounding, and task planning for Chinese apps →read the paper

🌟 Servant, Stalker, Predator: How An Honest, Helpful, And Harmless (3H) Agent Unlocks Adversarial Skills
expose vulnerabilities in Model Context Protocol (MCP) agents, showing how benign tasks can chain into adversarial attack sequences that bypass service isolation and compromise security → read the paper