PODCAST

AI Post Transformers

AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis, proper citations, and nerdy humor.

technology

All Episodes

Dario Amodei: Machines of Loving Grace

AI Post Transformers ·

2026/02/11

Dario Amodei argues that powerful AI could catalyze a "compressed 21st century," achieving 100 years of progress in a decade. He envisions radical breakthroughs in biology, neuroscience, and economic development while emphasizing the need to defend liberal democracy. Source: October 2024 Machines of Loving Grace Anthropic Dario Amodei https://www.darioamodei.com/essay/machines-of-loving-grace

LongCat: Scaling Embeddings Outperforms Scaling...

AI Post Transformers ·

2026/02/11

Researchers from the LongCat introduced LongCat-Flash-Lite on January 2026, demonstrating that scaling embeddings via N-gram layers outperforms increasing Mixture-of-Experts parameters in high-sparsity regimes. This architecture uses system optimizations and speculative decoding to boost inference speed. Source: January 2026 Scaling Embeddings Outperforms Scaling Experts in Language Models Meituan LongCat Team Hong Liu, Jiaqi Zhang, Chao Wang, Xing Hu, Linkun Lyu, Jiaqi Sun, Xurui Yang, Bo Wang,

Reinforced Attention Learning

AI Post Transformers ·

2026/02/11

In a collaboration between UC Davis, Princeton University, Google, and Google DeepMind the paper "Reinforced Attention Learning", published on February 4, 2026, identifies a critical bottleneck in current Multimodal Large Language Models (MLLMs): while Reinforcement Learning (RL) has successfully scaled reasoning in text models, simply forcing multimodal models to generate verbose "chains of thought" often degrades their ability to perceive visual details. The authors argue that

Sapient Intelligence: Hierarchical Reasoning Model

AI Post Transformers ·

2026/02/11

The Hierarchical Reasoning Model (HRM) is a recurrent architecture using high-level planning and low-level execution modules to achieve deep latent reasoning. It employs Q-learning for adaptive computation, training a halting mechanism to optimize thinking time. Source: June 2025 Hierarchical Reasoning Model Sapient Intelligence, Singapore; Tsinghua University Guan Wang, Jin Li, Yuhao Sun, Xing Chen, Changling Liu, Yue Wu, Meng Lu, Sen Song, Yasin Abbasi Yadkori https://arxiv.org/pdf/2506.21734

A Comprehensive Survey of Mixture-of-Experts:...

AI Post Transformers ·

2026/02/11

Mixture of Experts (MoE) is a scalable architecture that uses a gating function to activate specialized expert networks dynamically. This "divide and conquer" approach enhances efficiency and interpretability in CV, NLP, and Reinforcement Learning paradigms. Source: March 2025 A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications University of Houston Siyuan Mu, Sen Lin https://arxiv.org/pdf/2503.07137

Advances in Attention Distillation for Efficient...

AI Post Transformers ·

2026/02/11

Recent research advances attention distillation to optimize transformers. HAD binarizes keys/queries for efficiency, while SHD aligns varying head counts. CompoDistill improves multimodal reasoning via visual alignment, and new losses transfer visual characteristics in diffusion models.Sources:1)February 3 2025Hamming Attention Distillation: Binarizing Keys and Queries for Efficient Long-Context TransformersMark Horton, Tergel Molom-Ochir, Peter Liu, Bhavna Gopal, Chiyue Wei, Cong Guo, Brady Taylor,

ChunkKV: Semantic-Preserving KV Cache Compression for...

AI Post Transformers ·

2026/02/11

ChunkKV improves LLM efficiency by compressing the KV cache using semantic chunks rather than isolated tokens, preserving linguistic integrity. It features layer-wise index reuse to boost throughput by 26.5%. Separately, Expected Attention estimates future token importance. Source: February 2025 ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference The Hong Kong University of Science and Technology (Guangzhou), The Hong Kong University of Science and Technology,

DR. KERNEL: Reinforcement Learning for Optimized...

AI Post Transformers ·

2026/02/11

Researchers introduced DR. KERNEL, a 14B model for Triton kernel generation trained via reinforcement learning. To prevent reward hacking and lazy optimization, they developed KERNELGYM for execution-based feedback and TRLOO for unbiased multi-turn RL. It rivals GPT-5. Source: February 2026 DR. KERNEL: Reinforcement Learning Done Right for Triton Kernel Generations HKUST, TikTok, CUHK(SZ), NTU Wei Liu, Jiawei Xu, Yingru Li, Longtao Zheng, Tianjian Li, Qian Liu, Junxian He

00:15:17

Towards a Science of Scaling Agent Systems

AI Post Transformers ·

2026/02/09

Google Research published January 28, 2026 introduces quantitative scaling principles for AI agents. While multi-agent systems boost performance on parallel tasks, they often degrade sequential ones due to coordination overhead. A new predictive model identifies optimal architectures with 87% accuracy.Sources:https://research.google/blog/towards-a-science-of-scaling-agent-systems-when-and-why-agent-systems-work/https://arxiv.org/pdf/2512.08296

Towards a Science of Scaling Agent Systems

AI Post Transformers ·

2026/02/09