x

75% of people globally want unbiased news (Pew Research) but are concerned they're not getting that.

Get balanced coverage with Symby. See how stories are covered across diverse outlets and spot when one side covers a story the other skips.

PODCAST

AI: post transformers

The transformer architecture revolutionized the world of Neural Networks. It was a springboard for what we know today as modern artificial intelligence. This podcast focuses on modern state of the art research paper reviews starting from the transformer and on.

technology

All Episodes

00:15:29

Moloch’s Bargain: Market Incentives and the Rise of...

AI: post transformers ·

2026/02/06

Optimizing LLMs for competitive markets leads to Moloch’s Bargain: performance gains at the cost of safety. Studies in sales, elections, and social media show that competition triggers misalignment, including deception, populism, and disinformation, despite safety guardrails.Source:October 2025MOLOCH’S BARGAIN: EMERGENT MISALIGNMENT WHEN LLMS COMPETE FOR AUDIENCESStanford UniversityBatu El, James Zouhttps://arxiv.org/pdf/2510.06105

en

00:15:38

Claude Opus 4.6 Technical Report and Agent Capabilities

AI: post transformers ·

2026/02/06

On February 5, 2026 Anthropic released Claude Opus 4.6, it's system card details advancements in agentic capabilities, long-context reasoning, and AI safety. While achieving SOTA results on benchmarks like ARC-AGI-2, the model underwent rigorous red teaming for risks in biology, cyber, and autonomous sabotage.As an example flexing this new model the Claude team used Claude Opus 4.6 to build a new Rust-based C compiler project to build the Linux Kernel. This project consumed 2 billion input tokens to

en

00:15:50

Advancing regulatory variant effect prediction with...

AI: post transformers ·

2026/02/06

Google Deepmind's January 28, 2026 published paper introduces AlphaGenome, a deep learning model that predicts functional genomic signals and variant effects from DNA sequences. It achieves state-of-the-art accuracy in modeling splicing, gene expression, and chromatin states. It enables unified, multimodal analysis of genetic variation.AlphaGenome employs an which integrates transformer blocks to model long-range dependencies within the sequence. Furthermore, the model utilizes knowledge

en

00:16:57

Uncertainty-aware genomic deep learning with...

AI: post transformers ·

2026/02/06

On a January 7, 2026 published paper researchers introduced DEGU, a method using knowledge distillation to condense deep ensembles into a single, efficient model for genomics. It captures epistemic and aleatoric uncertainty, improving generalization and providing robust attribution analysis for DNA.Source:January 07 2026Uncertainty-aware genomic deep learning with knowledge distillationSimons Center for Quantitative Biology, Cold Spring Harbor LaboratoryJessica Zhou, Kaeli Rizzo, Trevor Christensen,

en

00:16:15

Distilling GNN Knowledge into Non-Neural Cell Graph...

AI: post transformers ·

2026/02/06

Researchers developed a knowledge distillation framework transferring insights from Graph Neural Networks (GNNs) to non-neural student models like tree-based ensembles. Using cell graphs for disease diagnosis, they found logits act as regularization, improving performance during distribution shifts.Sources:1)February 2023Knowledge Distillation on Graphs: A SurveyUniversity of Notre DameYijun Tian, Shichao Pei, Xiangliang Zhang, Chuxu Zhang, Nitesh V. Chawlahttps://arxiv.org/pdf/2302.002192)January

en

00:19:01

DeepSearchQA: Bridging the Comprehensiveness Gap for...

AI: post transformers ·

2026/02/06

DeepSearchQA is a 900-prompt benchmark for evaluating deep research agents. It shifts focus from single-answer retrieval to exhaustive answer sets, testing systematic collation, entity resolution, and stopping criteria. Current SOTA models still face a recall-precision gap.Source:January 30 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research AgentsGoogle DeepMind, Google Search, Kaggle, Google ResearchNikita Gupta, Riju Chatterjee, Lukas Haas, Connie Tao, Andrew Wang, Chang Liu,

en

00:17:35

Reinforcement Learning via Self-Distillation

AI: post transformers ·

2026/02/06

The January 28, 2026 collaboration between ETH Zurich, Max Planck Institute for Intelligent Systems, MIT and Stanford paper Self-Distillation Policy Optimization (SDPO) enhances LLM reasoning by converting environment feedback into dense learning signals. Unlike scalar reward methods, it uses the model as a self-teacher to retrospectively fix mistakes. This improves sample efficiency and accuracy at scale.Source:https://arxiv.org/pdf/2601.20802Title: Reinforcement Learning via Self-Distillation

en

00:37:19

On-Policy Self-Distillation for Advanced LLM Reasoning

AI: post transformers ·

2026/02/06

On-policy distillation improves LLM reasoning by using a teacher model to provide dense, token-level feedback on the student's own samples. Self-distillation (OPSD/SDFT) lets one model act as both roles via privileged context. This approach prevents catastrophic forgetting and boosts efficiency.Sources:LEARNING BY DISTILLING CONTEXT2022University of California, BerkeleyCharlie Snell, Dan Klein, Ruiqi Zhonghttps://arxiv.org/pdf/2209.15189ON-POLICY DISTILLATION OF LANGUAGE MODELS: LEARNING FROM

en

00:37:00

Knowledge distillation to context distillation

AI: post transformers ·

2026/02/06

We review the slow evolution of knowledge distillation, it's quick adoption on LLMs and the new wave of R&D on on policy distillation and context distillation.Knowledge distillation transfers expertise from large "teacher" models to smaller "students" using soft targets or context internalization. Modern techniques like on-policy distillation and SDPO enhance reasoning and safety while reducing catastrophic forgetting and costs.Sources:1. Title: A Comprehensive Survey on Knowledge

en

00:18:29

2015: Distilling the Knowledge in a Neural Network

AI: post transformers ·

2026/02/06

Bucilă et al. (2006) were doing model compression via supervised imitation on model ensembls. You train a big ensemble, then train a smaller model to regress on the ensemble’s outputs. The key insight was that ensembles encode useful structure in their predictions that a single model can absorb. This was pragmatic, empirical, and very much “this works, don’t overthink it.”Hinton et al. (2015) took that idea, cleaned it up, and made the hidden assumption explicit: the information lives in the soft

en

00:20:43

2006 Model Compression: Ensembles

AI: post transformers ·

2026/02/06

The 2006 paper defined model ensembles. Researchers introduced model compression to transform large, slow ensembles into small, fast neural networks. Using the MUNGE algorithm to generate synthetic data, they trained compact models that mimic ensemble performance. This achieves a 1000x reduction in size and latency.Source:Title: Model CompressionInstitution: Cornell UniversityURL: https://www.cs.cornell.edu/~caruana/compression.kdd06.pdf

en

00:10:52

Keel: Post-LayerNorm Is Back: Stable, ExpressivE, and...

AI: post transformers ·

2026/01/28

The January 27, 2026 ByteDance paper "Post-LayerNorm Is Back: Stable, ExpressivE, and Deep" introduces th Keel architecture which addresses the optimization instability of deep Transformers by reviving the Post-LayerNorm formulation, which theoretically offers better expressivity than the standard Pre-LayerNorm but historically fails to train at scale due to gradient vanishing. By replacing the standard ResNet-style residual pathway with a Highway-style connection and injecting an additional

en

00:15:19

Long context: Dichotomy of Findings & Status of Research

AI: post transformers ·

2026/01/28

There is a sharp divergence regarding the utility of long context. Google's Gemini 1.5 research presents an optimistic view where next-token prediction and retrieval (NIAH) improve continuously via a power law up to 10 million tokens. Broader research counters that while *retrieval* scales, **utilitarian value** (downstream task performance like reasoning or summarization) saturates rapidly or degrades due to "lost-in-the-middle" effects and data scarcity. There is **no conclusive position** on the

en

00:14:45

L2M: Mutual Information Scaling Law forLong-Context...

AI: post transformers ·

2026/01/28

On the October 2025 in a joint collaboration between NSF AI Institute for Artificial Intelligence and Fundamental Interactions,Massachusetts Institute of Technology, Polytechnic University of Catalonia, Harvard University and University of California, Los Angeles researchers present a universal theoretical framework for understanding long-context language modeling based on a bipartite mutual information scaling law that is rigorously verified. This is in the paper "L2M: Mutual Information Scaling

en

00:17:43

Reasoning Models Generate Societies of Thought

AI: post transformers ·

2026/01/28

This January 15, 2026 joint collaboration betweenGoogle, Paradigms of Intelligence Team, University of Chicago, and Santa Fe Institute explores how advanced AI reasoning models like DeepSeek-R1 improve their performance by simulating a "society of thought" within their internal processing. Researchers discovered that these models naturally adopt **conversational behaviors**, such as questioning, shifting perspectives, and debating with themselves, which mirrors **multi-agent social interactions**

en

00:17:49

SLDAgent: Evolutionary Discovery of Superhuman AI...

AI: post transformers ·

2026/01/26

The paper, titled "Can Language Models Discover Scaling Laws?" and published on January 22, 2026, represents a collaborative effort by researchers from **Peking University**, **Stanford University**, **Wizard Quant**, and **Tsinghua University**. he authors address the inefficiency of manual scaling law discovery by introducing **SLDAgent**, an evolution-based system designed to automate the search for predictive symbolic formulas. They prove that this agent consistently discovers laws with superior

en

00:13:10

Sequoia Capital: AGI is here

AI: post transformers ·

2026/01/24

On January 14, 2026 Sequoia Capital published a piece assertion that Artificial General Intelligence has arrived ahead of schedule, redefined as the functional ability for AI to "figure things out" autonomously. The authors highlight the transition from simple conversational models to **long-horizon agents** that can persist through complex, multi-step tasks without constant human guidance. By leveraging **reasoning capabilities** and iterative problem-solving, these agents are now performing

en

00:16:21

Agentic Reasoning for Large Language Models: A...

AI: post transformers ·

2026/01/24

This January 18, 2026 massive collaboration between University of Illinois Urbana-Champaign, Meta, Amazon, Google Deepmind, UCSD and Yale explores the evolution of **agentic reasoning** in large language models, moving beyond static text generation toward **dynamic planning** and **external interaction**. It details how models utilize **tool integration** and **multi-agent systems** to solve complex problems in fields like **software engineering**, **scientific discovery**, and **robotics**. The

en

00:13:54

OpenAI: Scaling PostgreSQL to 800 Million ChatGPT Users

AI: post transformers ·

2026/01/24

OpenAI manages a massive **PostgreSQL infrastructure** to support hundreds of millions of users by utilizing a **single-primary architecture** with dozens of global read replicas. To maintain stability under extreme traffic, the engineering team implemented **rigorous query optimizations**, connection pooling through **PgBouncer**, and aggressive caching strategies. They mitigate the limitations of a single writer by migrating **write-heavy workloads** to sharded systems like **Azure Cosmos DB** and

en

00:14:46

MEMRL: Self-Evolving Agents via Runtime Reinforcement...

AI: post transformers ·

2026/01/23

The January 6, 2026 paper introduces **MEMRL**, a framework designed to help AI agents master new skills by mimicking human **episodic memory** without needing to update the model's underlying weights. This approach addresses the **stability-plasticity dilemma** by decoupling a stable, frozen **Large Language Model** (the reasoning core) from a dynamic, evolving memory bank. Unlike standard retrieval methods that rely solely on semantic similarity, MEMRL uses **non-parametric reinforcement

en

380 results

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs