PODCAST

AI Post Transformers

AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis, proper citations, and nerdy humor.

technology

All Episodes

BurstGPT: A Real-World LLM Serving Workload Dataset

AI Post Transformers ·

2025/10/01

The May 2025 academic paper introduces BurstGPT, a novel, real-world workload dataset consisting of over ten million traces from regional Azure OpenAI GPT services collected over 213 days, which aims to optimize Large Language Model (LLM) serving systems. The authors argue that existing LLM serving optimizations are often evaluated using unrealistic synthetic or non-LLM workloads, leading to performance degradation in real-world deployments. BurstGPT provides empirical data on user concurrency

00:12:27

Qwen3-Next & Qwen3-Omni technical report

AI Post Transformers ·

2025/09/30

These May and September 2025 technical reports introduce and evaluate two distinct but related large language models: the **Qwen3 family** and the **Qwen3-Omni** multimodal system. The first source focuses on the text-based Qwen3 models, highlighting their development process, which includes a sophisticated **multilingual data annotation system** and a multi-stage training pipeline incorporating **Strong-to-Weak Distillation** and "Thinking Mode" for complex tasks like coding and mathematics. The

Qwen3-Next & Qwen3-Omni technical report

AI Post Transformers ·

2025/09/30

These May and September 2025 technical reports introduce and evaluate two distinct but related large language models: the Qwen3 family and the Qwen3-Omni multimodal system. The first source focuses on the text-based Qwen3 models, highlighting their development process, which includes a sophisticated multilingual data annotation system and a multi-stage training pipeline incorporating Strong-to-Weak Distillation and "Thinking Mode" for complex tasks like coding and mathematics. The second,

00:16:07

Variational Reasoning Framework for Language Models

AI Post Transformers ·

2025/09/29

This September 26 2025 paper is an excerpt from a research paper introducing a variational reasoning framework designed to enhance the reasoning capabilities of language models (LLMs). This framework conceptualizes thinking traces as latent variables and uses variational inference to optimize them, building upon the Evidence Lower Bound (ELBO) and extending it with a tighter, multi-trace IWAE-style bound. Crucially, the paper proposes a forward-KL objective for stabilizing the training of the

Variational Reasoning Framework for Language Models

AI Post Transformers ·

2025/09/29

00:16:51

Federated Learning with Soft Embeddings for Retrieval

AI Post Transformers ·

2025/09/27

This September 20 2025 paper introduce a novel, efficient architecture for training **retrieval models** used in retrieval-augmented generation (RAG) systems. This architecture addresses the inefficiency of fine-tuning large models by combining **adapters for soft embeddings** with a **Classifier-as-Retriever (CaR)** approach. The soft embeddings, created by lightweight layers in a frozen small language model (SLM), efficiently adapt the model to new corpora, while the CaR replaces static maximum

00:14:09

Schoenfeld Theory Applied to Large Reasoning Models

AI Post Transformers ·

2025/09/27

This September 18 2025 paper introduces a research project that applies **Schoenfeld’s Episode Theory**, a classic cognitive framework for analyzing human mathematical problem-solving, to understand the reasoning processes of **Large Reasoning Models (LRMs)**. The authors created a novel, publicly available benchmark by annotating thousands of sentences and paragraphs from model-generated solutions to math problems, using seven cognitive labels such as **Plan, Implement, and Verify**. This approach

00:14:26

CWM: Code Generation with World Models

AI Post Transformers ·

2025/09/27

This Meta September 24 2025 paper provides an extensive overview of **Code World Model (CWM)**, a 32-billion-parameter dense decoder-only Transformer designed for **coding and reasoning tasks**, highlighting its architecture and multi-stage training process. Training involves **pre-training, mid-training, and post-training stages** which include supervised fine-tuning (SFT) and joint reinforcement learning (RL) across environments like software engineering (SWE) tasks, coding problems, and

Schoenfeld Theory Applied to Large Reasoning Models

AI Post Transformers ·

2025/09/27

This September 18 2025 paper introduces a research project that applies Schoenfeld’s Episode Theory, a classic cognitive framework for analyzing human mathematical problem-solving, to understand the reasoning processes of Large Reasoning Models (LRMs). The authors created a novel, publicly available benchmark by annotating thousands of sentences and paragraphs from model-generated solutions to math problems, using seven cognitive labels such as Plan, Implement, and Verify. This approach offers a

CWM: Code Generation with World Models

AI Post Transformers ·

2025/09/27

This Meta September 24 2025 paper provides an extensive overview of Code World Model (CWM), a 32-billion-parameter dense decoder-only Transformer designed for coding and reasoning tasks, highlighting its architecture and multi-stage training process. Training involves pre-training, mid-training, and post-training stages which include supervised fine-tuning (SFT) and joint reinforcement learning (RL) across environments like software engineering (SWE) tasks, coding problems, and mathematics. A core

Federated Learning with Soft Embeddings for Retrieval

AI Post Transformers ·

2025/09/27

This September 20 2025 paper introduce a novel, efficient architecture for training retrieval models used in retrieval-augmented generation (RAG) systems. This architecture addresses the inefficiency of fine-tuning large models by combining adapters for soft embeddings with a Classifier-as-Retriever (CaR) approach. The soft embeddings, created by lightweight layers in a frozen small language model (SLM), efficiently adapt the model to new corpora, while the CaR replaces static maximum inner product

00:14:21

EmbeddingGemma: Powerful Lightweight Text...

AI Post Transformers ·

2025/09/26

The September 24 2025 paper introduces **EmbeddingGemma**, a novel, lightweight text embedding model developed by **Google DeepMind**, built upon the **Gemma 3 language model family**. The paper details the innovative training methodology, which involves **encoder-decoder initialization** and **geometric embedding distillation** from larger models like Gemini Embedding, alongside a "spread-out" regularizer and model souping for **improved expressiveness and generalizability**. Through extensive

00:17:41

CE-GPPO: Controlling Entropy via Gradient-Preserving...

AI Post Transformers ·

2025/09/26

The September 25 2035 paper introduces a novel reinforcement learning (RL) algorithm, **Controlling Entropy via Gradient-Preserving Policy Optimization (CE-GPPO)**, designed to fine-tune large language models (LLMs) for complex reasoning tasks. The authors analyze how **policy entropy**, which represents the balance between exploration and exploitation, becomes unstable in existing methods like Proximal Policy Optimization (PPO) due to the clipping of **low-probability tokens**. CE-GPPO addresses

00:12:31

Seedream 4.0: Multimodal Image Generation System

AI Post Transformers ·

2025/09/26

The September 24 2025 paper is a technical report from **ByteDance Seed** detailing the **Seedream 4.0** system, an advanced multimodal image generation model. This single framework efficiently unifies **text-to-image synthesis**, image editing, and multi-image composition. The core innovation is an efficient **diffusion transformer** and powerful VAE that facilitates fast generation of **high-resolution images (up to 4K)**, outperforming competitors like GPT-4o and Gemini 2.5 in human and automatic

00:15:40

Tree-based Group Policy Optimization for LLM Agents

AI Post Transformers ·

2025/09/26

The September 25 2025 paper introduces **Tree-based Group Relative Policy Optimization (Tree-GRPO)**, a new reinforcement learning (RL) method designed to enhance the agentic capabilities of large language models (LLMs) in multi-turn tasks where supervision is typically sparse. Tree-GRPO addresses the challenges of sparse rewards and heavy rollout costs associated with existing chain-based RL by employing a **tree-search sampling strategy** where each node represents a complete agent interaction

GDPval: Measuring AI Performance on Real-World Work

AI Post Transformers ·

2025/09/26

The September 25 2025 dated sources introduce GDPval, a novel benchmark created by OpenAI to evaluate the performance of AI models on economically valuable, real-world tasks. This evaluation spans 44 knowledge work occupations across the top nine sectors contributing to the U.S. GDP, using tasks meticulously crafted by experienced industry professionals. Results indicate that the best frontier models are approaching human expert quality on these tasks, with models like Claude Opus 4.1 and GPT-5

Seedream 4.0: Multimodal Image Generation System

AI Post Transformers ·

2025/09/26

The September 24 2025 paper is a technical report from ByteDance Seed detailing the Seedream 4.0 system, an advanced multimodal image generation model. This single framework efficiently unifies text-to-image synthesis, image editing, and multi-image composition. The core innovation is an efficient diffusion transformer and powerful VAE that facilitates fast generation of high-resolution images (up to 4K), outperforming competitors like GPT-4o and Gemini 2.5 in human and automatic evaluations. The

CE-GPPO: Controlling Entropy via Gradient-Preserving...

AI Post Transformers ·

2025/09/26

The September 25 2035 paper introduces a novel reinforcement learning (RL) algorithm, Controlling Entropy via Gradient-Preserving Policy Optimization (CE-GPPO), designed to fine-tune large language models (LLMs) for complex reasoning tasks. The authors analyze how policy entropy, which represents the balance between exploration and exploitation, becomes unstable in existing methods like Proximal Policy Optimization (PPO) due to the clipping of low-probability tokens. CE-GPPO addresses this by

Tree-based Group Policy Optimization for LLM Agents

AI Post Transformers ·

2025/09/26

The September 25 2025 paper introduces Tree-based Group Relative Policy Optimization (Tree-GRPO), a new reinforcement learning (RL) method designed to enhance the agentic capabilities of large language models (LLMs) in multi-turn tasks where supervision is typically sparse. Tree-GRPO addresses the challenges of sparse rewards and heavy rollout costs associated with existing chain-based RL by employing a tree-search sampling strategy where each node represents a complete agent interaction step,

EmbeddingGemma: Powerful Lightweight Text...

AI Post Transformers ·

2025/09/26

The September 24 2025 paper introduces EmbeddingGemma, a novel, lightweight text embedding model developed by Google DeepMind, built upon the Gemma 3 language model family. The paper details the innovative training methodology, which involves encoder-decoder initialization and geometric embedding distillation from larger models like Gemini Embedding, alongside a "spread-out" regularizer and model souping for improved expressiveness and generalizability. Through extensive evaluation on the

1,015 results

AI Post Transformers

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs