PODCAST

AI Post Transformers

AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis, proper citations, and nerdy humor.

technology

All Episodes

00:15:40

HALoS: Hierarchical Asynchronous LLM Training over...

AI Post Transformers ·

2025/11/04

The June 5, 2025 research paper introducing **HALoS: Hierarchical Asynchronous Local SGD**, a novel optimization framework designed for training large language models (LLMs) across **geographically distributed** accelerators and slow, high-latency networks. The core challenge addressed is the inefficiency of standard synchronous training methods due to **slow inter-region communication** and **heterogeneous hardware speeds**. HALoS mitigates these issues through a **two-tier architecture** featuring

00:14:59

Anchored Diffusion Language Model: Superior...

AI Post Transformers ·

2025/11/04

The May 24, 2025 UT Austin paper introduces the **Anchored Diffusion Language Model (ADLM)**, a novel approach that aims to improve discrete language modeling by addressing the limitations of traditional **Autoregressive (AR)** and standard **Diffusion Language Models (DLMs)**. AR models, such as GPT-3, generate text sequentially and struggle with complex reasoning, while existing DLMs, which use iterative masked-token prediction, lag behind AR models in quality. ADLM enhances DLMs by incorporating

00:18:25

Gumbel-Softmax for Differentiable Categorical...

AI Post Transformers ·

2025/11/04

These two papers (years 2017, 2022) introduce and then apply the **Gumbel-Softmax distribution** as a differentiable gradient estimator for **categorical and discrete latent variables** in neural networks. The first paper, "Categorical Reparameterization with Gumbel-Softmax," proposes this distribution to address the challenge of **backpropagating through non-differentiable sampling operations**, demonstrating its effectiveness in tasks like structured output prediction and generative modeling,

00:16:11

PolicySmith: Automated Systems Heuristic Generation...

AI Post Transformers ·

2025/11/04

The October 9, 2025 paper from UT Austin paper introduces **PolicySmith**, a novel framework that automates the design of system policies, arguing that the traditional manual creation of heuristics by experts is becoming inefficient due to rapidly changing environments. PolicySmith leverages **Large Language Models (LLMs)** and evolutionary search to generate **instance-optimal heuristic code** that is tailored to specific workloads and hardware contexts. The authors demonstrate the framework's

PolicySmith: Automated Systems Heuristic Generation...

AI Post Transformers ·

2025/11/04

The October 9, 2025 paper from UT Austin paper introduces PolicySmith, a novel framework that automates the design of system policies, arguing that the traditional manual creation of heuristics by experts is becoming inefficient due to rapidly changing environments. PolicySmith leverages Large Language Models (LLMs) and evolutionary search to generate instance-optimal heuristic code that is tailored to specific workloads and hardware contexts. The authors demonstrate the framework's

Samsung: zFLoRA: Zero-Latency Fused Low-Rank Adapters

AI Post Transformers ·

2025/11/04

The October 28, 2025 Samsung research paper introduces zFLoRA (zero-latency fused low-rank adapter), a novel parameter-efficient fine-tuning (PEFT) method designed to address the significant inference latency overheads associated with current adapter methods like LoRA in large language models (LLMs). The core contribution is a carefully engineered fusion of adapter blocks with the base model to achieve zero or negligible latency overhead during inference, leveraging optimized matrix multiplication

MorphKV: Constant-Sized KV Caches for LLM Inference

AI Post Transformers ·

2025/11/04

The June 7, 2025 UT Austin and University of British Colombia collaboration academic paper introduces MorphKV, a novel inference-time technique designed to address the excessive memory consumption caused by Key-Value (KV) caches in Large Language Models (LLMs) during extended responses. The core problem is that KV cache size grows linearly with sequence length, straining GPU memory, leading to prior methods sacrificing accuracy by dropping context or using lossy compression. MorphKV resolves this by

SuperBPE: Space Travel for Language Models

AI Post Transformers ·

2025/11/04

The August 26, 2025 collaboration between the University of Washington, NVIDIA and the Allen Institute for AI paper introduces "SuperBPE: Space Travel for Language Models," introduces SuperBPE, a novel tokenization method that challenges the standard practice of limiting tokens to subword boundaries. The authors argue that conventional Byte-Pair Encoding (BPE) is inefficient because it cannot create "superword" tokens that bridge whitespace, ignoring common multi-word expressions

Google: Supervised Reinforcement Learning for...

AI Post Transformers ·

2025/11/04

The October 29 2025 Google research paper introduces Supervised Reinforcement Learning (SRL), a novel framework designed to improve the complex, multi-step reasoning abilities of large language models (LLMs). The core issue addressed is that conventional training methods like Supervised Fine-Tuning (SFT) and outcome-based Reinforcement Learning with Verifiable Rewards (RLVR) struggle with difficult problems because they either overfit rigid expert paths or receive only sparse, uninformative final

HALoS: Hierarchical Asynchronous LLM Training over...

AI Post Transformers ·

2025/11/04

The June 5, 2025 research paper introducing HALoS: Hierarchical Asynchronous Local SGD, a novel optimization framework designed for training large language models (LLMs) across geographically distributed accelerators and slow, high-latency networks. The core challenge addressed is the inefficiency of standard synchronous training methods due to slow inter-region communication and heterogeneous hardware speeds. HALoS mitigates these issues through a two-tier architecture featuring local parameter

Gumbel-Softmax for Differentiable Categorical...

AI Post Transformers ·

2025/11/04

These two papers (years 2017, 2022) introduce and then apply the Gumbel-Softmax distribution as a differentiable gradient estimator for categorical and discrete latent variables in neural networks. The first paper, "Categorical Reparameterization with Gumbel-Softmax," proposes this distribution to address the challenge of backpropagating through non-differentiable sampling operations, demonstrating its effectiveness in tasks like structured output prediction and generative modeling, where

Anchored Diffusion Language Model: Superior...

AI Post Transformers ·

2025/11/04

The May 24, 2025 UT Austin paper introduces the Anchored Diffusion Language Model (ADLM), a novel approach that aims to improve discrete language modeling by addressing the limitations of traditional Autoregressive (AR) and standard Diffusion Language Models (DLMs). AR models, such as GPT-3, generate text sequentially and struggle with complex reasoning, while existing DLMs, which use iterative masked-token prediction, lag behind AR models in quality. ADLM enhances DLMs by incorporating anchor

RetNet: Retentive Networks: Transformer Successor for...

AI Post Transformers ·

2025/11/02

The August 9, 2023 paper introduces the Retentive Network (RetNet), a proposed foundational architecture for large language models intended to succeed the Transformer model. RetNet aims to overcome the Transformer's inefficiencies during inference by simultaneously achieving training parallelism, low-cost inference, and strong performance, a combination previously considered an "impossible triangle." The core of RetNet is the retention mechanism, which supports three computation

Kimi Linear: Efficient Expressive Attention Architecture

AI Post Transformers ·

2025/11/02

The October 30, 2025 technical report details the development and evaluation of Kimi Linear, a novel hybrid linear attention architecture for large language models (LLMs). The core innovation is the Kimi Delta Attention (KDA) module, which refines existing linear attention mechanisms to achieve superior performance and efficiency compared to traditional full attention, particularly in long-context scenarios. Empirical results from extensive pretraining and fine-tuning experiments demonstrate that

00:13:08

RetNet: Retentive Networks: Transformer Successor for...

AI Post Transformers ·

2025/11/02

The August 9, 2023 paper introduces the **Retentive Network (RetNet)**, a proposed foundational architecture for large language models intended to succeed the **Transformer** model. RetNet aims to overcome the Transformer's inefficiencies during inference by simultaneously achieving **training parallelism**, **low-cost inference**, and **strong performance**, a combination previously considered an "impossible triangle." The core of RetNet is the **retention mechanism**, which supports three

00:15:26

Kimi Linear: Efficient Expressive Attention Architecture

AI Post Transformers ·

2025/11/02

The October 30, 2025 **technical report** details the development and evaluation of **Kimi Linear**, a novel **hybrid linear attention architecture** for large language models (LLMs). The core innovation is the **Kimi Delta Attention (KDA)** module, which refines existing linear attention mechanisms to achieve superior performance and efficiency compared to traditional full attention, particularly in **long-context scenarios**. Empirical results from extensive pretraining and fine-tuning experiments

00:12:02

ALiBi: Attention with Linear Biases Enables Length...

AI Post Transformers ·

2025/11/01

The April 22, 2022 collaboration between University of Washington, Facebook AI and the Allen Institute for AI introduces Attention with Linear Biases (ALiBi), a novel and efficient method for position representation in transformer models that effectively addresses the challenge of **extrapolation**—a model's ability to maintain performance on input sequences longer than those used during training. The authors demonstrate that traditional position encoding methods, like sinusoidal embeddings, fail to

ALiBi: Attention with Linear Biases Enables Length...

AI Post Transformers ·

2025/11/01

The April 22, 2022 collaboration between University of Washington, Facebook AI and the Allen Institute for AI introduces Attention with Linear Biases (ALiBi), a novel and efficient method for position representation in transformer models that effectively addresses the challenge of extrapolation—a model's ability to maintain performance on input sequences longer than those used during training. The authors demonstrate that traditional position encoding methods, like sinusoidal embeddings, fail

00:12:29

Quest: Query-Aware Sparsity for Efficient LLM Inference

AI Post Transformers ·

2025/10/31

The August 26, 2024 academic paper introduces **Quest**, a novel algorithm designed to improve the inference efficiency of **Long-Context Large Language Models (LLMs)** by addressing the costly self-attention process caused by a large Key-Value (KV) cache. Quest utilizes **Query-Aware Sparsity** to dynamically identify and select only the **critical KV cache pages** based on the current query token, which significantly reduces the required memory movement during decoding. Unlike previous

00:17:17

Flash-LLM: Efficient LLM Inference with Unstructured...

AI Post Transformers ·

2025/10/31

The September 19, 2025 Alibaba paper introduces **Flash-LLM**, a novel software framework designed to enable **cost-effective and highly-efficient inference for large generative models** by supporting unstructured sparsity on high-performance tensor cores. The authors observe that the primary bottleneck in large language model (LLM) inference is the memory bandwidth limitation during "skinny" matrix multiplications, rather than the arithmetic processing of tensor cores. Flash-LLM addresses this

990 results

AI Post Transformers

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs