x

75% of people globally want unbiased news (Pew Research) but are concerned they're not getting that.

Get balanced coverage with Symby. See how stories are covered across diverse outlets and spot when one side covers a story the other skips.

PODCAST

AI: post transformers

The transformer architecture revolutionized the world of Neural Networks. It was a springboard for what we know today as modern artificial intelligence. This podcast focuses on modern state of the art research paper reviews starting from the transformer and on.

technology

All Episodes

00:14:57

Random Walk Methods for Graph Learning and Networks

AI: post transformers ·

2025/11/10

We provide a review of the evolution of value of Page Rank to Random Walk with Random Restart and it's application to neural networks focusing on five research papers dating from the original page rank to 2025. They collectively focus on methods for learning on graphs, particularly through the use of **Random Walk Neural Networks (RWNNs)** and related random walk algorithms. One primary source introduces RWNNs, detailing their architecture, which involves a random walk generating a machine-readable

en

00:14:26

AlphaEvolve: Mathematical Discovery at Scale

AI: post transformers ·

2025/11/10

The November 3, 2025 paper provide an overview of the **AlphaEvolve** system, an AI-powered evolutionary approach for mathematical exploration and discovery, often involving large language models (LLMs) to generate and mutate programs that act as search heuristics for finding extremal mathematical objects. The first source is a **GitHub repository** from Google DeepMind that serves as a **collection of 67 mathematical problems** used to test AlphaEvolve, noting that the code for the system itself is

en

00:12:12

AdaFlow: Variance-Adaptive Flow-Based Imitation Learning

AI: post transformers ·

2025/11/10

The November 22, 2024 paper from UT Texas introduces **AdaFlow**, a novel imitation learning framework designed to improve both the efficiency and diversity of policy generation, addressing computational bottlenecks found in previous diffusion-based methods. AdaFlow utilizes **flow-based generative modeling** represented by ordinary differential equations (ODEs) and incorporates a **variance-adaptive ODE solver** that dynamically adjusts the number of inference steps based on the complexity of the

en

00:14:57

zFLoRA: Zero-Latency Fused Low-Rank Adapters

AI: post transformers ·

2025/11/04

The October 28, 2025 Samsung research paper introduces **zFLoRA (zero-latency fused low-rank adapter)**, a novel parameter-efficient fine-tuning (PEFT) method designed to address the significant inference latency overheads associated with current adapter methods like **LoRA** in large language models (LLMs). The core contribution is a carefully engineered fusion of adapter blocks with the base model to achieve **zero or negligible latency overhead** during inference, leveraging optimized matrix

en

00:13:32

SuperBPE: Space Travel for Language Models

AI: post transformers ·

2025/11/04

The August 26, 2025 collaboration between the University of Washington, NVIDIA and the Allen Institute for AI paper introduces **"SuperBPE: Space Travel for Language Models,"** introduces **SuperBPE**, a novel tokenization method that challenges the standard practice of limiting tokens to subword boundaries. The authors argue that conventional **Byte-Pair Encoding (BPE)** is inefficient because it cannot create "superword" tokens that bridge whitespace, ignoring common multi-word expressions that

en

00:15:26

Google: Supervised Reinforcement Learning for...

AI: post transformers ·

2025/11/04

The October 29 2025 Google research paper introduces **Supervised Reinforcement Learning (SRL)**, a novel framework designed to improve the complex, multi-step reasoning abilities of large language models (LLMs). The core issue addressed is that conventional training methods like **Supervised Fine-Tuning (SFT)** and outcome-based **Reinforcement Learning with Verifiable Rewards (RLVR)** struggle with difficult problems because they either overfit rigid expert paths or receive only sparse,

en

00:13:52

MorphKV: Constant-Sized KV Caches for LLM Inference

AI: post transformers ·

2025/11/04

The June 7, 2025 UT Austin and University of British Colombia collaboration academic paper introduces **MorphKV**, a novel inference-time technique designed to address the excessive memory consumption caused by Key-Value (KV) caches in Large Language Models (LLMs) during extended responses. The core problem is that KV cache size grows linearly with sequence length, straining GPU memory, leading to prior methods sacrificing accuracy by dropping context or using lossy compression. MorphKV resolves

en

00:15:40

HALoS: Hierarchical Asynchronous LLM Training over...

AI: post transformers ·

2025/11/04

The June 5, 2025 research paper introducing **HALoS: Hierarchical Asynchronous Local SGD**, a novel optimization framework designed for training large language models (LLMs) across **geographically distributed** accelerators and slow, high-latency networks. The core challenge addressed is the inefficiency of standard synchronous training methods due to **slow inter-region communication** and **heterogeneous hardware speeds**. HALoS mitigates these issues through a **two-tier architecture** featuring

en

00:14:59

Anchored Diffusion Language Model: Superior...

AI: post transformers ·

2025/11/04

The May 24, 2025 UT Austin paper introduces the **Anchored Diffusion Language Model (ADLM)**, a novel approach that aims to improve discrete language modeling by addressing the limitations of traditional **Autoregressive (AR)** and standard **Diffusion Language Models (DLMs)**. AR models, such as GPT-3, generate text sequentially and struggle with complex reasoning, while existing DLMs, which use iterative masked-token prediction, lag behind AR models in quality. ADLM enhances DLMs by incorporating

en

00:18:25

Gumbel-Softmax for Differentiable Categorical...

AI: post transformers ·

2025/11/04

These two papers (years 2017, 2022) introduce and then apply the **Gumbel-Softmax distribution** as a differentiable gradient estimator for **categorical and discrete latent variables** in neural networks. The first paper, "Categorical Reparameterization with Gumbel-Softmax," proposes this distribution to address the challenge of **backpropagating through non-differentiable sampling operations**, demonstrating its effectiveness in tasks like structured output prediction and generative modeling,

en

00:16:11

PolicySmith: Automated Systems Heuristic Generation...

AI: post transformers ·

2025/11/04

The October 9, 2025 paper from UT Austin paper introduces **PolicySmith**, a novel framework that automates the design of system policies, arguing that the traditional manual creation of heuristics by experts is becoming inefficient due to rapidly changing environments. PolicySmith leverages **Large Language Models (LLMs)** and evolutionary search to generate **instance-optimal heuristic code** that is tailored to specific workloads and hardware contexts. The authors demonstrate the framework's

en

00:13:08

RetNet: Retentive Networks: Transformer Successor for...

AI: post transformers ·

2025/11/02

The August 9, 2023 paper introduces the **Retentive Network (RetNet)**, a proposed foundational architecture for large language models intended to succeed the **Transformer** model. RetNet aims to overcome the Transformer's inefficiencies during inference by simultaneously achieving **training parallelism**, **low-cost inference**, and **strong performance**, a combination previously considered an "impossible triangle." The core of RetNet is the **retention mechanism**, which supports three

en

00:15:26

Kimi Linear: Efficient Expressive Attention Architecture

AI: post transformers ·

2025/11/02

The October 30, 2025 **technical report** details the development and evaluation of **Kimi Linear**, a novel **hybrid linear attention architecture** for large language models (LLMs). The core innovation is the **Kimi Delta Attention (KDA)** module, which refines existing linear attention mechanisms to achieve superior performance and efficiency compared to traditional full attention, particularly in **long-context scenarios**. Empirical results from extensive pretraining and fine-tuning experiments

en

00:12:02

ALiBi: Attention with Linear Biases Enables Length...

AI: post transformers ·

2025/11/01

The April 22, 2022 collaboration between University of Washington, Facebook AI and the Allen Institute for AI introduces Attention with Linear Biases (ALiBi), a novel and efficient method for position representation in transformer models that effectively addresses the challenge of **extrapolation**—a model's ability to maintain performance on input sequences longer than those used during training. The authors demonstrate that traditional position encoding methods, like sinusoidal embeddings, fail to

en

00:12:29

Quest: Query-Aware Sparsity for Efficient LLM Inference

AI: post transformers ·

2025/10/31

The August 26, 2024 academic paper introduces **Quest**, a novel algorithm designed to improve the inference efficiency of **Long-Context Large Language Models (LLMs)** by addressing the costly self-attention process caused by a large Key-Value (KV) cache. Quest utilizes **Query-Aware Sparsity** to dynamically identify and select only the **critical KV cache pages** based on the current query token, which significantly reduces the required memory movement during decoding. Unlike previous

en

00:17:17

Flash-LLM: Efficient LLM Inference with Unstructured...

AI: post transformers ·

2025/10/31

The September 19, 2025 Alibaba paper introduces **Flash-LLM**, a novel software framework designed to enable **cost-effective and highly-efficient inference for large generative models** by supporting unstructured sparsity on high-performance tensor cores. The authors observe that the primary bottleneck in large language model (LLM) inference is the memory bandwidth limitation during "skinny" matrix multiplications, rather than the arithmetic processing of tensor cores. Flash-LLM addresses this

en

00:12:31

ELASTIC: Linear Attention for Sequential Interest...

AI: post transformers ·

2025/10/31

The February 12, 2025 KuaiShou Inc paper introduces **ELASTIC**, an Efficient Linear Attention for SequenTial Interest Compression framework designed to address the **scalability issues** of traditional transformer-based sequential recommender systems, which suffer from quadratic complexity with respect to sequence length. ELASTIC achieves this by proposing a **Linear Dispatcher Attention (LDA) layer** that compresses long user behavior sequences into a more compact representation, leading to

en

00:14:01

Anthropic: Introspective Awareness in LLMs

AI: post transformers ·

2025/10/31

On October 29, 2025 Anthropic presented research investigating the existence of **functional introspective awareness** in large language models (LLMs), specifically focusing on Anthropic's Claude models. The core methodology involves using **concept injection**, where researchers manipulate a model's internal activations with representations of specific concepts to see if the model can accurately **report on these altered internal states**. Experiments demonstrate that models can, at times, notice

en

00:14:03

Small Versus Large Models for Requirements...

AI: post transformers ·

2025/10/31

The October 24, 2025 collaboration between many universities have published a paper thst compares the performance of **Large Language Models (LLMs)** and **Small Language Models (SLMs)** on requirements classification tasks within software engineering. Researchers conducted a preliminary study using eight models across three datasets to address concerns about the **high computational cost and privacy risks** associated with using proprietary LLMs. The results indicate that while LLMs achieved an

en

00:17:25

Hyper-Scaling LLM Inference with KV Cache Compression

AI: post transformers ·

2025/10/31

The June 5, 2025 collaboration between University of Edinburgh and Nvidia paper introduces the concept of **inference-time hyper-scaling** for large language models (LLMs), which aims to boost reasoning accuracy by allowing for longer or more parallel token sequences within the same computational budget. The core bottleneck is identified as the size of the key–value (KV) cache, which grows linearly and dominates inference cost. To address this, the authors propose **Dynamic Memory Sparsification

en

340 results

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs