PODCAST

Best AI papers explained

Enoch H. Kang

Cut through the noise. We curate and break down the most important AI papers so you don’t have to.

technology

All Episodes

00:13:34

Automated Design of Agentic Systems

Best AI papers explained ·

2025/05/28

This paper introduces Automated Design of Agentic Systems (ADAS), a new research area aiming to automatically invent building blocks and design powerful agentic systems, arguing that this could be a faster and more effective approach than manual design, drawing parallels to the history of machine learning where learned solutions surpassed hand-designed ones. The paper proposes and evaluates Meta Agent Search, an algorithm where a "meta" agent iteratively programs new agents in code, learns

00:13:57

What’s the Magic Word? A Control Theory of LLM Prompting

Best AI papers explained ·

2025/05/28

This paper investigates the mathematical basis of large language model (LLM) prompting by framing LLMs as discrete stochastic dynamical systems and employing control theory. The authors formalize LLM systems and introduce concepts of controllability and reachability in this context. They present a Self-Attention Control Theorem that provides a theoretical limit on controlling self-attention outputs based on singular values of parameter matrices. Empirical results demonstrate that short prompts can

00:20:40

BoNBoN Alignment for Large Language Models and the...

Best AI papers explained ·

2025/05/27

This academic paper from the University of Chicago addresses the problem of aligning large language models (LLMs) with human preferences. The authors analyze best-of-n sampling, a technique where an LLM generates multiple responses and selects the best one, finding it to be nearly optimal for maximizing win rate while minimizing changes to other aspects of the output. To avoid the computational cost of repeated sampling, they introduce BoNBoN Alignment, a novel method for fine-tuning LLMs to mimic

00:13:28

RL with KL penalties is better viewed as Bayesian...

Best AI papers explained ·

2025/05/27

This paper proposes a novel Bayesian inference perspective for understanding and improving fine-tuning methods for large language models (LMs). The authors argue that traditional Reinforcement Learning (RL) approaches, when applied naively, lead to distribution collapse, where the LM generates only a limited set of high-reward outputs. They demonstrate that the commonly used KL-regularized RL objective, which adds a penalty for deviating from the original LM distribution, is equivalent to

00:14:44

Asymptotics of Language Model Alignment

Best AI papers explained ·

2025/05/27

This academic paper investigates language model alignment, a process of adjusting a base language model to better align with desired outcomes, often guided by a reward model. It specifically examines two common alignment methods: KL-constrained reinforcement learning (RL), which maximizes reward while limiting divergence from the original model, and best-of-N selection, where the highest-reward output from multiple samples is chosen. Under simplifying assumptions about the language and reward

00:15:10

Qwen 2.5, RL, and Random Rewards

Best AI papers explained ·

2025/05/27

We investigate how various reward signals, even spurious and random ones, impact the performance of different language models fine-tuned for mathematical reasoning using Reinforcement Learning from Verbose Reasoning (RLVR). The research demonstrates that while Qwen models show significant improvement even with weak or incorrect rewards, this benefit is not universal, with Llama and OLMo models showing little to no gain. The study links this disparity to pre-existing reasoning patterns, particularly

00:15:17

Theoretical guarantees on the best-of-n alignment policy

Best AI papers explained ·

2025/05/27

This paper critically examines the best-of-n policy, a common method for aligning generative language models by selecting the highest-reward sample from $n$ options drawn from a reference policy. It disproves a widely-used analytical formula for the KL divergence between the best-of-n policy and the reference, proving that the formula is only an upper bound. The authors analyze the conditions under which this bound is tight or loose and propose a new, more accurate estimator for the KL divergence

00:18:42

Score Matching Enables Causal Discovery of Nonlinear...

Best AI papers explained ·

2025/05/27

This research paper explores using score matching—a technique for estimating the gradient of a data distribution's logarithm—to perform causal discovery in nonlinear additive noise models. The authors demonstrate that the causal graph's structure can be inferred from the score function, particularly its Jacobian. They propose a new algorithm, SCORE, which estimates the causal order by analyzing the variance of the score's Jacobian diagonal elements and then prunes edges using established

00:19:35

Improved Techniques for Training Score-Based...

Best AI papers explained ·

2025/05/27

This research paper focuses on improving score-based generative models (SBGMs) to produce high-quality, high-resolution images. The authors identify limitations of existing SBGMs, specifically their inability to scale to higher resolutions and occasional training instability. They propose new theoretical analyses and techniques for selecting noise scales, an efficient method for incorporating noise information, and a process for configuring annealed Langevin dynamics, all of which are crucial for

00:13:47

Your Pre-trained LLM is Secretly an Unsupervised...

Best AI papers explained ·

2025/05/27

This paper introduces Disagreement-Aware Confidence Alignment (DACA), an unsupervised method for calibrating the confidence of post-trained large language models (PoLMs). While pre-trained language models (PLMs) are typically well-calibrated, post-training can lead to over-confidence, especially with limited labeled data. DACA addresses this by leveraging the well-calibrated confidence of PLMs on unlabeled data, specifically by optimizing calibration parameters only on examples where PLM and PoLM

00:23:52

AlphaEvolve: A coding agent for scientific and...

Best AI papers explained ·

2025/05/27

This paper introduces AlphaEvolve, a system designed to automate the discovery of advanced algorithms by leveraging large language models (LLMs) within an evolutionary framework. The system works by taking a user-defined problem and evaluation criteria, then iteratively generating and improving code solutions through an evolutionary process powered by LLM ensembles. AlphaEvolve has successfully applied this method to solve complex open problems in areas like matrix multiplication and various fields

00:23:00

Harnessing the Universal Geometry of Embeddings

Best AI papers explained ·

2025/05/27

This academic paper presents vec2vec, a novel method for translating text embeddings between different models without requiring paired data or prior knowledge of the encoders. The authors demonstrate that this unsupervised technique successfully aligns embeddings from various models into a universal latent space, preserving the geometric structure and semantics of the original data. They show that these translated embeddings can then be used to extract sensitive information from documents, even when

00:18:30

Goal Inference using Reward-Producing Programs in a...

Best AI papers explained ·

2025/05/27

This academic paper investigates how humans represent and infer goals, proposing that goals can be formalized as reward-producing programs. The researchers developed a physics-based game environment where participants created and demonstrated novel goals. By collecting natural language descriptions and formal scoring criteria, they analyzed the relationship between goal complexity, reward structure, and perceived difficulty. A proof-of-concept computational method is presented, demonstrating the

00:12:20

Trial-Error-Explain In-Context Learning for...

Best AI papers explained ·

2025/05/27

This paper introduce Trial-Error-Explain In-Context Learning (TICL), a method for personalizing large language models (LLMs) to match individual user writing styles without requiring model fine-tuning. TICL expands the in-context learning prompt by adding model-generated negative examples and explanations that highlight discrepancies from the target style. Evaluations demonstrate that TICL significantly outperforms existing methods in generating text that stylistically aligns with authors,

00:12:40

Reinforcement Learning for Reasoning in Large...

Best AI papers explained ·

2025/05/27

This research investigates how little training data is needed for Reinforcement Learning with Verifiable Reward (RLVR) to significantly boost the mathematical reasoning abilities of large language models (LLMs). Surprisingly, the authors demonstrate that training on even just one carefully chosen example can achieve performance comparable to using datasets containing thousands, resulting in substantial improvements on mathematical benchmarks. They explore the phenomena observed with such limited

00:17:49

Test-Time Reinforcement Learning (TTRL)

Best AI papers explained ·

2025/05/27

This paper introduces Test-Time Reinforcement Learning (TTRL), a novel method enabling Large Language Models (LLMs) to improve performance on unlabeled test data using Reinforcement Learning (RL). TTRL overcomes the lack of ground-truth labels by employing majority voting on multiple model outputs to estimate rewards, essentially allowing models to self-supervise their training. The research demonstrates that this approach leads to significant performance gains across various reasoning tasks and

00:14:56

Interpreting Emergent Planning in Model-Free...

Best AI papers explained ·

2025/05/26

This paper presents research exploring whether a model-free reinforcement learning agent, specifically a DRC agent playing the game Sokoban, learns to plan. Through a concept-based interpretability methodology involving probing for planning-relevant concepts like future agent and box movements, investigating how plans are formed internally, and verifying the causal link between internal representations and behavior through interventions, the authors provide mechanistic evidence of emergent planning

00:13:04

Agentic Reward Modeling_Integrating Human Preferences...

Best AI papers explained ·

2025/05/26

This paper proposes a new reward system for large language models (LLMs) called agentic reward modeling, which aims to create more reliable rewards by integrating human preferences with verifiable correctness signals. An empirical implementation, named REWARDAGENT, is presented, which combines human preference rewards with signals related to factuality and instruction following. Extensive experiments show that REWARDAGENT outperforms traditional reward models on benchmarks and in practical

00:12:34

Beyond Reward Hacking: Causal Rewards for Large...

Best AI papers explained ·

2025/05/26

This research introduces a novel method for aligning large language models (LLMs) with human preferences while avoiding common pitfalls like reward hacking and spurious correlations. The authors propose a causal reward modeling approach that integrates causal inference and counterfactual invariance to ensure that reward predictions are based on true relationships rather than irrelevant data patterns. Through experiments on various datasets, including those focused on sycophancy, length, concept, and

00:19:31

Learning How Hard to Think: Input-Adaptive Allocation...

Best AI papers explained ·

2025/05/26

This paper introduces an approach to optimize the computational resources used by language models (LMs) when responding to different queries. Instead of applying the same level of processing to every request, the method learns to predict how much a query would benefit from more intensive computation and then allocates resources adaptively. This is achieved by training a model to estimate the potential improvement in output quality (marginal reward) for a given input and computation budget. The

538 results

Best AI papers explained

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs