x

75% of people globally want unbiased news (Pew Research) but are concerned they're not getting that.

Get balanced coverage with Symby. See how stories are covered across diverse outlets and spot when one side covers a story the other skips.

PODCAST

Mechanical Dreams

Mechanical Dirk

An automatically generated podcast about machine learning and natural language processing. The two fictional hosts talk about papers that I want to learn more about on my way to work.It"s not good, but it"s useful.

science natural sciences

All Episodes

17:11

The Quantization Model of Neural Scaling

Mechanical Dreams ·

2026/01/15

In this episode: • Introduction: The Mystery of the Straight Line: Professor Norris and Linda introduce the paper 'The Quantization Model of Neural Scaling' by Michaud et al., setting the stage by discussing the ubiquity of power laws in deep learning and the puzzle of why scaling curves are so predictable. • The Quantization Hypothesis: Linda explains the core theory that neural network knowledge is not continuous but composed of discrete, indivisible chunks called 'quanta,' analogous to Max...

en-us

17:52

EAGLE-3

Mechanical Dreams ·

2026/01/14

In this episode: • Introduction: The Wait for Tokens: Professor Norris and Linda introduce the episode's paper, EAGLE-3, and discuss the persistent bottleneck of autoregressive generation costs in modern LLMs. • The Speculative Ceiling: Linda explains how previous speculative sampling methods like EAGLE hit a performance wall where adding more training data failed to improve the draft model, identifying the feature prediction constraint as the culprit. • Innovation: Training-Time Test: A deep...

en-us

17:35

Engram Paper

Mechanical Dreams ·

2026/01/12

In this episode: • The Memory Bottleneck: Professor Norris and Linda introduce the paper 'Conditional Memory via Scalable Lookup' and debate the inefficiency of using expensive neural computation to simulate simple knowledge retrieval. • Engram: N-grams Strike Back: Linda breaks down the 'Engram' module, explaining how it uses hashed N-grams and context-aware gating to inject static embeddings directly into the Transformer backbone. • The U-Shaped Curve of Sparsity: The hosts discuss the 'Spa...

en-us

19:55

From Entropy to Epiplexity- Rethinking Information...

Mechanical Dreams ·

2026/01/09

In this episode: • Introduction: Is Shannon Information Theory Broken?: Professor Norris and Linda introduce the episode, with Norris expressing skepticism about challenging the foundations of information theory. Linda introduces the paper 'From Entropy to Epiplexity' and the premise that traditional theory fails to account for computational bounds. • The Paradox of Deterministic Creation: The hosts discuss the first major paradox: how deterministic processes like AlphaZero or synthetic data ...

en-us

19:42

Completed Hyperparameter Transfer across Modules,...

Mechanical Dreams ·

2026/01/08

In this episode: • Introduction: The Alchemy of Training: Professor Norris and Linda introduce the episode, joking about the 'black art' of hyperparameter tuning before unveiling the paper of the week: 'Completed Hyperparameter Transfer' by researchers at Apple. • Beyond Width: The Limits of muP: Linda explains the background of the Maximal Update Parametrization (muP) and why scaling only across model width isn't enough for modern LLMs, prompting skepticism from Norris about adding more comp...

en-us

19:08

NorMuon- Making Muon more efficient and scalable

Mechanical Dreams ·

2026/01/07

In this episode: • Introduction: The Optimizer Menagerie: Professor Norris and Linda kick off the episode by discussing the explosion of new optimizers in the LLM space. Linda introduces 'NorMuon,' a paper from Georgia Tech and Microsoft that attempts to bridge the gap between the industry standard, AdamW, and the geometric newcomer, Muon. • The Geometry Problem: Why Adam and Muon Fall Short: Linda explains the fundamental trade-off: Adam handles coordinate-wise scaling well but ignores matri...

en-us

18:40

Dion- Distributed Orthonormalized Updates

Mechanical Dreams ·

2026/01/06

In this episode: • The GPU Bill Blues: Professor Norris laments the exorbitant cost of training large models, setting the stage for Linda to introduce the episode's focus: 'Dion: Distributed Orthonormalized Updates' by researchers from Microsoft and Harvard. • Muon's Heavy Lifting: Linda explains the predecessor, the Muon optimizer, and its orthonormalization benefits. Norris questions why a new method is needed, leading to a discussion on how Newton-Schulz iterations become a communication b...

en-us

20:01

How Learning Rate Decay Wastes Your Best Data in...

Mechanical Dreams ·

2026/01/06

In this episode: • Dessert Before Vegetables?: Professor Norris and Linda introduce the concept of Curriculum Learning in LLMs and discuss why the intuitive idea of saving the best data for last has historically failed to produce significant results. • The Invisible antagonist: Learning Rate Decay: Linda reveals the paper's core insight: standard learning rate schedules decay to near-zero just as the high-quality data arrives, effectively wasting the most valuable training tokens. • Signal,...

en-us

12:25

Latent State Models of Training Dynamics

Mechanical Dreams ·

2025/10/28

en-us

13:59

DeepSeek OCR

Mechanical Dreams ·

2025/10/24

en-us

15:18

The Coverage Principle - How Pre-training Enables...

Mechanical Dreams ·

2025/10/23

en-us

13:33

Continual Learning via Sparse Memory Finetuning

Mechanical Dreams ·

2025/10/22

en-us

11:56

Untitled Episode

Mechanical Dreams ·

2025/10/10

en-us

13:44

Characterization and Mitigation of Training...

Mechanical Dreams ·

2025/10/08

en-us

14:19

Demystifying Synthetic Data in LLM Pre-training- A...

Mechanical Dreams ·

2025/10/07

en-us

13:09

Data Mixing Can Induce Phase Transitions in Knowledge...

Mechanical Dreams ·

2025/10/06

en-us

13:10

Apertus Tech Report

Mechanical Dreams ·

2025/09/21

en-us

14:46

Learning Facts at Scale with Active Reading

Mechanical Dreams ·

2025/09/20

en-us

13:50

Fantastic Pretraining Optimizers and Where to Find Them

Mechanical Dreams ·

2025/09/19

en-us

16:17

Benchmarking Optimizers for Large Language Model...

Mechanical Dreams ·

2025/09/19

en-us

99 results

Similar Podcasts

Classic Radio Science Fiction

solgoodmedia.com

Mission to Mars

Inception Point Ai

The 365 Days of Astronomy

365DaysOfAstronomy.org

Creek Devil

william Jevning

UFO Chronicles Podcast

Nik Hunter