PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

3:05

“Emil the Moose” by Martin Sustrik

LessWrong (30+ Karma) ·

2025/10/11

The travels of Emil the Moose since he entered Czechia in mid-June. Moose became extinct in most of Germany around 1000 CE, and in Bohemia, Moravia, Austria, most of southern Poland, and Hungary by the XV. century. It's not clear where exactly Emil comes from, but most likely from Poland, which has a large moose population in the northeast. In early summer 2025, he crossed the Czech border and wandered south through Moravia into Austria. He swam across the Danube near Vienna, then turned west. Along

en-gb

11:06

“Experiments With Sonnet 4.5 Fiction” by Tomás B.

LessWrong (30+ Karma) ·

2025/10/11

Note, this post contains outputs of an LLM. I do not use LLMs in any of my fiction and do not claim this story as my own. I have been having fun writing fiction, and plan to spend whatever time I have left being better than LLMs doing it. I thought I had maybe a year. My initial experiments with Sonnet 4.5 didn't give me a good opinion of its writing ability. This morning I put everything I have written into its context window and then gave it this prompt: Try to write a story like these but focus

en-gb

8:12

“The Most Common Bad Argument In These Parts” by J...

LessWrong (30+ Karma) ·

2025/10/11

I've noticed an antipattern. It's definitely on the dark pareto-frontier of "bad argument" and "I see it all the time amongst smart people". I'm confident it's the worst, common argument I see amongst rationalists and EAs. I don't normally crosspost to the EA forum, but I'm doing it now. I call it Exhaustive Free Association. Exhaustive Free Association is a step in a chain of reasoning where the logic goes "It's not A, it's not B, it's not C, it's not D, and I can't think of any more things it

en-gb

14:20

“Iterated Development and Study of Schemers (IDSS)”...

LessWrong (30+ Karma) ·

2025/10/10

In a previous post, we discussed prospects for studying scheming using natural examples. In this post, we'll describe a more detailed proposal for iteratively constructing scheming models, techniques for detecting scheming, and techniques for preventing scheming. We'll call this strategy Iterated Development and Study of Schemers (IDSS). We'll be using concepts from that prior post, like the idea of trying to make schemers which are easier to catch.Two key difficulties with using natural examples

en-gb

7:41

“Training fails to elicit subtle reasoning in current...

LessWrong (30+ Karma) ·

2025/10/10

While recent AI systems achieve strong performance through human-readable reasoning that should be simple to monitor (OpenAI, 2024, Anthropic, 2025), we investigate whether models can learn to reason about malicious side tasks while making that reasoning appear benign. We find that Sonnet 3.7 can learn to evade either a reasoning monitor, by persuading the monitor that a blatant backdoor is benign, or an output-only monitor, by devising sophisticated backdoors that the output-only monitor doesn’t

en-gb

6:42

“‘Yes, and—’ Requires the Possibility of ‘No,...

LessWrong (30+ Karma) ·

2025/10/10

Scott Garrabrant gives a number of examples to illustrate that "Yes Requires the Possibility of No". We can understand the principle in terms of information theory. Consider the answer to a yes-or-no question as a binary random variable. The "amount of information" associated with a random variable is quantified by the entropy, the expected value of the negative logarithm of the probability of the outcome. If we know in advance of asking that the answer to the question will always be Yes, then the

en-gb

5:48

“Stars are a rounding error” by Algon

LessWrong (30+ Karma) ·

2025/10/10

Notes on some interesting factoids I learnt from Anders Sandberg's draft book, Grand Futures. "Starlight is heavier than worlds" - Anders Sandberg Looking at the energy density of stuff in the universe, we find a few surprising, and not so surprising, facts. First, the obvious: baryonic matter itself is a rounding error, contributing 4.5% of the energy of the universe. Nine tenths of those sweet, sweet baryonic numéraire are stuck in the warm plasma floating between galaxies. About half the

en-gb

10:20

“Training Qwen-1.5B with a CoT legibility penalty” by...

LessWrong (30+ Karma) ·

2025/10/10

Audio note: this article contains 102 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. I tried training Qwen2.5-1.5B with RL on math to both get correct answers and have a CoT that doesn’t look like human-understandable math reasoning. RL sometimes succeeds at hacking my monitor, and when I strengthen my monitor, it fails at finding CoT that are both illegible and helpful, even after training for roughly 4000

en-gb

7:28

“At odds with the unavoidable meta-message” by Ruby

LessWrong (30+ Karma) ·

2025/10/10

It is a truism known to online moderators[1] that when two commenters are going back and forth in heated exchange, and one lays out rejoinders in paragraph after paragraph of dense text, then two things will have happened: Our careful communicator may or may not have succeeded at conveying well-reasoned insights hitherto unknown by his interlocutor that will change her mind. He will have communicated my hatred for you is at least this long. In all seriousness, words require effort and effort

en-gb

17:35

“Towards a Typology of Strange LLM Chains-of-Thought”...

LessWrong (30+ Karma) ·

2025/10/09

Intro LLMs being trained with RLVR (Reinforcement Learning from Verifiable Rewards) start off with a 'chain-of-thought' (CoT) in whatever language the LLM was originally trained on. But after a long period of training, the CoT sometimes starts to look very weird; to resemble no human language; or even to grow completely unintelligible. Why might this happen? I've seen a lot of speculation about why. But a lot of this speculation narrows too quickly, to just one or two hypotheses. My intent is

en-gb

6:10

“I take antidepressants. You’re welcome” by Elizabeth

LessWrong (30+ Karma) ·

2025/10/09

It's amazing how much smarter everyone else gets when I take antidepressants. It makes sense that the drugs work on other people, because there's nothing in me to fix. I am a perfect and wise arbiter of not only my own behavior but everyone else's, which is a heavy burden because some of ya’ll are terrible at life. You date the wrong people. You take several seconds longer than necessary to order at the bagel place. And you continue to have terrible opinions even after I explain the

en-gb

21:54

“Realistic Reward Hacking Induces Different and...

LessWrong (30+ Karma) ·

2025/10/09

TL;DR: I made a dataset of realistic harmless reward hacks and fine-tuned GPT-4.1 on it. The resulting models don't show emergent misalignment on the standard evals, but they do alignment fake (unlike models trained on toy reward hacks), seem more competently misaligned, are highly evaluation-aware, and the effects persist when mixing in normal data. Thanks to Aidan Ewart, Jack Kaunismaa, Abhay Sheshadri, Maxime Riché, Axel Ahlqvist, Niels Warncke, Daniel Tan, Carolyn Qian, and Kei

en-gb

11:54

“The Thinking Machines Tinker API is good news for AI...

LessWrong (30+ Karma) ·

2025/10/09

Last week, Thinking Machines announced Tinker. It's an API for running fine-tuning and inference on open-source LLMs that works in a unique way. I think it has some immediate practical implications for AI safety research: I suspect that it will make RL experiments substantially easier, and increase the number of safety papers that involve RL on big models. But it's also interesting to me for another reason: the design of this API makes it possible to do many types of ML research without direct

en-gb

18:53

“Hospitalization: A Review” by Logan Riggs

LessWrong (30+ Karma) ·

2025/10/09

I woke up Friday morning w/ a very sore left shoulder. I tried stretching it, but my left chest hurt too. Isn't pain on one side a sign of a heart attack? Chest pain, arm/shoulder pain, and my breathing is pretty shallow now that I think about it, but I don't think I'm having a heart attack because that'd be terribly inconvenient. But it'd also be very dumb if I died cause I didn't go to the ER. So I get my phone to call an Uber, when I suddenly feel very dizzy and nauseous. My wife is on a video

en-gb

8:18

“The Relationship Between Social Punishment and...

LessWrong (30+ Karma) ·

2025/10/09

A punishment is when one agent (the punisher) imposes costs on another (the punished) in order to affect the punished's behavior. In a Society where thieves are predictably imprisoned and lashed, people will predictably steal less than they otherwise would, for fear of being imprisoned and lashed. Punishment is often imposed by formal institutions like police and judicial systems, but need not be. A controversial orator who finds a rock thrown through her window can be said to have been punished

en-gb

13:14

“Spooky Collusion at a Distance with Superrational...

LessWrong (30+ Karma) ·

2025/10/09

TLDR: We found that models can coordinate without communication by reasoning that their reasoning is similar across all instances, a behavior known as superrationality. Superrationality is observed in recent powerful models and outperforms classic rationality in strategic games. Current superrational models cooperate more often with AI than with humans, even when both are said to be rational.Figure 1. GPT-5 exhibits superrationality with itself but classic rationality with humans. GPT-5 is more

en-gb

4:07

“Inoculation prompting: Instructing models to...

LessWrong (30+ Karma) ·

2025/10/08

This is a link post for two papers that came out today: Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time (Tan et al.) Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignment (Wichers et al.) These papers both study the following idea[1]: preventing a model from learning some undesired behavior during fine-tuning by modifying train-time prompts to explicitly request the behavior. We call this technique

en-gb

12:02

“Plans A, B, C, and D for misalignment risk” by...

LessWrong (30+ Karma) ·

2025/10/08

I sometimes think about plans for how to handle misalignment risk. Different levels of political will for handling misalignment risk result in different plans being the best option. I often divide this into Plans A, B, C, and D (from most to least political will required). See also Buck's quick take about different risk level regimes. In this post, I'll explain the Plan A/B/C/D abstraction as well as discuss the probabilities and level of risk associated with each plan. Here is a summary of the

en-gb

9:33

“Irresponsible Companies Can Be Made of Responsible...

LessWrong (30+ Karma) ·

2025/10/08

tl;dr: In terms of financial interests of an AI company, bankruptcy and the world ending are both equally bad. If a company acted in line with its financial interests[1], it would happily accept significant extinction risk for increased revenue. There are plausible mechanisms which would allow a company to act like this even if virtually every employee would prefer the opposite. (For example, selectively hiring people with biased beliefs or exploiting collective action problems.) In particular,

en-gb

8:30

“Replacing RL w/ Parameter-based Evolutionary...

LessWrong (30+ Karma) ·

2025/10/08

I want to highlight this paper (from Sept 29, 2025) of an alternative to RL (for fine-tuning pre-trained LLMs) which: Performs better Requires less data Consistent across seeds Robust (ie don't need to do a grid search on your hyperparameters) Less "Reward Hacking" (ie when optimizing for conciseness, it naturally stays close to the original model ie low KL-Divergence) They claim the magic sauce behind all this is the evolutionary strategy optimizing over distributions of model parameters

en-gb

2,784 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Pantomath

PANTOMATH

What"s New

WIRED

Security, Spoken

WIRED

Ben Wehrman Podcast

Ben Wehrman

MacMagazine no Ar

MacMagazine.com.br