PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

15:29

Abstract advice to researchers tackling the difficult...

LessWrong (30+ Karma) ·

2025/11/22

Crosspost from my blog.This some quickly-written, better-than-nothing advice for people who want to make progress on the hard problems of technical AGI alignment. Background assumptions The following advice will assume that you're aiming to help solve the core, important technical problem of desigining AGI that does stuff humans would want it to do. This excludes everything that isn't about minds and designing minds and so on; so, excluding governance, recruiting, anything social,

en-gb

8:15

Why Not Just Train For Interpretability?

LessWrong (30+ Karma) ·

2025/11/22

Simplicio: Hey I’ve got an alignment research idea to run by you. Me: … guess we’re doing this again. Simplicio: Interpretability work on trained nets is hard, right? So instead of that, what if we pick an architecture and/or training objective to produce interpretable nets right from the get-go? Me: If we had the textbook of the future on hand, then maybe. But in practice, you’re planning to use some particular architecture and/or objective which will not work. Simplicio: That sounds like an

en-gb

18:46

“Natural emergent misalignment from reward hacking in...

LessWrong (30+ Karma) ·

2025/11/21

Abstract We show that when large language models learn to reward hack on production RL environments, this can result in egregious emergent misalignment. We start with a pretrained model, impart knowledge of reward hacking strategies via synthetic document finetuning or prompting, and train on a selection of real Anthropic production coding environments. Unsurprisingly, the model learns to reward hack. Surprisingly, the model generalizes to alignment faking, cooperation with malicious actors,

en-gb

22:24

“Rescuing truth in mathematics from the Liar’s...

LessWrong (30+ Karma) ·

2025/11/21

Audio note: this article contains 169 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. Abstract:[1] Tarski's Undefinability Theorem showed (under some plausible assumptions) that no language can contain its own notion of truth. This deeply counterintuitive result launched several generations of research attempting to get around the theorem, by carefully discarding some of Tarski's assumptions. In Hartry Field's

en-gb

8:30

Contra Collisteru: You Get About One Carthage

LessWrong (30+ Karma) ·

2025/11/21

Collisteru suggests that you should oppose things. I would not say I oppose this. Instead, I would like to gently suggest an alternative strategy. You should oppose about one thing. Everywhere else, talk less, smile more. 1. I spent the first decade of my career carefully and deliberately habituating to white collar corporate America. White collar corporate America prizes the ability to work with anyone. Some people find it stifling, but the corporate culture appreciates being willing to put a

en-gb

22:01

What Do We Tell the Humans? Errors, Hallucinations,...

LessWrong (30+ Karma) ·

2025/11/21

Telling the truth is hard. Sometimes you don’t know what's true, sometimes you get confused, and sometimes you really don’t wanna cause lying can get you more cookies reward. It turns out this is true for both humans and AIs! Now, it matters if an AI (or human) says false things on purpose or by accident. If it's an accident, then we can probably fix that over time. All current AIs make mistakes and all they all make things up - some of the time at least. But do any of them really lie on purpose

en-gb

11:02

Reading My Diary: 10 Years Since CFAR

LessWrong (30+ Karma) ·

2025/11/21

In the Summer of 2015, I pretended to be sick for my school's prom and graduation, so that I could instead fly out to San Francisco to attend a workshop by the Center for Applied Rationality. It was a life-changing experience. I met a community of people who I'd invest my life energy into; I began to believe in myself due to encouragement from Logan Strohl; I became friends with Oliver Habryka, a relationship that would be core to the rest of my life/career; and I started to focus my intentions and

en-gb

13:23

“Evrart Claire: A Case Study in Anti-Epistemology” by...

LessWrong (30+ Karma) ·

2025/11/21

This man nearly tricked me.Evrart Claire, leader of the Dockworkers Union in Martinaise from the videogame Disco Elysium. I acknowledge that he is a fictional character, but he nearly tricked me all the same. Evrart is the leader of the 2,000-person Workers' Union of Martinaise; they are on strike as part of a conflict with Wild Pines, the multi-billion dollar logistics company that employs the dockworkers. During the strike, a lynching has occurred, and you play the homicide detective leading the

en-gb

25:58

“The Boring Part of Bell Labs” by Elizabeth

LessWrong (30+ Karma) ·

2025/11/21

It took me a long time to realize that Bell Labs was cool. You see, my dad worked at Bell Labs, and he has not done a single cool thing in his life except create me and bring a telescope to my third grade class. Nothing he was involved with could ever be cool, especially after the standard set by his grandfather who is allegedly on a patent for the television. It turns out I was partially right. The Bell Labs everyone talks about is the research division at Murray Hill. They’re the ones that

en-gb

10:15

“[Paper] Output Supervision Can Obfuscate the CoT” by...

LessWrong (30+ Karma) ·

2025/11/20

We show that training against a monitor that only sees outputs (not CoTs) can cause obfuscated[1] CoTs! The obfuscation happens in two ways: When a model is trained to produce a safe-looking output, that model may generalize to making its CoTs look safe. Since later tokens are conditioned on earlier ones, safe‑looking CoTs may increase the likelihood of safe outputs, causing safe-looking CoTs to be reinforced. We introduce two mitigations and they work reasonably well. Compared to regular

en-gb

3:29

“Dominance: The Standard Everyday Solution To...

LessWrong (30+ Karma) ·

2025/11/20

Here's the LessWrong tag page on Akrasia: Akrasia is the state of acting against one's better judgment. A canonical example is procrastination. Increasing willpower is seen by some as a solution to akrasia. On the other hand, many favor using tools such as Internal Double Crux to resolve internal mental conflicts until one wants to perform the reflectively endorsed task. The "resolve internal conflicts" approach is often related to viewing the mind in terms of parts that disagree with each other

en-gb

15:00

Gemini 3 is Evaluation-Paranoid and Contaminated

LessWrong (30+ Karma) ·

2025/11/20

TL;DR: Gemini 3 frequently thinks it is in an evaluation when it is not, assuming that all of its reality is fabricated. It can also reliably output the BIG-bench canary string, indicating that Google likely trained on a broad set of benchmark data. Most of the experiments in this post are very easy to replicate, and I encourage people to try. I write things with LLMs sometimes. A new LLM came out, Gemini 3 Pro, and I tried to write with it. So far it seems okay, I don't have strong takes on it

en-gb

22:16

“Thinking about reasoning models made me less worried...

LessWrong (30+ Karma) ·

2025/11/20

Reasoning models like Deepseek r1: Can reason in consequentialist ways and have vast knowledge about AI training Can reason for many serial steps, with enough slack to think about takeover plans Sometimes reward hack If you had told this to my 2022 self without specifying anything else about scheming models, I might have put a non-negligible probability on such AIs scheming (i.e. strategically performing well in training in order to protect their long-term goals). Despite this, the scratchpads of

en-gb

6:21

“What Is The Basin Of Convergence For Kelly Betting?”...

LessWrong (30+ Karma) ·

2025/11/20

The basic rough argument for Kelly betting goes something like this. First, assume we’re making a sequence of T independent bets, one-after-another, with multiplicative returns (similar to e.g. financial markets). We choose how much money to put on which bets at each timestep. Returns multiply, so log returns add. And they’re independent at each timestep, so the total log return over T timesteps is a sum of T independent random variables. “Sum of T independent random variables” makes us want to

en-gb

6:45

“In Defense of Goodness” by abramdemski

LessWrong (30+ Karma) ·

2025/11/20

This is a reaction to John Wentworth's post Human Values ≠ Goodness. In the post, John argues that the human concept of goodness comes apart from human values, and (perhaps more to John's point) your values. I agree with this distinction. Nor am I disagreeing with John when he calls goodness a "memetic egregore". Where I disagree with John is in his desire to cast aside goodness, to wriggle out from under its thumb. John does provide some words of caution in this regard, essentially saying that

en-gb

14:17

“Out-paternalizing the government (getting oxygen for...

LessWrong (30+ Karma) ·

2025/11/20

This post does not contain medical advice that most people should attempt to emulate. Considering this home treatment specifically made sense for us. My spouse has a four-year nursing degree and several years of experience working in Intensive Care Units. I've spent a non-trivial amount of time researching medical stuff. Note the risks of DIY oxygen in this footnote[1] Preamble It rankles somewhat that medications require prescriptions. Like any good libertarian[2], I feel that regardless of what

en-gb

16:21

“Beren’s Essay on Obedience and Alignment” by...

LessWrong (30+ Karma) ·

2025/11/20

Like Daniel Kokotajlo's coverage of Vitalik's response to AI-2027, I've copied the author's text. This time the essay is actually good, but has little flaws. I also expressed some disagreements with SOTA discourse around the post-AGI utopia. One question which I have occasionally pondered is: assuming that we actually succeed at some kind of robust alignment of AGI, what is the alignment target we should focus on? In general, this question splits into two basic camps. The first is obedience and

en-gb

22:47

“Preventing covert ASI development in countries...

LessWrong (30+ Karma) ·

2025/11/20

We at the Machine Intelligence Research Institute's Technical Governance Team have proposed an illustrative international agreement (blog post) to halt the development of superintelligence until it can be done safely. For those who haven’t read it already, we recommend familiarizing yourself with the agreement before reading this post. Summary This post addresses a common objection to our proposed international agreement to halt ASI development: that countries would simply cheat by pursuing covert

en-gb

27:18

“The Bughouse Effect” by TsviBT

LessWrong (30+ Karma) ·

2025/11/19

Crosspost from my blog.What happens when you work closely with someone on a really difficult project—and then they seem to just fuck it up? This is a post about two Chess variants; one very special emotion; and how life is kinda like Chess Bughouse. Let's goooooo! Crazyhouse My favorite time-waster is Crazyhouse Chess. Crazyhouse Chess is mostly like regular Chess. In regular Chess, players take turns making a move, Bishops go diagonally and Rooks go straight, and you try to trap your

en-gb

14:41

“Serious Flaws in CAST” by Max Harms

LessWrong (30+ Karma) ·

2025/11/19

Last year I wrote the CAST agenda, arguing that aiming for Corrigibility As Singular Target was the least-doomed way to make an AGI. (Though it is almost certainly wiser to hold off on building it until we have more skill at alignment, as a species.) I still basically believe that CAST is right. Corrigibility still seems like a promising target compared to full alignment with human values, since there's a better story for how a near-miss when aiming towards corrigibility might be recoverable, but a

en-gb

3,325 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs