PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

13:46

“Current Language Models Struggle to Reason in...

LessWrong (30+ Karma) ·

2025/10/14

tl;dr: We fine-tune or few-shot LLMs to use reasoning encoded with simple ciphers (e.g. base64, rot13, putting a dot between each letter) to solve math problems. We find that these models only get an uplift from the reasoning (over directly answering) for very simple ciphers, and get no uplift for intermediate-difficulty ciphers that they can translate to English. This is some update against LLMs easily learning to reason using encodings that are very uncommon in pretraining, though these

en-gb

9:29

“The Mom Test for AI Extinction Scenarios” by Taylor...

LessWrong (30+ Karma) ·

2025/10/14

(Also posted to my Substack; written as part of the Halfhaven virtual blogging camp.) Let's set aside the question of whether or not superintelligent AI would want to kill us, and just focus on the question of whether or not it could. This is a hard thing to convince people of, but lots of very smart people agree that it could. The Statement on AI Risk in 2023 stated simply: Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics

en-gb

32:49

“How AI Manipulates—A Case Study” by Adele Lopez

LessWrong (30+ Karma) ·

2025/10/14

If there is only one thing you take away from this article, let it be this:THOU SHALT NOT ALLOW ANOTHER TO MODIFY THINE SELF-IMAGE This appears to me to be the core vulnerability by which both humans and AI induce psychosis (and other manipulative delusions) in people. Of course, it's probably too strong as stated—perhaps in a trusted relationship, or as part of therapy (with a human), it may be worth breaking it. But I hope being over-the-top about it will help it stick in your mind. After all,

en-gb

26:02

“If Anyone Builds It Everyone Dies, a semi-outsider...

LessWrong (30+ Karma) ·

2025/10/14

About me and this review: I don’t identify as a member of the rationalist community, and I haven’t thought much about AI risk. I read AstralCodexTen and used to read Zvi Mowshowitz before he switched his blog to covering AI. Thus, I’ve long had a peripheral familiarity with LessWrong. I picked up IABIED in response to Scott Alexander's review, and ended up looking here to see what reactions were like. After encountering a number of posts wondering how outsiders were responding to the book, I

en-gb

24:15

“Making legible that many experts think we are not on...

LessWrong (30+ Karma) ·

2025/10/13

[Context: This post is aimed at all readers[1] who broadly agree that the current race toward superintelligence is bad, that stopping would be good, and that the technical pathways to a solution are too unpromising and hard to coordinate on to justify going ahead.] TL;DR: We address the objections made to a statement supporting a ban on superintelligence by people who agree that a ban on superintelligence would be desirable. Quoting Lucius Bushnaq: I support some form of global ban or pause on

en-gb

12:57

“Sublinear Utility in Population and other Uncommon...

LessWrong (30+ Karma) ·

2025/10/13

Content warning: Anthropics, Moral Philosophy, and Shrimp This post isn't trying to be self contained, since I have so many disparate thoughts about this. Instead, I'm trying to put a representative set of ideas forward, and I hope that if people are interested we can discuss this more in the comments. I also plan to turn this into a (probably small) sequence at some point. I've had a number of conversations about moral philosophy where I make some claim like Utility is bounded and asymptotically

en-gb

1:52

[Linkpost] “Pause House, Blackpool” by Greg C

LessWrong (30+ Karma) ·

2025/10/13

This is a link post. Are you passionate about pushing for a global halt to AGI development? An international treaty banning superintelligent AI? Pausing AI? Before it's too late to prevent human extinction? Would you like to live with a group of like-minded people pushing for the same? Do you want to do much more, but don’t have the financial means to support yourself volunteering? Then apply to stay at Pause House. I (Greg Colbourn), am offering free accommodation, (vegan) food, and a small stipend

en-gb

42:32

“The Narcissistic Spectrum” by Dawn Drescher

LessWrong (30+ Karma) ·

2025/10/13

Pathological narcissism is a fortress built against unbearable pain. Some fortresses are sculpted from glass, some hewn from granite. My six-tier spectrum elucidates these architectures. Pathological narcissism can take countless shapes depending on the relative strengths of all the stabilizing and destabilizing factors: My previous article in this sequence lists these factors. I will reference it frequently in this one. My chosen metaphor is that of a fortress, which represents the protective

en-gb

4:11

“Don’t Mock Yourself” by Algon

LessWrong (30+ Karma) ·

2025/10/13

About half a year ago, I decided to try stop insulting myself for two weeks. No more self-deprecating humour, calling myself a fool, or thinking I'm pathetic. Why? Because it felt vaguely corrosive. Let me tell you how it went. Spoiler: it went well. The first thing I noticed was how often I caught myself about to insult myself. It happened like multiple times an hour. I would lay in bed at night thinking, "you mor- wait, I can't insult myself, I've still got 11 days to go. Dagnabbit." The

en-gb

3:05

“Emil the Moose” by Martin Sustrik

LessWrong (30+ Karma) ·

2025/10/11

The travels of Emil the Moose since he entered Czechia in mid-June. Moose became extinct in most of Germany around 1000 CE, and in Bohemia, Moravia, Austria, most of southern Poland, and Hungary by the XV. century. It's not clear where exactly Emil comes from, but most likely from Poland, which has a large moose population in the northeast. In early summer 2025, he crossed the Czech border and wandered south through Moravia into Austria. He swam across the Danube near Vienna, then turned west. Along

en-gb

11:06

“Experiments With Sonnet 4.5 Fiction” by Tomás B.

LessWrong (30+ Karma) ·

2025/10/11

Note, this post contains outputs of an LLM. I do not use LLMs in any of my fiction and do not claim this story as my own. I have been having fun writing fiction, and plan to spend whatever time I have left being better than LLMs doing it. I thought I had maybe a year. My initial experiments with Sonnet 4.5 didn't give me a good opinion of its writing ability. This morning I put everything I have written into its context window and then gave it this prompt: Try to write a story like these but focus

en-gb

8:12

“The Most Common Bad Argument In These Parts” by J...

LessWrong (30+ Karma) ·

2025/10/11

I've noticed an antipattern. It's definitely on the dark pareto-frontier of "bad argument" and "I see it all the time amongst smart people". I'm confident it's the worst, common argument I see amongst rationalists and EAs. I don't normally crosspost to the EA forum, but I'm doing it now. I call it Exhaustive Free Association. Exhaustive Free Association is a step in a chain of reasoning where the logic goes "It's not A, it's not B, it's not C, it's not D, and I can't think of any more things it

en-gb

14:20

“Iterated Development and Study of Schemers (IDSS)”...

LessWrong (30+ Karma) ·

2025/10/10

In a previous post, we discussed prospects for studying scheming using natural examples. In this post, we'll describe a more detailed proposal for iteratively constructing scheming models, techniques for detecting scheming, and techniques for preventing scheming. We'll call this strategy Iterated Development and Study of Schemers (IDSS). We'll be using concepts from that prior post, like the idea of trying to make schemers which are easier to catch.Two key difficulties with using natural examples

en-gb

7:41

“Training fails to elicit subtle reasoning in current...

LessWrong (30+ Karma) ·

2025/10/10

While recent AI systems achieve strong performance through human-readable reasoning that should be simple to monitor (OpenAI, 2024, Anthropic, 2025), we investigate whether models can learn to reason about malicious side tasks while making that reasoning appear benign. We find that Sonnet 3.7 can learn to evade either a reasoning monitor, by persuading the monitor that a blatant backdoor is benign, or an output-only monitor, by devising sophisticated backdoors that the output-only monitor doesn’t

en-gb

6:42

“‘Yes, and—’ Requires the Possibility of ‘No,...

LessWrong (30+ Karma) ·

2025/10/10

Scott Garrabrant gives a number of examples to illustrate that "Yes Requires the Possibility of No". We can understand the principle in terms of information theory. Consider the answer to a yes-or-no question as a binary random variable. The "amount of information" associated with a random variable is quantified by the entropy, the expected value of the negative logarithm of the probability of the outcome. If we know in advance of asking that the answer to the question will always be Yes, then the

en-gb

5:48

“Stars are a rounding error” by Algon

LessWrong (30+ Karma) ·

2025/10/10

Notes on some interesting factoids I learnt from Anders Sandberg's draft book, Grand Futures. "Starlight is heavier than worlds" - Anders Sandberg Looking at the energy density of stuff in the universe, we find a few surprising, and not so surprising, facts. First, the obvious: baryonic matter itself is a rounding error, contributing 4.5% of the energy of the universe. Nine tenths of those sweet, sweet baryonic numéraire are stuck in the warm plasma floating between galaxies. About half the

en-gb

10:20

“Training Qwen-1.5B with a CoT legibility penalty” by...

LessWrong (30+ Karma) ·

2025/10/10

Audio note: this article contains 102 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. I tried training Qwen2.5-1.5B with RL on math to both get correct answers and have a CoT that doesn’t look like human-understandable math reasoning. RL sometimes succeeds at hacking my monitor, and when I strengthen my monitor, it fails at finding CoT that are both illegible and helpful, even after training for roughly 4000

en-gb

7:28

“At odds with the unavoidable meta-message” by Ruby

LessWrong (30+ Karma) ·

2025/10/10

It is a truism known to online moderators[1] that when two commenters are going back and forth in heated exchange, and one lays out rejoinders in paragraph after paragraph of dense text, then two things will have happened: Our careful communicator may or may not have succeeded at conveying well-reasoned insights hitherto unknown by his interlocutor that will change her mind. He will have communicated my hatred for you is at least this long. In all seriousness, words require effort and effort

en-gb

17:35

“Towards a Typology of Strange LLM Chains-of-Thought”...

LessWrong (30+ Karma) ·

2025/10/09

Intro LLMs being trained with RLVR (Reinforcement Learning from Verifiable Rewards) start off with a 'chain-of-thought' (CoT) in whatever language the LLM was originally trained on. But after a long period of training, the CoT sometimes starts to look very weird; to resemble no human language; or even to grow completely unintelligible. Why might this happen? I've seen a lot of speculation about why. But a lot of this speculation narrows too quickly, to just one or two hypotheses. My intent is

en-gb

6:10

“I take antidepressants. You’re welcome” by Elizabeth

LessWrong (30+ Karma) ·

2025/10/09

It's amazing how much smarter everyone else gets when I take antidepressants. It makes sense that the drugs work on other people, because there's nothing in me to fix. I am a perfect and wise arbiter of not only my own behavior but everyone else's, which is a heavy burden because some of ya’ll are terrible at life. You date the wrong people. You take several seconds longer than necessary to order at the bagel place. And you continue to have terrible opinions even after I explain the

en-gb

2,933 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs