PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

5:50

[Linkpost] “Reasoning-Finetuning Repurposes Latent...

LessWrong (30+ Karma) ·

2025/07/25

This is a link post. Authors: Jake Ward*, Chuqiao Lin*, Constantin Venhoff, Neel Nanda (*Equal contribution). This work was completed during Neel Nanda's MATS 8.0 Training Phase. TL;DR We computed a steering vector for backtracking using base model activations. It causes the associated fine-tuned reasoning model to backtrack. But, it doesn't cause the base model to backtrack. That's weird! Introduction Reasoning models output Wait, a lot. How did they learn to do this? Backtracking is an emergent

en-gb

11:21

“Building and evaluating alignment auditing agents”...

LessWrong (30+ Karma) ·

2025/07/24

TL;DR: We develop three agents that autonomously perform alignment auditing tasks. When tested against models with intentionally-inserted alignment issues, our agents successfully uncover an LLM's hidden goal, build behavioral evaluations, and surface concerning LLM behaviors. We are using these agents to assist with alignment audits of frontier models like Claude 4. Read the full Anthropic Alignment Science blog post. The X thread and blog post introduction are reproduced below. Thread (Original

en-gb

6:36

“The Whole Check” by JustisMills

LessWrong (30+ Karma) ·

2025/07/24

This is a cross-post from my blog; historically, I've cross-posted about a square rooth of my posts here. First two sections are likely to be familiar concepts to LessWrong readers, though I don't think I've seen their application in the third section before. Polonius and Arbitrage If you’re poor, debt is very bad. Shakespeare says “neither a borrower nor a lender be”, which is probably good advice when money is tight. Don’t borrow, because if circumstances don’t improve you’ll be unable to honor

en-gb

21:29

“‘Behaviorist’ RL reward functions lead to scheming”...

LessWrong (30+ Karma) ·

2025/07/24

1. Introduction & tl;drThis post is basically a 5x-shorter version of Self-dialogue: Do behaviorist rewards make scheming AGIs? (Feb 2025).[1]1.1 tl;dr I will argue that a large class of reward functions, which I call “behaviorist”, and which includes almost every reward function in the RL and LLM literature, are all doomed to eventually lead to AI that will “scheme”—i.e., pretend to be docile and cooperative while secretly looking for opportunities to behave in egregiously bad ways such as

en-gb

1:32

[Linkpost] “A brief perspective from an IMO...

LessWrong (30+ Karma) ·

2025/07/23

This is a link post. I would be somewhat skeptical about any claims suggesting that results have been verified in some form by coordinators. At the closing party, AI company representatives were, disappointingly, walking around with laptops and asking coordinators to evaluate these scripts on-the-spot (presumably so that results could be published quickly). This isn't akin to the actual coordination process, in which marks are determined through consultation with (a) confidential marking schemes*,

en-gb

30:06

“On ‘ChatGPT Psychosis’ and LLM Sycophancy” by jdp

LessWrong (30+ Karma) ·

2025/07/23

As a person who frequently posts about large language model psychology I get an elevated rate of cranks and schizophrenics in my inbox. Often these are well meaning people who have been spooked by their conversations with ChatGPT (it's always ChatGPT specifically) and want some kind of reassurance or guidance or support from me. I'm also in the same part of the social graph as the "LLM whisperers" (eugh) that Eliezer Yudkowsky described as "insane", and who in many cases are in fact insane. This

en-gb

19:53

“Unfaithful chain-of-thought as nudged reasoning” by...

LessWrong (30+ Karma) ·

2025/07/23

This piece is based on work conducted during MATS 8.0 and is part of a broader aim of interpreting chain-of-thought in reasoning models.tl;dr Research on chain-of-thought (CoT) unfaithfulness shows how models’ CoTs mayomit information that is relevant to their final decision. Here, we sketch hypotheses for why key information may be omitted from CoTs: Training regimes could teach LLMs to omit information from their reasoning. But perhaps more importantly, statements within a CoT generally have

en-gb

10:01

“Subliminal Learning: LLMs Transmit Behavioral Traits...

LessWrong (30+ Karma) ·

2025/07/22

Authors: Alex Cloud*, Minh Le*, James Chua, Jan Betley, Anna Sztyber-Betley, Jacob Hilton, Samuel Marks, Owain Evans (*Equal contribution, randomly ordered) tl;dr. We study subliminal learning, a surprising phenomenon where language models learn traits from model-generated data that is semantically unrelated to those traits. For example, a "student" model learns to prefer owls when trained on sequences of numbers generated by a "teacher" model that prefers owls. This same phenomenon can transmit

en-gb

13:35

“Directly Try Solving Alignment for 5 weeks” by Kabir...

LessWrong (30+ Karma) ·

2025/07/22

The Moonshot Alignment Program is a 5-week research sprint from August 2nd to September 6th, focused on the hard part of alignment: finding methods to get an AI to do what we want and not what don't want, which we have strong evidence will scale to superintelligence. You’ll join a small team, choose a vetted research direction, and run experiments to test whether your approach actually generalizes. Mentors include: @Abram Demski @Cole Wyeth Research Assistants include: Leonard Piff, Péter

en-gb

3:08

[Linkpost] “Why Reality Has A Well-Known Math Bias”...

LessWrong (30+ Karma) ·

2025/07/22

This is a link post. I've written up a post offering my take on the "unreasonable effectiveness of mathematics." My core argument is that we can potentially resolve Wigner's puzzle by applying an anthropic filter, but one focused on the evolvability of mathematical minds rather than just life or consciousness. The thesis is that for a mind to evolve from basic pattern recognition to abstract reasoning, it needs to exist in a universe where patterns are layered, consistent, and compounding. In other

en-gb

6:25

“Do ‘adult developmental stages’ theories have any...

LessWrong (30+ Karma) ·

2025/07/22

(This is a comment that has been turned into a post.) I have seen much talk on Less Wrong of “development stages” and “Kegan” and so forth. Naturally I am skeptical; so I do endorse any attempt to figure out if any of this stuff is worth anything. To aid in our efforts, I’d like to say a bit about what might convince me be a little less skeptical. A theory should explain facts; and so the very first thing we’d have to do, as investigators, is figure out if there's anything to explain

en-gb

2:43

“If Anyone Builds It, Everyone Dies: Call for...

LessWrong (30+ Karma) ·

2025/07/21

If Anyone Builds It, Everyone Dies is a book Eliezer and Nate have coming out this September. In our other posts talking about the book, some kind souls have volunteered their services as translators. This post is our explicit call for help of this kind. The book is currently being translated into several different languages (the translation service comes bundled with the book deal for a given language). However, in tandem with the book, MIRI has been working on supplementary materials to be hosted

en-gb

7:17

“HRT in Menopause: A candidate for a case study of...

LessWrong (30+ Karma) ·

2025/07/21

I recently came across a 2024 update on a 2018 book making the still-controversial case that hormone replacement therapy (HRT) after menopause is highly beneficial and that rumors of its risks are unfounded, or at least highly exaggerated. According to their narrative (apparently not contested), HRT started with the physiologically reasonable idea that menopausal symptoms could be treated with estrogens (which at the time were extracted from pregnant horse urine). Early observational

en-gb

0:57

[Linkpost] “GDM also claims IMO gold medal” by Yair...

LessWrong (30+ Karma) ·

2025/07/21

This is a link post. Google DeepMind announces that they've also achieved a gold medal in the IMO. They've exactly matched OpenAI, getting perfect scores for the first 5 questions and flunking the 6th. They're using what sounds like an experimental general version of Gemini which they're then fine tuning for IMO rather than a maths specific model. Their solutions were checked by the IMO (unlike OpenAI) and look much more like a neat little mathematics proof instead of the raw scratchpad that

en-gb

6:48

“[Fiction] Our Trial” by Nina Panickssery

LessWrong (30+ Karma) ·

2025/07/21

As I was making my morning coffee, the words SIMULATION 1099 flashed across my vision. I immediately felt exactly as I'd always imagined I would in philosophical thought experiments. There was a lot to figure out: who are our simulators, where is the simulation running, are there bugs to exploit? I also felt compelled to share my revelation with others. I cornered Mrs Chan at the bus stop. "We're in a simulation!" I announced. "Of course, dear," she replied, not looking up from her phone. "I saw

en-gb

8:37

“LLMs Can’t See Pixels or Characters” by Brendan Long

LessWrong (30+ Karma) ·

2025/07/21

This might be beating a dead horse, but there are several "mysterious" problems LLMs are bad at that all seem to have the same cause. I wanted an article I could reference when this comes up, so I wrote one. LLMs can't count the number of R's in strawberry. LLMs used to be bad at math. Claude can't see the cuttable trees in Pokemon. LLMs are bad at any benchmark that involves visual reasoning. What do these problems all have in common? The LLM we're asking to solve these problems can't see what

en-gb

12:19

“Plato’s Trolley” by dr_s

LessWrong (30+ Karma) ·

2025/07/21

Epistemic status: I'm sure most of my arguments have already been made and argued to death by philosophers that I'm not knowledgeable enough to name. I'm only writing this to outline what I genuinely, spontaneously have come to think on the matter, regardless of how many wheels I may be reinventing. I'm of course interested in knowing about any related debates or arguments I may have missed, so feel free to bring them up in comments if something comes to mind. Context: This is written as a sort of

en-gb

6:05

“Your AI Safety org could get EU funding up to...

LessWrong (30+ Karma) ·

2025/07/20

Thanks to @Manuel Allgaier of AI Safety Berlin for his suggestion to write this post and his helpful feedback. And thanks to LW/AI Alignment Moderator Oliver for looking over the post. LessWrong and AI safety have a unique opportunity: The EU is funding important projects to research AI safety, ranging from AI risk modelling, AI accountability, driverless transport, robotics, to information security and many other topics. Here's the problem: We’re not applying. I work in EU procurement support and

en-gb

3:26

“Shallow Water is Dangerous Too” by jefftk

LessWrong (30+ Karma) ·

2025/07/20

Content warning: risk to childrenJulia and I know drowning is the biggest risk to US children under 5, and we try to take this seriously. But yesterday our 4yo came very close to drowning in a fountain. (She's fine now.) This week we were on vacation with my extended family: nine kids, eight parents, and ten grandparents/uncles/aunts. For the last few years we've been in a series of rental houses, and this time on arrival we found a fountain in the backyard: I

en-gb

23:26

“Make More Grayspaces” by Duncan Sabien (Inactive)

LessWrong (30+ Karma) ·

2025/07/20

Author's note: These days, my thoughts go onto my substack by default, instead of onto LessWrong. Everything I write becomes free after a week or so, but it's only paid subscriptions that make it possible for me to write. If you find a coffee's worth of value in this or any of my other work, please consider signing up to support me; every bill I can pay with writing is a bill I don’t have to pay by doing other stuff instead. I also accept and greatly appreciate one-time donations of any size. 1

en-gb

4,487 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Download This Show

ABC Australia

The DroneOn Show

Mike & Kevin

The Good Tech Companies

HackerNoon

TechCrunch Industry News

TechCrunch

Tech News Briefing

Tech News Briefing