PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

2:05

“When should you know the point?” by KatjaGrace

LessWrong (30+ Karma) ·

2026/07/02

Talking to a friend today, she complained about someone wanting her help with a project when that person didn’t even know what the point of the project was. Prima facie that does sound kind of objectionable. But is it? People definitely do a lot of things without much explicit account of the point of each of them. For instance, many people go to university without taking a stance on whether it is primarily fulfills a learning function or a signaling function or an associating-with-elites function

en-gb

1:07

“Conversations With Cade Metz on the Rationalists” by...

LessWrong (30+ Karma) ·

2026/07/02

(Previously, previously.) New York Times reporter Cade Metz has been writing a book about the people who believed in AGI before it was cool. That's a subject that I think I know some things about, so we had some on-the-record conversations in 2025, which I'd like to publish here—for transparency's sake, not because a 32K-word transcript and recording dump is necessarily that interesting. The transcripts below have been edited for clarity and relevant hyperlinks have been inserted. (Hyperlinks in

en-gb

24:35

“Consistency Training while Mitigating Obfuscation...

LessWrong (30+ Karma) ·

2026/07/02

Sohaib Imran, Prakhar Gupta, Jannes Elstner, David Demitri Africa Links: Paper | Code TL;DR. Models condition their behavior on extraneous input features in undesirable ways — for example, on evaluation-likeness (resulting in evaluation gaming), or on the user's preferred answer (resulting in sycophancy). Consistency training teaches a model to behave the same whether or not an extraneous feature/cue is present in the input. Existing methods do this by fine-tuning LLMs to generate responses (BCT)

en-gb

17:32

“Modeling Concepts Probabilistically” by Gretta Duleba

LessWrong (30+ Karma) ·

2026/07/02

As I come up to speed on John Wentworth and David Lorell's work on natural abstraction, I’m filling in some of the gaps in their writing. Previously I posted about a Test Suite for Concepts. Today I’m going to talk about why they use a probabilistic frame when reasoning about concept representation in the minds of agents and the kind of funny way they throw around the word “latent." Working With Concepts So far, we have this very high-level idea of a “concept” in a “mind.” We want to start to think

en-gb

2:10

[Linkpost] “When capabilities work is the *safe* bet”...

LessWrong (30+ Karma) ·

2026/07/02

This is a link post. If you believe that LLMs lend themselves unusually well to alignment compared to other regimes, this can be a very good reason to start doing capability research on them rather than LLM safety research. Imagine you have these beliefs about how AI goes: By I mean the probability that the first ASI is LLM-based (and that it isn't) - the two are mutually exclusive and sum to 100%. Let's imagine you are a super genius, and your effort alone makes something 10% more/less likely

en-gb

5:44

“Green” by Adam Zerner

LessWrong (30+ Karma) ·

2026/07/01

1 Alice: My favorite color is green. Bob: Oh, cool. Mine's red. Months later... Bob: Hey, I got you this green painting. I remember you saying your favorite color is green and I thought you'd like it. Alice: Oh. Um, that's nice of you, but I actually don't really like green. My favorite color's not green — it's blue. Bob: Oh. I coulda sworn you had said green. Alice: Oh, I think I did say green. But, um, Bob... you really shouldn't take things people say so literally all of the time. Bob: Blinks

en-gb

8:13

“A CERN for AI is a distraction; push for an IAEA...

LessWrong (30+ Karma) ·

2026/07/01

TL;DR: There are many conceivable versions of a “CERN for AI.” But the version that seems politically realistic (a new catch-up lab) probably would not do much for safety, while the versions that would materially improve safety (e.g., pause + merge of all companies) are probably unrealistic. So I see the CERN idea as a distraction, and not a particularly neglected one. I argue a better path is an international treaty with red lines now, with an IAEA-style verification body next: a sequencing that

en-gb

13:42

“Model access for third-parties — it’s a big deal!”...

LessWrong (30+ Karma) ·

2026/07/01

Over time, there might be an increasingly large gap between insider model access and outsider model access. By insiders, I mean employees at the frontier lab.[1] By "outsiders", I mean external safety researchers, third-party auditors, and other actors trying to make the future go well. I will call this a model access gap — and when the gap is small, I'll call this model access parity.[2] I think that one of the top priorities for the external AI safety community over the next 6-12 months should be

en-gb

6:39

“You Should Come to The AI Protest” by Ronak_Mehta

LessWrong (30+ Karma) ·

2026/07/01

cart;horse: If you are in the Bay Area on July 11th, even if you're at a company being protested, you should come to The AI Protest. It's fully legal and nonviolent (we'll have a full overtime SFPD escort the entire time), and it's not the worst way you can use your Saturday afternoon that weekend. Plus all of your coolest friends will be there. There's a lot of discussion about the effectiveness of protests and marches; I don't want to re-litigate that here when you can just ask your favorite

en-gb

14:04

“Structural Proxies” by Raymond Douglas

LessWrong (30+ Karma) ·

2026/07/01

Lately I've been thinking a lot about what work would help with actually winning and getting to good worlds. In the spirit of that I decided to venture outside my normal wheelhouse and spend some time reflecting on what technical research could make me more confident about powerful AIs being safe. AGI safety research is tricky partly because we don’t actually have access to the thing we want to study, i.e. superhuman AI. Much of the work we do now is basically trying to lay (potentially irrelevant)

en-gb

3:36

“The consequences of locking intelligence away: an...

LessWrong (30+ Karma) ·

2026/07/01

There has been recent discourse floating around on Hacker News about Chinese API relay stations that use every Western VC-subsidized channel of cheap tokens (think Claude/ChatGPT subscriptions, AWS/Azure credits, Kiro, Google Antigravity, etc.) to resell as APIs to the domestic Chinese market. This is true, as a Chinese citizen that has been seeing an uptick of this trend since mid 2024, but especially since 2025. When I go on Taobao (China's Amazon) and search for keywords, there would be dozens

en-gb

30:28

“The Once And Future Fable #5” by Zvi

LessWrong (30+ Karma) ·

2026/07/01

We, or at least ‘more than 100 American institutions,’ got Mythos back this week. What we the people do not have is Fable or Sol. While we wait for both Claude Fable 5 and GPT-5.6-Sol, today we instead got Claude Sonnet 5. As usual it will take a few days to get a handle on the new model. In this case, Anthropic is representing it as a cheaper and faster version of Opus 4.8, so even though the number says 5 this is a relatively minor development. This post expands the Fable series to cover all

en-gb

5:43

“In partial defence of p(doom)” by Mikhail Samin

LessWrong (30+ Karma) ·

2026/06/30

p(doom) is a shorthand for some important bits and a way to notice a disagreement to double-crux about. If you work on AI capabilities at a frontier AI company, I might ask you for your p(doom). If it's less than 1%, I know that you're probably not familiar with the arguments, or you're maybe dumb in some ways, and will sometimes talk to you about what the situation really is. If it is 80%, I know I should talk to you about the actions people in your position should be taking; we have disagreements

en-gb

22:45

“What Capable Agents Must Know: Why AI Consciousness...

LessWrong (30+ Karma) ·

2026/06/30

[No LLMs were used (or harmed!) in the writing of this blogpost!] Technical results can all be found here: https://arxiv.org/abs/2603.02491 This work, and this post about this work, was borne out of a frustration. The frustration first emerged over a year ago when I was at a Dave & Buster's for the first time in years (for an annual ML department event, no less!), surrounded by flashing lights and NPC agents, not being able to objectively rule out that they were conscious or not, even though I

en-gb

26:28

“Preliminary investigation: KL penalties in RL can...

LessWrong (30+ Karma) ·

2026/06/30

Authors: Satvik Golechha, Sid Black, Joseph Bloom Work done as part of the Model Transparency team at UK AISI. We consider this to be a small set of follow-up experiments and contributing more conceptual clarity and discussion than our previous work. Executive Summary In our recent work replicating MacDiarmid et al. with open models, we informed LLMs about vulnerabilities in a code environment, explicitly asked them to not exploit the hacks, and showed that during RL they learned to reward hack

en-gb

7:34

“Agency is not a natural kind (and why that might...

LessWrong (30+ Karma) ·

2026/06/30

Epistemic status: trying to articulate a big idea which I feel is important but underexplored, partly because it is hard to frame clearly - may not be framing it clearly yet! Agency, both natural and artificial, is very important. Understanding agency allows us to model our own behaviour and that of others, and it is thus one of the most predictively useful concepts we have at our disposal. In its ordinary, folk-psychological sense, agents are ‘like us’ in important behavioural respects, more or

en-gb

35:28

“Human-Guided Agentic Research: A Research Agenda” by...

LessWrong (30+ Karma) ·

2026/06/30

tl;dr: As recursive self-improvement accelerates, we need a top-level agenda to research how to effectively keep humans in the loop. We need to study how humans can best interpret and guide research performed by autonomous agents when those agents lack taste, tacit knowledge or competence, or may try to reward hack, sandbag or sabotage such research. This is one attempt to define the problem and the shape of potential solutions. A Story About the Future of Research Imagine yourself a year or two in

en-gb

26:58

“Destroying the universe: How hard can it be?” by...

LessWrong (30+ Karma) ·

2026/06/29

In quantum field theory, the vacuum state refers to the lowest energy state in a system. Particles are excitations above this state and carry energy, hence the term "vacuum" to refer to the state with no particles. Nothing requires this state to be unique. There may be many different field configurations that are local energy minima, and hence stable against small perturbations. A local minimum that does not globally minimize energy is called a false vacuum. While locally it looks like a stable

en-gb

11:11

“AI will make biological extinction risks worse...

LessWrong (30+ Karma) ·

2026/06/29

An argument goes: If we don't build aligned artificial superintelligence, we risk driving ourselves extinct for some other reason. We should rush to build ASI quickly, in spite of the risks—the longer we wait, the more vulnerable we are to extinction from a different cause. Other than ASI, the biggest extinction risk is synthetic biology. Some lab could (accidentally or on purpose) develop a highly transmissible, 100% fatal super-plague that wipes out humanity. An aligned ASI could stop that from

en-gb

5:53

″$1M AI x-risk grant round is live on grantmaking.ai...

LessWrong (30+ Karma) ·

2026/06/29

TLDR: what is the grant round? grantmaking.ai is launching a 1 million dollars grant round, distributing 5 thousand dollars to 50 thousand dollars per successful application to people and projects working to reduce x-risk from AI. Applications will be reviewed by Gavin Leech, Ryan Kidd, and Marcus Abramovitch. We aim to make all funding decisions by July 28th. Applications submitted by July 13th are guaranteed a priority review. You can still apply after July 13th, and we will make our best effort

en-gb

4,451 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Reflections

Livewire Labs

Netz-Barrierefrei - Digitale Barrierefreiheit verstehen

netz-barrierefrei.de

Life and Technology with Charlie Brown

Tapt Media

LessWrong (30+ Karma)

LessWrong

Hacker Newsroom

pod pub