PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

36:13

“Open Distillation of Hereditary Traits” by Arthur Conmy

LessWrong (30+ Karma) ·

2026/07/14

TL;DR Josh and Neelshow that distillation from a teacher model to a base pretrained student model transfers some of the teacher model's traits (such as displaying negative emotion in the Gemma Needs Helpevals)On its own this is pretty unsurprising, but Josh and Neel additionally show that even filtering out all the prompts and rollouts where the trait is mentioned doesn’t generally prevent the trait transferIn this post, I show a simple way to replicate and study these phenomena without access to a

en-gb

19:34

“An analysis of AI-generated content at the...

LessWrong (30+ Karma) ·

2026/07/14

Introduction Over the past few years, AI tools have become useful for conducting technical AI research. In the early ChatGPT era (~2023–2024), chat assistants were maybe useful as sounding boards for research ideas, or as editors for polishing a paper draft. In the more recent Claude Code era, competent coding agents can code up and run experiments; with enough direction, they can do much of the technical heavy lifting on a PhD-level research project (Schwartz, 2026); and in well-defined settings,

en-gb

4:37

“Some Quick Thoughts AI 2027” by Tomás B.

LessWrong (30+ Karma) ·

2026/07/14

My biggest problem with AI 2027 is I don't think it is science-fictional enough. That is, towards the end of the scenario seems optimized for respectability over accuracy - here I refer to the "special economic zones" and "robot economy" parts. Their industrial explosion assumes humanoid robots will be building robot factories to build more humanoid robotics factories. This is a respectable assumption and one that is fun to model, but seems likely false to me. I applaud Kokotajlo and his team for

en-gb

31:33

“Prism: Automating Science-of-Evals Research” by...

LessWrong (30+ Karma) ·

2026/07/14

tl;dr – we present [Prism], a scaffold for automating science-of-evals research: work that makes the evaluation the primary object of study. The scaffold provides Claude Code with sub-agents and resources for carrying out scientifically rigorous investigations into eval dynamics and, by extension, model behaviours. We talk through an autonomous Prism run on the Agentic Misalignment setting which demonstrates how minor perturbations to GPT-4.1's prompt cause the model to adopt more indirect methods

en-gb

28:08

“The Flood, by Anton Leicht” by Austin Chen

LessWrong (30+ Karma) ·

2026/07/14

Note: I'm crossposting Anton's newest article from his blog. Anton covers AI policy angles in a singular fashion; every article he writes is worth reading, and this one is unusually topical to Manifund. As someone who went on the ground for Bores, I'd like to see us learn & update from past failures; and how could I not love a phrase like "advance market commitment for PACs"? AI safety money could save American AI politics. As it stands, it won’t. Every year, the Nile floods. For thousands of

en-gb

21:49

“Toy Models of Initialisation Effects on RL Dynamics”...

LessWrong (30+ Karma) ·

2026/07/14

This is a follow-up to two posts Geodesic released last week on our current research direction. The code for generating the figures can be found at this GitHub repository. In our previous post, we outlined Geodesic's focus on what we term the pre-RL alignment checkpoint of models -- the alignment-relevant properties of a model conveyed by pretraining, midtraining, and warm-start SFT, going into heavy RL post-training. In this post, we analyse a toy model of RL learning dynamics, with a particular

en-gb

31:05

“Our response to Séb Krier on Plan A” by MKodama,...

LessWrong (30+ Karma) ·

2026/07/14

This criticism of AI 2040: Plan A by Séb Krier unfortunately seriously mischaracterizes our proposal. It also mostly contains flat assertions, not real argumentation, and the argumentation in it seems quite weak. While we appreciate constructive criticisms of Plan A, such as the ones by Tom Davidson, Richard Ngo, and 1a3orn, we feel the need to correct the issues in Séb's response. First, we’ll go over the specific false representations, and then we’ll give a point-by-point response. False

en-gb

56:29

“Better Call Sol The Workhorse” by Zvi

LessWrong (30+ Karma) ·

2026/07/14

OpenAI's GPT-5.6-Sol is finally here, along with the cheaper Terra and Luna. We’ve seen the early hype as reported on Thursday, but as always that is biased. As usual, the bulk of this is collecting a gestalt based on reactions. I included everything up to a point, but I got a lot of feedback, so after a while I only took the interesting ones. Sol and Fable are both excellent models, sir. They both represent big moves forward. There is room in your workflow for both of them. Sol and Fable are

en-gb

4:03

“Pausing AI at human level seems harder than pausing...

LessWrong (30+ Karma) ·

2026/07/13

Cross-posted from my website. Some people think we should pause AI, but not now. They say we should wait until AI reaches human level, [1] because: It's not (catastrophically) dangerous until after then. Human-level AI will help us do safety research. Alternatively, other people (like me) think we should pause AI as soon as possible. Katja Grace wrote a nice concise case for pausing ASAP. I have something I'd like to add: pausing at human level seems harder than pausing ASAP. [2] Pausing

en-gb

40:53

“It’s 2030 and we fucked up. How did it happen?” by...

LessWrong (30+ Karma) ·

2026/07/13

[Linkpost to this. Loosely based on a lecture I gave in the recursive conference with the same title. Don’t take “2030” literally¹ —it could also be 2035 or 2040. As always, opinions are my own and do not represent OpenAI or Harvard.] It is quite possible that AI will usher in a golden age for humanity. As Amodei writes in “Machines of Loving Grace,” AI might enable the “prevention and treatment of nearly all natural infectious diseases,” the “elimination of most cancer,” “prevention of

en-gb

17:08

“The Whitney Biennial Should Admit That Emilie...

LessWrong (30+ Karma) ·

2026/07/13

content warnings: depictions of human and anthro nudity, discussion of bestiality, modern art Credit where it's due: it is genuinely, unironically baller for the Whitney museum to make the exhibit about how a disabled artist wants to fuck their dog the first one that people see when they attend the prestigious Whitney Biennial, their every-two-year showcase of new and emerging American talents. You know, the one that's supposed to be a barometer of where America is at these days. Unfortunately, not

en-gb

3:24

“The US Government may find it difficult to seize...

LessWrong (30+ Karma) ·

2026/07/13

Epistemic status: conditioning on things I consider unlikely, many undiscussed considerations, not the whole story, etc. I'm not trying to advance any claims about whether this is good or bad, or what to do about it (if anything). I sometimes see concern about loss of most future value as a result of e.g. the US government[1] taking control of the future by seizing control of superintelligence[2] and then having bad values/doing dumb things with it. (A couple random[3] examples, though I feel like

en-gb

3:39

“One-Pager Brief on Pangram Labs” by Sheikh Abdur...

LessWrong (30+ Karma) ·

2026/07/13

Pangram Labs builds the most accurate AI text detector in the world. Team is >25 FTE; they are active on Twitter, you can engage directly, look for "affiliates" tab of @pangram. Here is a table of their performance on adversarially modified AI text (source paper): Language AI Text Detection % Humanized AI Text Detection % GPTZero 95.60% 34.53% Binoculars 94.40% 29.73% Pangram Baseline 100.00% 73.07% Pangram Humanizers (current model!) 100.00% 93.66% Note that "current model!" is not current as

en-gb

7:42

“5 “Plan A” scenarios” by Dave Orr

LessWrong (30+ Karma) ·

2026/07/13

https://ai-2040.com/ is a compelling look at one path forward in a world where the US decides to cooperate with China to slow down transformative AI, to buy us time to do it safely. (Ok, it has a bunch of potential paths forward, but one key path involving cooperation.) To me, the key insight in Plan A is that you need a set of layered defenses, a full stack that covers what you govern (e.g. chips, weights, capabilities, outcomes), how you verify (e.g. hardware, facilities, inspectors), and

en-gb

16:44

“The Human Substitution Test as a Sanity Check for AI...

LessWrong (30+ Karma) ·

2026/07/12

TL;DR: We suggest a sanity check for proposed evaluation or AI oversight schemes: Imagine the AI was replaced by a competent, strategic human — someone who knows they might get evaluated and has their own agenda. Would the evaluation still work? When we apply this mental move broadly, to all AIs and evaluations at once, we get a rather discouraging picture: The questions we care about the most — such as "Is it safe to give this AI more power?" — correspond closely to questions where we already know

en-gb

47:16

“The current bottleneck is political will, not...

LessWrong (30+ Karma) ·

2026/07/12

Abstract: We already know enough to act.I wish we were in a world where research was the bottleneck, but the main constraint on AI safety is no longer a shortage of clever policy ideas: best practices already exist and are not being applied or enforced, and a serious international (or even just national) regulatory regime would probably cut most of the risk.They are not applied because awareness is low.The people who narrate and enforce AI policy mostly do not believe in the problem. I estimate

en-gb

8:40

“Freeing Thucydides” by djbinder

LessWrong (30+ Karma) ·

2026/07/11

Prompted by discussion with Buck Shlegeris and others at the Forethought retreat. The idea that AI could bring an end to Thucydides traps is Buck's. Speculative. I think it's plausible that we will not see sustained competition between actors for control over the future, due to one of two outcomes: If we have AGI, growth and shifts in the distribution of power could become much more rapid. One actor plausibly just outgrows the others and forms a hegemon. If that fails to occur and there is

en-gb

14:33

“Additional Research for Plan A” by Thomas Larsen

LessWrong (30+ Karma) ·

2026/07/11

Yesterday we released AI 2040: Plan A, but there's lots of work left to do. We still have tons of uncertainty about the future of AI and the best strategies for how humanity can successfully chart a path through the intelligence explosion. You can read much of our current thinking on the supplements page. In this post, I'll outline some research areas related to the feasibility/desirability of Plan A that I am excited about further research into. We at AI Futures Project will do some of this

en-gb

15:27

“Plan A’s problem with dry tinder” by Tom Davidson

LessWrong (30+ Karma) ·

2026/07/11

A group is worried about an approaching fire spreading rapidly through their city. They manage to halt the fire outside the city gates. Meanwhile they build massive physical structures to help them study and guide the fire safely. But these structures are all made of highly flammable dry tinder! If they lose control of the fire, it will now rip through the city much more quickly. This is a (flawed!1) analogy for Plan A, AIFP's plan for how the world can safely develop superintelligence. The key

en-gb

11:31

“The easiest pathway to control is through executive...

LessWrong (30+ Karma) ·

2026/07/10

When people in the AI safety community outline loss-of-control scenarios, they often spend a lot of time on relatively elaborate mechanisms — scheming AIs developing nanotech, labs leveraging superintelligence into hard power like drone armies, or bioweapons, or perhaps softer means like massive cyber attacks or mass persuasion. I think this underrates the extent to which power is already highly centralised within the persons of the US President and Chinese General Secretary. They have the easiest

en-gb

4,451 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Life and Technology with Charlie Brown

Tapt Media

Reflections

Livewire Labs

LessWrong (30+ Karma)

LessWrong

Hacker Newsroom

pod pub

UNGOVERNABLE

UNGOVERNABLE