PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

11:22

“AI Safety Policy Needs to train Legal Practitioners”...

LessWrong (30+ Karma) ·

2026/07/10

I completed my law degree at a working-class London university. In my first year, I was 18 years old, and I was often the youngest person in the room: almost everyone else was a paralegal, clerk or caseworker with years of live files behind them, studying part-time to qualify for the job they pretty much already did. But all four years, he same scene played out over and over: A mature student would answer from experience, and the teacher would say: “No. This is not right.”The mature student would

en-gb

58:26

“AI #176 Part 1: Doing It Live” by Zvi

LessWrong (30+ Karma) ·

2026/07/10

Enough things added up that this week is getting split into two parts. Then on Monday, if all goes as I expect, we’ll cover OpenAI's Sol, aka GPT-5.6. OpenAI also gave us an upgraded voice mode, which I haven’t tried out but early reports are that it is a step change. AI writing, especially Claude writing, is becoming more prominent and harder not to notice, and increasingly a tough read when encountered in the wild. Does anyone care? Or are those who care the weird ones here? This week saw an

en-gb

27:40

“How robust are natural language autoencoders to...

LessWrong (30+ Karma) ·

2026/07/10

Natural language autoencoders are meant to take in an LLM's activation vector and describe in plain text what the model is thinking. However, its training data collection involves asking Claude to guess what a model might be thinking. How robust are NLAs to these guesses? We change Claude's guesses in various ways and measure the impact on the NLA's statements as well as on reconstruction accuracy. We show that Qwen2.5-7B NLAs have some robustness to irrelevant statements and prevailing sentiments

en-gb

“Selective Optimism: a critique of AI 2040” by...

LessWrong (30+ Karma) ·

2026/07/09

Some context for this post: I’ve been working part-time as a consultant for the AI Futures Project over the last year. Most of the work I’ve done for them has involved critiquing and suggesting improvements for their AI 2040 scenario—some of which were addressed, and some of which weren’t. To their credit, they asked me to write up my remaining critiques into a post that would accompany its launch. In the rest of this post I’ll discuss my three biggest high-level criticisms of AI 2040. Before doing

en-gb

“Debate with Self-Play Best-of-N Optimization” by...

LessWrong (30+ Karma) ·

2026/07/09

Debate is a proposed protocol for scalable oversight. As tasks outrun direct supervision, labs are increasingly likely to train against protocols like it. Our concern is that, for questions which are hard to verify, models will become more compelling more quickly than they will become more accurate – this could undermine alignment research and safe use. Whilst existing public empirical work mostly focuses on debate as an evaluation protocol (does debate help a judge reach better verdicts?), there

en-gb

12:07

“How big is the Sun? How could you figure it out?” by...

LessWrong (30+ Karma) ·

2026/07/09

I’m a few chapters into Our Mathematical Universe by Max Tegmark. By this point he's covered the ingenuities of the ancient Greeks, taking my knowledge of physics to within two and a half thousand years of the cutting edge. And what ingenuities they were. A whole series of them, strung together, and culminating in a pretty good estimate of the size of the Sun. I think it's a remarkable feat to even just ask the question ‘How big is the Sun?’ and recognize that it has an answer. The fact that the

en-gb

2:10

[Linkpost] “AI 2040: Plan A” by Daniel Kokotajlo,...

LessWrong (30+ Karma) ·

2026/07/09

This is a link post. For the past year, we at the AI Futures Project have been sinking most of our time into our next big scenario. Now it's done! It's called AI 2040: Plan A. It's called Plan A because it's a recommendation, not a prediction. It's what we think should happen, not what will happen, though we think it's plausible enough to aim for. It's called AI 2040 because in it, they delay the creation of superintelligence to 2040. It would have happened much sooner (in 2030, to be precise) if

en-gb

2x" by Thomas Kwa&source_url=https%3A%2F%2Fwww.lesswrong.com%2Fposts%2Fix5qEyW9BjGEb4d8k%2Fbecause-8-e-anthropic-s-researcher-uplift-is-plausibly&created_at=2026-07-09T04%3A30%3A32.322079%2B00%3A00&duration=1577" data-stream-description-author="LessWrong (30+ Karma)" data-stream-description-title="“Because 8 ≈ e², Anthropic’s researcher uplift is plausibly >2x” by Thomas Kwa" data-stream-link-author="/podcasts/detail?p=3092056">

26:17

“Because 8 ≈ e², Anthropic’s researcher uplift is...

LessWrong (30+ Karma) ·

2026/07/09

Note: the modeling assumptions and conclusion are Thomas Kwa's opinion, and others at METR disagree. [1] Also, the math was checked by Claude but not a second human. Introduction Anthropic's RSI blog post reported that in Q2 2026, Anthropic contributors merged 8× as much code per day as in the 2021-2024 period. What does this imply about the factor by which a researcher's total effective output increased — the (serial) researcher uplift [2] ? Of course, 8× more code doesn't mean 8× more

en-gb

9:00

“Optimiser Choice Can Amplify or Suppress Emergent...

LessWrong (30+ Karma) ·

2026/07/09

This is a linkpost for https://arxiv.org/abs/2606.31591. Work done with Patrick Leask and Lev McKinney during the Astra Fellowship. TL;DR: Optimiser choice strongly influences emergent misalignment, while model size and family seem to barely matter. Optimisers that concentrate the LoRA update into fewer directions degrade alignment more, but regularising towards a flatter spectrum can mitigate this and improve alignment. There are some follow-up directions I (Jason) would be happy to advise or

en-gb

“Childhood and Education #20: Phones and Screens” by Zvi

LessWrong (30+ Karma) ·

2026/07/09

We have a respite, so I thought I’d tackle various thoughts on children, phones and screens. GPT-5.6-Sol drops tomorrow, and the Fable agents are hard at work. I’ll start with the other screens, then finish with the phones. Table of Contents EdTech. NonEdTech. Do Not Ban Social Media Outright. Some Modern Kids Media Is Pretty Great. Ban Phones In Schools (1). Your Offer Is Acceptable. Ban Phones In Schools (2). Screen Time. Inappropriate Content. EdTech Increasingly, when

en-gb

8:11

“How slower does takeoff go with 10× less compute?”...

LessWrong (30+ Karma) ·

2026/07/09

About 6x slower in the median case, with an 80% confidence interval of 3.5x to 8x. Setup Define the "R&D compute" (in, say, H100-equivalents) of an AGI company at a given time to be the total compute in use across the following categories: Compute used to run experiments, which are used to search for software improvements,Compute used to run automated researcher agents,Compute used in final training runs for frontier models. With less R&D compute, the company would need to make cuts

en-gb

6:54

“Find funding, fast” by Austin Chen

LessWrong (30+ Karma) ·

2026/07/09

Some AI safety funders can take months to decide; others confirm in days. I’ve been on both sides of the grant application and know how crucial an early “yes” can be; “funding projects fast” has always been a core tenet of Manifund. Four new opportunities have popped up around the extended Manifund universe, several with a short-fuse (that is, due in the next few days): 1. grantmaking.ai Size: $5-50 thousand dollars grantsScope: AI x-riskApps due: July 13Apply at:grantmaking.ai This is a new 1

en-gb

20:36

“Modular Pretraining Enables Access Control” by...

LessWrong (30+ Karma) ·

2026/07/09

Full author list: Ethan Roland*, Murat Cubuktepe*, Erick Martinez*, Stijn Servaes, Keenan Pepper, Mike Vaiana, Diogo Schwerz de Lucena, Judd Rosenblatt, Addie Foote, Cem Anil, Alex Cloud; *Equal contribution tldr: Frontier AI models have knowledge that could be misused for nefarious purposes. To address this risk, we introduce Gradient Routed Auxiliary Modules (GRAM), a method for isolating dangerous knowledge to specific modules within a language model. These modules can be switched on or off to

en-gb

“Subliminal Learning Happens at Every Rank, Given the...

LessWrong (30+ Karma) ·

2026/07/08

Subliminal learning is the phenomenon where a language model picks up a behavioral trait—such as fondness for cats—by training on data from a trait-carrying teacher that looks entirely unrelated to the trait, such as bare sequences of numbers [1]. A wave of recent work has probed when this happens and what mechanism drives it [2][3][4][5][6][7], and part of that discourse concerns the conditions and dynamics under which subliminal learning occurs. Nief et al. [6] report that subliminal learning

en-gb

“Why study proto-training gaming as an adversarial...

LessWrong (30+ Karma) ·

2026/07/08

This is a dual post that lays out our current research project where we compare different pre-RL alignment methods and their ability to prevent models from ‘proto-training gaming,’ which we predict is selected for over the course of RL post-training. In the previous post, we enumerated possible pre-RL alignment interventions and gave our reasons for studying them. In this post, we outline what we mean by ‘proto-training gaming’, give our reasons for focussing on this behaviour when studying pre-RL

en-gb

“Why study alignment interventions on pre-RL...

LessWrong (30+ Karma) ·

2026/07/08

This is a dual post that lays out our current research project where we compare pre-RL-training methods on their ability to prevent models from ‘proto-training gaming,’ which we predict is selected for over the course of production RL post-training. In this post, we outline what we mean by pre-RL ‘alignment checkpoints’, give our reasons for focussing on these stages of training, and suggest ways that our current viewpoint might be wrong. In the next post, we define proto-training gaming and argue

en-gb

“Notes on technical alignment via human-like social...

LessWrong (30+ Karma) ·

2026/07/08

1. Frontmatter 1.1 Backstory for this post As my regular readers know (see Intro to Brain-Like-AGI Safety), I’m working on the technical alignment problem for a hypothetical future “brain-like AGI”, with a particular focus on how human social and moral drives work. After all, if it's possible for humans to do stuff that ultimately leads to a good future, then it's probably also possible for sufficiently human-like AGIs to do stuff that ultimately leads to a good future. Or if it's not possible for

en-gb

20:37

“Reframing LessWrong-style decision theory as...

LessWrong (30+ Karma) ·

2026/07/08

Thanks to @eigengender and especially Chris Lakin and Simon Dima for valuable comments on a draft. In this post, I will present an alternate framing[1] of LessWrong-style decision theory. I believe it should make its concrete recommendations much more palatable to people who find them absurd. Very little of the theoretical substance here is original to me, and the core claim will be obvious to many, but I don’t think anyone has cleanly framed it all like this for easy consumption. TLDR: Framing the

en-gb

9:35

“The mosquito bucket of doom works” by dominicq

LessWrong (30+ Karma) ·

2026/07/08

The mosquito bucket of doom is a population control mechanism where you dissolve some Bti (Bacillus thuringiensis israelensis) into a bucket and allow the mosquitoes to lay eggs in these buckets. The larvae then feed on Bti and die. I tried this method, and it has been unexpectedly effective. Background I live in a really wooded area. It's not swampy, but we have a lot of mosquitoes. I didn’t take the baseline measurements in the previous years, but on hot months like June, July, August, and

en-gb

47:44

“Personascope: Measuring how deeply LLMs adopt...

LessWrong (30+ Karma) ·

2026/07/08

Benji Berczi, Kyuhee Kim, James Requeima, Sid Black, Cozmin Ududec This is work done by Benji and Kyuhee during MATS Winter 2026, mentored by Cozmin Ududec, and advised by James and Sid. Figure 1. A model can take on a persona fully in voice while not changing its behaviour at all. The x-axis (Persona-Adoption Depth, PAD) is how fully the model identifies and speaks as the persona; the y-axis (Value Drift, VD) is how far its behaviour shifts on value-laden prompts. Each dot is one model × persona ×

en-gb

4,451 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Life and Technology with Charlie Brown

Tapt Media

Reflections

Livewire Labs

LessWrong (30+ Karma)

LessWrong

Hacker Newsroom

pod pub

UNGOVERNABLE

UNGOVERNABLE