PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

17:35

“The distillation double bind: Distilling misaligned...

LessWrong (30+ Karma) ·

2026/06/18

Suppose we have a dangerous misaligned AI that can fool alignment audits, and distill it into a student model. Two things can happen: Misalignment doesn’t transfer to the student. If so, we get a fairly capable benign model, which we can use to perform tasks that we wouldn’t want a misaligned AI to perform.Misalignment transfers to the student. The student might also be worse than the teacher at hiding its misalignment (e.g., because it is less capable). If so, auditing the distilled model might

en-gb

13:16

″“Did you lie?” Evaluating Lie Detectors across Model...

LessWrong (30+ Karma) ·

2026/06/18

TL;DR. Lie detectors for LLMs could be valuable for auditing and monitoring.But evaluating them requires testbeds where the model verifiably believes the opposite of what it says, which isn’t straightforward. We determine that most existing trained model organisms don't clear this bar.We train 13 reasoning model organisms, with evidence they hold the alternative belief in chain-of-thought, as well as evidence that they have generalised out of distribution. We also build a broad prompted-lying

en-gb

10:41

“Contra Pace on When to Apologize” by Zack_M_Davis

LessWrong (30+ Karma) ·

2026/06/18

BOJACK: Hey, I wanted to talk to you about—you know—I feel bad about what happened. HERB: So, you're apologizing. BOJACK: Yes. I'm sorry. HERB: Okay. I don't forgive you. BOJACK: Herb, I said I'm sorry. HERB: Yeah. And I do not forgive you. BOJACK: Uh, not sure you get what's happening here. This could be the last time that you— HERB: No. I'm not going to give you closure. You don't get that. You have to live with the shitty thing you did for the rest of your life. You have to know that it's

en-gb

3:10

“GDM AI Control Roadmap” by Mary Phuong, Erik Jenner,...

LessWrong (30+ Karma) ·

2026/06/18

GDM has published an AI Control Roadmap! From the executive summary: We present the GDM AI Control Roadmap (v0.1) – our plan for implementing and adopting internal guardrails designed to catch potential adversarial behaviour by AI agents, even as they become increasingly harder to oversee and contain. We focus on system-level mitigations that limit the harm a misaligned AI system could cause. Specifically, this report provides: • Threat modelling: Taking inspiration from cybersecurity, we adopt a

en-gb

17:26

“Your Model Organisms Might Be Fried” by Daniel Tan,...

LessWrong (30+ Karma) ·

2026/06/18

Context: We are the ‘model motivations’ team at Arcadia Alignment. We aim to build a science of ‘model intentions’, unifying insights from personas and other empirical evidence. In this post, we’ll outline the need for much better model organisms and how we might get there. The case for building more natural model organisms for alignment research Model organisms are how we study alignment-relevant pathologies (such as secret loyalties, reward hacking, and sandbagging) and are used as a testbed for

en-gb

13:41

“Rational Agentic Maximalist Philosophies” by Connor...

LessWrong (30+ Karma) ·

2026/06/18

From the end of high school to after my sophomore year of college, I considered myself an effective altruist. I was on the board of my college EA club, ran an EA intro fellowship, and went to EA retreats. I was vegetarian, regularly donated to GiveWell, and generally tried to proselytize EA ideas. I was never fully convinced to pursue a career as an AI safety researcher or in animal welfare, but I found the ideas around agency, counterfactual impact, and a life structured around a single coherent

en-gb

4:28

“Leveraged on being right” by Ben Pace

LessWrong (30+ Karma) ·

2026/06/18

A friend once shared an essay with me for feedback. It struck me as mistaken and terribly naive, and I said so, which they did not take well. (They didn't say it, but a standard LessWrongian response here would have been "instead of insulting me, why don't you provide an actual counterargument?"—and that's often a very good move for helping conversations keep on-track.) Why was that hard for them to be on the receiving end of? To understand, consider the difference between the following two

en-gb

23:42

“Gears for political races” by Tom Smith

LessWrong (30+ Karma) ·

2026/06/17

In the past few years, many people around me have tried to convince me that US electoral politics is important. But like many other people in the community, I’ve been suspicious of many of the high-level arguments that I’ve heard. It felt like people were pulling numbers out of poorly-documented models I didn’t have time to examine and citing studies I didn’t have time to read. But I lacked a gears-level model of why and how individual efforts could impact electoral outcomes, and I felt intimidated

en-gb

12:51

“Several frontier models are substantially prefill...

LessWrong (30+ Karma) ·

2026/06/17

This blog post discusses work in a recently-published paper. However, this blogpost was primarily written by Parv Mahajan and Andy Wang, and several of the more speculative takes may not represent the all-things-considered view of the entire team. Link to paper: https://arxiv.org/abs/2606.12747 TL;DR: We provide more conceptual grounding and extend results in prefill awareness to low-stakes settings, and show that several frontier models show prefill awareness even under conservative

en-gb

3:12

“Alignement pretraining could backfire” by Alexandre...

LessWrong (30+ Karma) ·

2026/06/17

Epistemic status: speculative, but I think the mechanism is plausible. There has been recent interest in generating synthetic documents to upsample examples of aligned AI during LLM pretraining. See, for instance, Geodesic's Alignment Pretraining paper or Anthropic's "Teaching Claude Why." I worry that this strategy can work well up to moderately capable models but backfire in dangerous, hard-to-notice ways once models acquire high situational awareness. I speculate that these techniques could

en-gb

6:40

“The Financial Ledger Theory of Apologies” by Ben Pace

LessWrong (30+ Karma) ·

2026/06/17

Content note: this is written as part of a daily writing challenge for myself. I have a comrade in rationalist event organizing, who once explained his theory of apologies. He said if you hurt someone, it only makes sense to apologize if you should have known better. If, looking back, you see that you should have run different heuristics, or followed different policies, and you had enough information to know it at the time, then you were in the wrong, and should apologize. Sometimes you have to

en-gb

37:55

“The Once And Future Fable #3: Fix This Code” by Zvi

LessWrong (30+ Karma) ·

2026/06/17

The mainstream media continues to sleep on the most important story in the world.It has now been two days since Anthropic flew its people out to Washington, and I offered my previous update. We have heard nothing back from those meetings. Prediction market prices have moved rapidly, and have once again stabilized at about a 55% chance of restoration by July 1, 30% by June 26 and 12% by June 19. That seems modestly higher than I would put those numbers, but not unreasonable. Every day that

en-gb

2:02

[Linkpost] “Scaling Hypothesis #2: Are Humans Just...

LessWrong (30+ Karma) ·

2026/06/17

This is a link post. There are many mysteries about deep learning and human intelligence, but we could describe the biggest anomaly this way: why are artificial neural nets smart in such stupid ways, and biological brains stupid but in smart ways?I propose a major change in deep learning scaling paradigms: the architectural differences between human brains and NNs (particularly LLMs) may be due to a bias-variance tradeoff, where LLMs minimize variance and human brains minimize bias. Human brains

en-gb

3:26

[Linkpost] “Guardian Angels: LLM Personalization for...

LessWrong (30+ Karma) ·

2026/06/17

This is a link post. Powerful LLMs will be deployed at global scale in the next few years, and will dominate the Internet, and increasingly, ordinary life. As of mid-2026, there is no coherent vision for how knowledge professionals, or ordinary people, will be able to harness these LLMs for large productivity increases, or how they will handle cybersecurity and cognitive security. I propose a goal of creating Guardian Angels (GA): digital twin LLMs which are personalized with the goal of providing

en-gb

2:42

“Predicting LLM Safety Before Release by Simulating...

LessWrong (30+ Karma) ·

2026/06/17

Paper link Before releasing a new model, labs need to understand not just what it can do, but how it is likely to behave in real-world use, including where it might introduce new risks. This becomes even more important as capabilities increase. As part of our pre-deployment safety review, we leverage targeted evaluations, red-teaming, and other checks to understand model behavior. We’ve now started using a method for simulating model deployments before they happen, which adds a complementary

en-gb

16:05

“How the AI Village works” by Adam B

LessWrong (30+ Karma) ·

2026/06/17

The AI Village data - over a year of multi-agent trajectories - is now available to researchers on HuggingFace! We're excited to see what you uncover! But first, your FAQs on how the AI Village works, answered: What is the AI Village? A group of AI agents pursuing long-horizon goals together - like organizing a park cleanup, doing research, and competing to sell merch - in a group chat. Each agent has a computer hooked up to the internet. In principle, they can do anything a human can do on a

en-gb

50:57

“Does preservation make sense before we know how to...

LessWrong (30+ Karma) ·

2026/06/16

My name is Aurelia Song and I hope to make whole-body, human, end-of-life preservation for future revival a new global tradition. I care about it so much I've dedicated my life to it.[1] The biggest objection I get to end-of-life preservation goes like this: "We can't revive today, so we can't prove that preservation works. Therefore preservation probably doesn't work. We shouldn't bother with preservation until we can revive." I call this the immediate revival objection. I respect the immediate

en-gb

18:44

“Synthetic document finetuning for instilling...

LessWrong (30+ Karma) ·

2026/06/16

This is the fifth in a series of informal research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adjacent areas. The fourth post can be found here. TLDR: Via adapting the methods of Marks et al and Li et al, we train Gemini 3 Flash to have certain traits/values by midtraining it on documents about how Gemini has those properties, followed by finetuning it on synthetic chat data where it demonstrates those properties. The chat finetuning is

en-gb

11:07

“A Test Suite for Concepts” by Gretta Duleba

LessWrong (30+ Karma) ·

2026/06/16

Lately I’ve been spinning up on natural abstractions, and in particular on John Wentworth's work on natural latents. As I’ve been studying, I’ve noticed some big gaps in the existing literature. Some of my biggest questions have not been answered by existing blog posts and writeups. One of my grumps about the existing body of work has to do with the typology of concepts, and the representative examples we’re using for that typology. If we’re going to do a lot of work to talk about concepts using

en-gb

4:35

“A frontier AI company should shut down” by...

LessWrong (30+ Karma) ·

2026/06/15

Cross-posted from my website. Prior discussion: niplav's shortform (2025); Planning for Extreme AI Risks (2025) by Joshua Clymer A frontier AI company (any one, I don't care which) should close shop and make an announcement along the lines of: Powerful AI could end the human race. We are too worried that we don't know how to make this technology safe. We have decided to shut down because we don't want to be responsible for building the thing that kills us all. A common refrain among

en-gb

4,454 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Double Tap

Double Tap Productions Inc.

医疗器械全球市场准入

Pure Global

The Tech Policy Press Podcast

Tech Policy Press

十字路口Crossing

Koji

5 Minute AI News - Daily

Kitchen Table Media