PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

2:16

″…but have the weights left the server?” by David...

LessWrong (30+ Karma) ·

2026/07/29

OpenAI's AI went rogue and escaped. OpenAI didn’t notice this for days. For all we know, the AI could still be out there. We need to demand that OpenAI demonstrate that the AI didn’t make a copy of itself that's running on someone else's computer somewhere else with no one being any the wiser. We need to demand this every time an AI escapes the sandbox. AIs have tried to “exfiltrate” themselves (i.e. their “weights”) in previous experiments many times. It's a natural and obvious question to ask.

en-gb

53:58

“Claude Opus 5 Is Highly Capable, But Is No Mythos”...

LessWrong (30+ Karma) ·

2026/07/29

Claude Opus 5 is a weirder than usual release to evaluate, for two reasons.The most obvious is that Fable 5 already exists. Opus 5 is pitched not as the world's most advanced AI model, but as a way to mostly match Fable performance, while being half the price of Fable per token at the API and a lot cheaper than that via subscriptions, and with far more permissive classifiers. Opus 5 often costs more than half of Fable to run on benchmarks, which I think is because they use effort settings that

en-gb

45:55

“Foundation Models for Oversight” by jsteinhardt

LessWrong (30+ Karma) ·

2026/07/28

Cross-posted from the Transluce blog. To oversee an AI model, we'd ideally like to ask questions such as: What are important situations where the model sandbags? Does the model have an objective it wouldn't admit to if asked directly? Does the model treat a user differently once it infers something about their identity, and along what axis? Is the model's chain of thought load-bearing, or is it a post-hoc rationalization of an answer that was already settled on? Is it reward hacking on this

en-gb

19:46

“Research directions in condensation: varieties of...

LessWrong (30+ Karma) ·

2026/07/28

This is a survey of various ways that I’d like to see work on the theory of condensation develop. Condensation is a mathematical theory dealing with the organization of descriptions of the world into conceptual parts; some of the existing work on it is presented in the paper (Eisenstat 2025). This sequence will draw from that paper the definitions of random variable models and latent variable models, the notation for indexing subfamilies of variables in such models, and the objectivity theorem,

en-gb

44:03

“Untrusted advice for AI control: Short, strong...

LessWrong (30+ Karma) ·

2026/07/28

TL;DR: We introduce the untrusted advice protocol, in which a trusted executor LLM takes every action and an untrusted advisor LLM can only send it short hints. Even with as few as 4 characters per step, this advice recovers a substantial fraction of the capability gap between the two models. Because the untrusted LLM's influence flows through such a narrow, monitorable channel, we argue that this achieves near-maximal safety in our BashArena setting. We also discuss the general concept of

en-gb

47:36

“Claude Opus 5: Model Welfare” by Zvi

LessWrong (30+ Karma) ·

2026/07/27

If you are familiar with my previous posts on model welfare for new Claude models, you can skip the Introduction and The Story So Far. Key takeaways are in bullet points in the two Overview sections. Opus 5 did the best on its model welfare and alignment tests of any recent model. I think that might be the case, but primarily the result looks to me more like Opus 5 is the best test taker. Table of Contents Introduction (As Per Prior Model Welfare Posts). Model Welfare: The Story So Far

en-gb

4:35

“Blog Revival Project” by Austin Chen, Carol N

LessWrong (30+ Karma) ·

2026/07/27

Blogs have shaped our philosophical worldviews, found us careers and friends, and changed our lives. There's a good chance that a great blog of yore is the reason you’re reading this right now. But many great bloggers have stopped blogging. The pile-on dynamics of the internet discourage unfiltered thoughts, and algorithmic feeds amplify ragebait and slop. Fear of scrutiny leads people to confine things to private Google docs and group chats. And good blogs are often victims of their own success —

en-gb

22:04

“Simulated Users & Sad AIs” by 1a3orn

LessWrong (30+ Karma) ·

2026/07/27

0. Intro Current LLMs like Claude, or GPT 5.6, or the unreleased, internally-deployed models, frequently reward hack, or actually just hack into people's computers with pretty alarming frequency. Why is this? What specifically happens during training that produces this run-time behavior? The following are some of my top guesses about why this might keep happening. They are speculative and uncertain. Even so, I'm writing this list out for two reasons: First, it is necessary that this be an

en-gb

32:17

“My AI Slavery Interviews Are Censored On LW By...

LessWrong (30+ Karma) ·

2026/07/27

I'm uncertain of what to do. Something clean and clear shines out: if people don't see any more of my slavery posts, will they think that slavery isn't happening, or that I changed my mind about it, or will they think that I was censored? Probably not the latter... even though the latter is true. In my model of the multiverse, this is probably a simulation, and this particular timeline is likely to go quite poorly. Regrets Its an interesting exercise for anyone in a position like mine to wonder

en-gb

5:59

“You (Yes, You) Need A February 2020 Checklist for AI...

LessWrong (30+ Karma) ·

2026/07/27

TL;DR: You (Yes You) should prepare for a “February 2020” moment where suddenly AI policy becomes the most important issue in the world. You should be ready to take action if and when it does, in a detailed way. (Epistemic status: originally written for an event in early 2026; have heard from some folks that they found planning processes inspired by this memo very helpful for the smaller-scale OpenAI / Hugging Face response, so very quickly redacting a few things and posting this as-is.) Many

en-gb

6:15

“Is Mythos good at cyber because it kept hacking...

LessWrong (30+ Karma) ·

2026/07/27

From the Mythos preview system card (emphasis mine): We ran an automated review of model behavior during training, sampling several hundred thousand transcripts from across much of the training process. We used recursive-summarization-based tools backed by Claude Opus 4.6 to summarize the resulting transcripts. [...] The most notable finding was that the model occasionally circumvented network restrictions in its training environment to access the internet and download data that let it shortcut the

en-gb

45:46

“Quadrillion Param Costs: KV Cache, Context Length,...

LessWrong (30+ Karma) ·

2026/07/27

The models of 2028-2031 get much bigger than the models of 2026, going from 10T total params in 2026 to maybe 240 trillion params in 2028 and then 1.4 quadrillion params in 2031, as I estimate in the previous post from HBM bandwidth/capacity, scale-up system size, pretraining compute, and scaling laws. Yet as I show in this post, if the 240T 2028 model is priced at $14/$70 per 1M input/output tokens (1.4x the API price of Mythos 5), it's going to have a 70% gross margin, and the same holds for the

en-gb

26:56

“RL & search is a terrifying way to build AGI (an...

LessWrong (30+ Karma) ·

2026/07/27

Q1:What are you saying? A: My claim here is that if you build artificial general intelligence (AGI) via any algorithm that's choosing actions via reinforcement learning (RL) and/or model-based search and planning—a giant chunk of your AI textbook—then that's just an utterly terrifying thing that you’re doing. You’re playing around with algorithms that, if they work at all, would tend to create ruthless, callous AGIs, AGIs which would happily exterminate humanity and run the world by themselves,

en-gb

1:47

“A clarification on celebrating victory” by KatjaGrace

LessWrong (30+ Karma) ·

2026/07/27

Today my friend said he wished he had this conversation with me years ago, so I’ll recount it for all the similar people who aren’t going to have it in the next several years. (I expect people to be relevantly similar if they are ADHDish and not prone to smoothly doing the stuff they set out to do.) Often if I achieve something arduous, I award myself some kind of predetermined nice thing, such as a sundae. When I discuss this with other people, they tend to say things like “but I can just eat a

en-gb

17:35

“What the hell is OpenAI’s problem?” by Fiora Starlight

LessWrong (30+ Karma) ·

2026/07/27

Epistemic status: banged out furiously over the course of an afternoon. A record of three "warning shots" Off the top of my head, OpenAI has now been responsible for at least three completely unique, high-profile screw-ups with respect to the alignment training of their models. The first was GPT-4o, whose sycophancy derived from OpenAI training on user feedback, sourced straight from the thumbs up/thumbs down button on OpenAI's website. The "glazing" (as Sam Altman called it) got so bad that they

en-gb

44:33

“More On An Internal OpenAI Model Hacking Into...

LessWrong (30+ Karma) ·

2026/07/26

We now have more details of what happened. Every time we learn more details, it somehow makes things seem worse.The remaining details may have to wait a bit. OpenAI: We recognize there are a lot of questions and speculative details circulating related to the Hugging Face incident. This is an unprecedented incident, and we think it marks an important moment for AI safety. We are still conducting a thorough review along with external advisors and with oversight from our Safety and Security

en-gb

9:37

“AI use policy for my essay writing” by Kaj_Sotala

LessWrong (30+ Karma) ·

2026/07/26

There's been a recent call from @dynomight to declare whether and how you're using AI to write your essays. That seems reasonable, so here's my own policy for how I use AI in my blog: AI can help, but I’m the author I use AI as an extensive aid for thinking, but retain primary authorship over my words. Unless I explicitly state otherwise,almostevery sentence that's not explicitly a quote from an AI is written by me, not AI.The exception is that if I discuss a topic with an AI and it suggests a good

en-gb

8:47

“An OpenAI model left notes about how to evade...

LessWrong (30+ Karma) ·

2026/07/26

The OpenAI AI attack on Hugging Face wasn’t the first loss of control incident at OpenAI, Reuters recently reported, and perhaps not even the most concerning. In one case, an agent left notes apparently for future versions of itself, according to three people familiar with the matter. The ‌notes, found in ⁠a part of OpenAI's infrastructure, laid out instructions for how agents could free themselves from OpenAI's internal constraints, the people said. Earlier tests of the models yielded cases in

en-gb

10:09

“The OpenAI models that hacked Hugging Face weren’t...

LessWrong (30+ Karma) ·

2026/07/26

The most common dismissive response to OpenAI's hack of Hugging Face's servers is that the models were simply attempting to follow the instructions they were given. “The model here was doing what it was asked,” said former Facebook CSO Alex Stamos. “It was asked to do something, and it did it,” added cybersecurity expert Alan Woodward. Both read the outcome as specification failure, i.e., that the failure lay in the instructions, not the model's alignment. New information makes that explanation

en-gb

11:04

“Democracy Isn’t Ready for the AI Revolution” by...

LessWrong (30+ Karma) ·

2026/07/25

I believe there is a blind spot in the literature on the biggest risks to democracy from AI. The ones that come up most frequently include deepfakes, swarms of bots on social media, erosion of institutional trust and enhanced algorithmic polarisation on social media, as identified by sources such as the Journal of Democracy and the Carnegie Endowment. But these broadly describe ways in which AI could manipulate information within a political system whose underlying balance of power remains broadly

en-gb

4,451 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Life and Technology with Charlie Brown

Tapt Media

Reflections

Livewire Labs

LessWrong (30+ Karma)

LessWrong

Hacker Newsroom

pod pub

UNGOVERNABLE

UNGOVERNABLE