PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

3:43

[Linkpost] “AI Gets IMO Gold Medal: via...

LessWrong (30+ Karma) ·

2025/07/19

This is a link post. I’m excited to share that our latest @OpenAI experimental reasoning LLM has achieved a longstanding grand challenge in AI: gold medal-level performance on the world's most prestigious math competition—the International Math Olympiad (IMO). We evaluated our models on the 2025 IMO problems under the same rules as human contestants: two 4.5 hour exam sessions, no tools or internet, reading the official problem statements, and writing natural language proofs. Why is this a big deal

en-gb

20:47

“A night-watchman ASI as a first step toward a great...

LessWrong (30+ Karma) ·

2025/07/19

I took a week off from my day job of aligning AI to visit Forethought and think about the question: if we can align AI, what should we do with it? This post summarizes the state of my thinking at the end of that week. (The proposal described here is my own, and is not in any way endorsed by Forethought.) Thanks to Mia Taylor, Tom Davidson, Ashwin Acharya, and a whole bunch of other people (mostly at Forethought) for discussion and comments. And a quick note: after writing this, I was told that Eric

en-gb

51:28

“Love stays loved (formerly ‘Skin’)” by Swimmer963...

LessWrong (30+ Karma) ·

2025/07/18

This is a short story I wrote in mid-2022. Genre: cosmic horror as a metaphor for living with a high p-doom. One The last time I saw my mom, we met in a coffee shop, like strangers on a first date. I was twenty-one, and I hadn’t seen her since I was thirteen. She was almost fifty. Her face didn’t show it, but the skin on the backs of her hands did. “I don’t think we have long,” she said. “Maybe a year. Maybe five. Not ten.” It says something about San Francisco, that you can casually talk

en-gb

6:58

“Why it’s hard to make settings for high-stakes...

LessWrong (30+ Karma) ·

2025/07/18

One of our main activities at Redwood is writing follow-ups to previous papers on control like the original and Ctrl-Z, where we construct a setting with a bunch of tasks (e.g. APPS problems) and a notion of safety failure (e.g. backdoors according to our specific definition), then play the adversarial game where we develop protocols and attacks on those protocols. It turns out that a substantial fraction of the difficulty here is developing the settings, rather than developing the protocols and

en-gb

6:37

“Trying the Obvious Thing” by PranavG, Gabriel Alfour

LessWrong (30+ Karma) ·

2025/07/18

I am quite harsh in impersonal settings, such as on the Internet or at work. Attention is a scarce resource, and I am stingy with it. The world is Out To Get Us. In many ways, nice or not. Social Media platforms explicitly try to capture our attention. Children, despite being wonderful, are foremost an attention pit. It's always possible to do more for them. The same goes for People Being Wrong On The Internet or Social Obligations. At large, too many get got, and do not have enough attention left

en-gb

66:59

“Video and transcript of talk on ‘Can goodness...

LessWrong (30+ Karma) ·

2025/07/17

(This is the video and transcript of a public talk I gave at Mox in San Francisco in July 2025, on long-term equilibria post-AGI. It's a longer version of the talk I gave at this workshop. The slides are also available here.) Introduction Thank you. Okay. Hi. Thanks for coming. Aims for this talk So: can goodness compete? It's a classic question, and it crops up constantly in a certain strand of futurism, so I'm going to try to analyze it and understand it more precisely. And in particular I

en-gb

4:35

“On being sort of back and sort of new here” by Loki zen

LessWrong (30+ Karma) ·

2025/07/17

So I'm "back" on Less Wrong, which is to say that I was surprised to find that I already had an account and had even, apparently, commented on some things. 11 years ago. Which feels like half a lifetime ago. More than half a my-adult-lifetime-so-far. A career change and a whole lot of changes in the world ago. I've got a funny relationship to this whole community I guess. I've been 'adj' since forever but I've never been a rat and never, until really quite recently, had much of an interest in the

en-gb

10:07

“Comment on ‘Four Layers of Intellectual...

LessWrong (30+ Karma) ·

2025/07/17

One of the most underrated essays in the post-Sequences era of Eliezer Yudkowsky's corpus is "Four Layers of Intellectual Conversation". The degree to which this piece of wisdom has fallen into tragic neglect in these dark ages of the 2020s may be related to its ephemeral form of publication: it was originally posted as a status update on Yudkowsky's Facebook account on 20 December 2016 and subsequently mirrored on Alyssa Vance's The Rationalist Conspiracy blog, which has since gone offline. (The

en-gb

18:14

“Selective Generalization: Improving Capabilities...

LessWrong (30+ Karma) ·

2025/07/17

Audio note: this article contains 53 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. Ariana Azarbal*, Matthew A. Clarke*, Jorio Cocola*, Cailley Factor*, and Alex Cloud. *Equal Contribution. This work was produced as part of the SPAR Spring 2025 cohort. TL;DR: We benchmark seven methods to prevent emergent misalignment and other forms of misgeneralization using limited alignment data We demonstrate a consistent

en-gb

4:03

“Bodydouble / Thinking Assistant matchmaking” by Raemon

LessWrong (30+ Karma) ·

2025/07/17

I keep meaning to write up a more substantive followup to the Hire (or Become) a Thinking Assistant post. But, I think this still basically the biggest productivity effect size I know of, and more people should be doing it, and seemed worth writing the simple version of the post. I tried out 4 assistants who messaged me after the previous post, and ultimately settled into a rhythm of using one who had availability that matched my needs best. I think all of them were net helpful. So far they've all

en-gb

26:46

“Kimi K2” by Zvi

LessWrong (30+ Karma) ·

2025/07/16

While most people focused on Grok, there was another model release that got uniformly high praise: Kimi K2 from Moonshot.ai.It's definitely a good model, sir, especially for a cheap-to-run open model. It is plausibly the best model for creative writing, outright. It is refreshingly different, and opens up various doors through which one can play. And it proves the value of its new architecture. It is not an overall SoTA frontier model, but it is not trying to be one. The reasoning model

en-gb

2:16

“Chain of Thought Monitorability: A New and Fragile...

LessWrong (30+ Karma) ·

2025/07/15

Twitter | Paper PDF Seven years ago, OpenAI five had just been released, and many people in the AI safety community expected AIs to be opaque RL agents. Luckily, we ended up with reasoning models that speak their thoughts clearly enough for us to follow along (most of the time). In a new multi-org position paper, we argue that we should try to preserve this level of reasoning transparency and turn chain of thought monitorability into a systematic AI safety agenda. This is a measure that improves

en-gb

131:00

“Do confident short timelines make sense?” by TsviBT,...

LessWrong (30+ Karma) ·

2025/07/15

TsviBT Tsvi's context Some context: My personal context is that I care about decreasing existential risk, and I think that the broad distribution of efforts put forward by X-deriskers fairly strongly overemphasizes plans that help if AGI is coming in <10 years, at the expense of plans that help if AGI takes longer. So I want to argue that AGI isn't extremely likely to come in <10 years. I've argued against some intuitions behind AGI-soon in Views on when AGI comes and on strategy to reduce

en-gb

4:12

[Linkpost] “LLM-induced craziness and base rates” by...

LessWrong (30+ Karma) ·

2025/07/15

This is a link post. One billion people use chatbots on a weekly basis. That's 1 in every 8 people on Earth. How many people have mental health issues that cause them to develop religious delusions of grandeur? We don’t have much to go on here, so let's do a very very rough guess with very flimsy data. This study says “approximately 25%-39% of patients with schizophrenia and 15%-22% of those with mania / bipolar have religious delusions.” 40 million people have bipolar disorder and 24 million have

en-gb

3:10

[Linkpost] “Bernie Sanders (I-VT) mentions AI loss of...

LessWrong (30+ Karma) ·

2025/07/14

This is a link post. Most of the interview is about technological unemployment and AI-driven inequality, but here is the part where Sanders talks about loss of control risk (which Gizmodo put as title of their article): Gizmodo: Have you spoken to tech CEOs or experts about this? Sen. Sanders: I have talked to CEOs. Funny that you mention it. I won’t mention his name, but I’ve just gotten off the phone with one of the leading experts in the world on artificial intelligence, two hours ago. Gizmodo:

en-gb

11:14

“Narrow Misalignment is Hard, Emergent Misalignment...

LessWrong (30+ Karma) ·

2025/07/14

Anna and Ed are co-first authors for this work. We’re presenting these results as a research update for a continuing body of work, which we hope will be interesting and useful for others working on related topics.TL;DR We investigate whymodels become misaligned in diverse contexts when fine-tuned on narrow harmful datasets (emergent misalignment), rather than learning the specific narrow task. We successfully train narrowly misaligned modelsusing KL regularization to preserve behavior in other

en-gb

23:55

“Do LLMs know what they’re capable of? Why this...

LessWrong (30+ Karma) ·

2025/07/14

Audio note: this article contains 270 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. This post is a companion piece to a forthcoming paper. This work was done as part of MATS 7.0 & 7.1. Abstract We explore how LLMs’ awareness of their own capabilities affects their ability to acquire resources, sandbag an evaluation, and escape AI control. We quantify LLMs' self-awareness of capability as their accuracy in

en-gb

19:16

“Self-preservation or Instruction Ambiguity?...

LessWrong (30+ Karma) ·

2025/07/14

This is a write-up of a brief investigation into shutdown resistance undertaken by the Google DeepMind interpretability team. TL;DR Why do models sometimes resist shutdown? Are they ignoring instructions to pursue their own agenda – in this case, self-preservation? Or is there a more prosaic explanation? We investigated a specific agentic environment introduced by Palisade Research, where shutdown resistance has previously been reported. By analysing Gemini 2.5 Pro's reasoning, we found the

en-gb

36:47

“How Does Time Horizon Vary Across Domains?” by...

LessWrong (30+ Karma) ·

2025/07/14

Summary In the paper Measuring AI Ability to Complete Long Software Tasks (Kwa & West et al. 2025), METR defined an AI model's 50% time horizon as the length of tasks (measured by how long they take human professionals) that it can complete autonomously with 50% probability. We estimated the time horizon of frontier models released since 2019 on a benchmark combining three sets of software and research tasks ranging from 1 second to 16 hours in length-for-humans (HCAST, RE-Bench, and SWAA,

en-gb

10:31

“xAI’s Grok 4 has no meaningful safety guardrails” by...

LessWrong (30+ Karma) ·

2025/07/14

This article includes descriptions of content that some users may find distressing. Testing was conducted on July 10 and 11; safety measures may have changed since then. I’m a longtime lurker who finally decided to make an account. I assume many other people have seen this behavior already, but I’d like to make it known to relevant parties. xAI released Grok 4 on July 9th, positioning it as their most advanced model yet and claiming benchmark leadership across multiple evaluations. They’ve

en-gb

4,487 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Download This Show

ABC Australia

The DroneOn Show

Mike & Kevin

The Good Tech Companies

HackerNoon

TechCrunch Industry News

TechCrunch

Tech News Briefing

Tech News Briefing