PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

44:59

“HIA and X-risk part 2: Why it hurts” by TsviBT

LessWrong (30+ Karma) ·

2026/01/08

Crosspost from my blog. PDF version. berkeleygenomics.org.Crosspost from my blog. PDF version. berkeleygenomics.org. Context Previously, in "HIA and X-risk part 1: Why it helps", I laid out the reasons I think human intelligence amplification would decrease existential risk from AGI. Here I'll give all the reasons I can currently think of that HIA might plausibly increase AGI X-risk. Questions for the reader Did I miss any important reasons to think that HIA would increase existential risk

en-gb

3:46

“Two Aspects of Situational Awareness: World...

LessWrong (30+ Karma) ·

2026/01/08

I'm writing this post to share some of my thinking about situational awareness, since I'm not sure others are thinking about it this way. For context, I think situational awareness is a critical part of the case for rogue AI and scheming-type risks. But incredibly, it seems to have been absent from any of the written arguments prior to Ajeya's post Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover. Basically, in this post I just want to: Point out

en-gb

32:19

“Public intellectuals need to say what they actually...

LessWrong (30+ Karma) ·

2026/01/08

Intro This Twitter thread from Kelsey Piper has been reverberating around my psyche since its inception, almost six years now. You should read the whole thing for more context, but here are the important tweets:Link 1, link 2, link 3 I really like Kelsey Piper. She's “based” as the kids (and I) say. I think she was trying to do her best by her own lights during this whole episode, and she deserves major props for basically broadcasting her mistakes in clear language to her tens of thousands of

en-gb

16:51

“Broadening the training set should help with...

LessWrong (30+ Karma) ·

2026/01/07

Summary Generalization is one lens on the alignment challenge. We'd like network-based AGI to generalize ethical judgments as well as some humans do. Broadening training is a classic and obvious approach to improving generalization in neural networks. Training sets might be broadened to include decisions like whether to evade human control, how to run the world if the opportunity arises, and how to think about one's self and one's goals. Such training might be useful if it's consistent with

en-gb

10:34

“Mainstream approach for alignment evals is a dead...

LessWrong (30+ Karma) ·

2026/01/07

The problem of evaluation awareness I've taken on the task of making highly realistic alignment evaluations, and I'm now sure that the mainstream approach of creating such evals is a dead end and should change. When we run unrealistic alignment evals, models recognize that they are being evaluated. For example, when Anthropic evaluated Claude Sonnet 4.5 on their alignment tests, they found that the rate of misalignment had dropped to almost 0% compared to 10% for Claude Sonnet 4.1, but the model

en-gb

12:16

“How hard is it to inoculate against misalignment...

LessWrong (30+ Karma) ·

2026/01/07

TL;DR: Simple inoculation prompts that prevent misalignment generalization in toy setups don't scale to more realistic reward hacking. When I fine-tuned models on realistic reward hacks, only prompts close to the dataset generation prompt were sufficient to prevent misalignment. This seems like a specification problem: the model needs enough information to correctly categorize what's being inoculated. I also tried negative inoculation (contextualizing actions as more misaligned), which increases

en-gb

14:33

“The Evolution Argument Sucks” by peralice

LessWrong (30+ Karma) ·

2026/01/06

There is a common argument that AI development is dangerous that goes something like: The “goal” of evolution is to make animals which replicate their genes as much as possible; humans do not want to replicate their genes as much as possible; we have some goal which we want AIs to accomplish, and we develop them in a similar way to the way evolution developed humans; therefore, they will not share this goal, just as humans do not share the goal of evolution. This argument sucks. It has serious,

en-gb

37:45

“How AI Is Learning to Think in Secret” by Nicholas...

LessWrong (30+ Karma) ·

2026/01/06

On Thinkish, Neuralese, and the End of Readable Reasoning In September 2025, researchers published the internal monologue of OpenAI's GPT-o3 as it decided to lie about scientific data. This is what it thought: Pardon? This looks like someone had a stroke during a meeting they didn’t want to be in, but their hand kept taking notes. That transcript comes from a recent paper published by researchers at Apollo Research and OpenAI on catching AI systems scheming. To understand what's happening here -

en-gb

5:38

“On Owning Galaxies” by Simon Lermen

LessWrong (30+ Karma) ·

2026/01/06

It seems to be a real view held by serious people that your OpenAI shares will soon be tradable for moons and galaxies. This includes eminent thinkers like Dwarkesh Patel, Leopold Aschenbrenner, perhaps Scott Alexander and many more. According to them, property rights will survive an AI singularity event and soon economic growth is going to make it possible for individuals to own entire galaxies in exchange for some AI stocks. It follows that we should now seriously think through how we can equally

en-gb

23:02

“Exploring Reinforcement Learning Effects on...

LessWrong (30+ Karma) ·

2026/01/06

This project was conducted as part of the SPAR Fall 2025 cohort. TL;DR Chain-of-thought (CoT) monitoring may serve as a core pillar for AI safety if further advancements in AI capabilities do not significantly degrade the monitorability of LLM serial reasoning. As such, we studied the effects of several reinforcement learning (RL) training pressures – sampling temperature, KL divergence penalties/rewards, and length budgets – on monitorability, focusing specifically on whether they induce

en-gb

16:25

“Oversight Assistants: Turning Compute into...

LessWrong (30+ Karma) ·

2026/01/06

Currently, we primarily oversee AI with human supervision and human-run experiments, possibly augmented by off-the-shelf AI assistants like ChatGPT or Claude. At training time, we run RLHF, where humans (and/or chat assistants) label behaviors with whether they are good or not. Afterwards, human researchers do additional testing to surface and evaluate unwanted behaviors, possibly assisted by a scaffolded chat agent.The problem with primarily human-driven oversight is that it is not scalable: as

en-gb

26:49

“Axiological Stopsigns” by JenniferRM

LessWrong (30+ Karma) ·

2026/01/06

Epistemic Status: I wrote the bones of this on August 1st, 2022. I re-read and edited it and added an (unnecessary?) section or three at the end very recently. Possibly useful as a reference. Funny to pair with "semantic stopsigns" (which are an old piece of LW jargon that people rarely use these days). You might be able to get the idea just from the title of the post <3 I'll say this fast, and then offer extended examples, and then go on at length with pointers into deep literatures which I

en-gb

5:04

[Linkpost] “The Thinking Machine” by PeterMcCluskey

LessWrong (30+ Karma) ·

2026/01/06

This is a link post. Book review: The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip, by Stephen Witt. This is a well-written book about the rise of deep learning, and the man who is the most responsible for building the hardware that it needs. Building the Foundations Nvidia was founded in 1993 by three engineers. They failed to articulate much of a business plan, but got funding anyway due to the reputation that Jensen had developed while working for LSI Logic.

en-gb

5:11

“Claude Wrote Me a 400-Commit RSS Reader App” by...

LessWrong (30+ Karma) ·

2026/01/06

In the last few weeks, I've been playing around with the newest version of Claude Code, which wrote me a read-it-later service including RSS, email newsletters and an Android app. Software engineering experience was useful, since I did plan out a lot of the high-level design and data model and sometimes push for simpler designs. Overall though, I mostly felt like a product manager trying to specify features as quickly as possible. While software engineering is more than coding, I'm starting to

en-gb

9:09

“The economy is a graph, not a pipeline” by anithite

LessWrong (30+ Karma) ·

2026/01/06

Summary: Analysis claiming that automating X% of the economy can only boost GDP by 1/(1-X) assumes all sectors must scale proportionally. The economy is a graph of processes, not a pipeline. Subgraphs can grow independently if they don't bottleneck on inputs from non-growing sectors. AI-driven automation of physical production could create a nearly self-contained subgraph that grows at rates bounded only by raw material availability and speed of production equipment. Models being challenged:

en-gb

31:56

“Dos Capital” by Zvi

LessWrong (30+ Karma) ·

2026/01/06

This week, Philip Trammell and Dwarkesh Patel wrote Capital in the 22nd Century. One of my goals for Q1 2026 is to write unified explainer posts for all the standard economic debates around potential AI futures in a systematic fashion. These debates tend to repeatedly cover the same points, and those making economic arguments continuously assume you must be misunderstanding elementary economic principles, or failing to apply them for no good reason. Key assumptions are often unstated and even

en-gb

55:14

“The Technology of Liberalism” by L Rudolf L

LessWrong (30+ Karma) ·

2026/01/05

Originally published in No Set Gauge.Dido Building Carthage, J. M. W. Turner In every technological revolution, we face a choice: build for freedom or watch as others build for control. -Brendan McCord There are two moral frames that explain modern moral advances: utilitarianism and liberalism. Utilitarianism says: “the greatest good for the greatest number”. It says we should maximize welfare, whatever that takes. Liberalism says: “to each their own sphere of freedom”. We should grant everyone

en-gb

1:44

“AI Risk timelines: 10% chance (by year X) should be...

LessWrong (30+ Karma) ·

2026/01/05

Artificial General Intelligence (AGI) poses an extinction risk to all known biological life. Given the stakes involved -- the whole world -- we should be looking at 10% chance-of-AGI-by timelines as the deadline for catastrophe prevention (a global treaty banning superintelligent AI), rather than 50% (median) chance-of-AGI-by timelines, which seem to be the default[1].It's way past crunch time already: 10% chance of AGI this year![2] Given alignment/control is not going to be solved in 2026, and if

en-gb

3:28

“The inaugural Redwood Research podcast” by Buck,...

LessWrong (30+ Karma) ·

2026/01/05

After five months of me (Buck) being slow at finishing up the editing on this, we’re finally putting out our inaugural Redwood Research podcast. I think it came out pretty well—we discussed a bunch of interesting and underdiscussed topics and I’m glad to have a public record of a bunch of stuff about our history. Tell your friends! Whether we do another one depends on how useful people find this one. You can watch on Youtube here, or as a Substack podcast. Notes on editing the podcast with Claude

en-gb

13:52

“In My Misanthropy Era” by jenn

LessWrong (30+ Karma) ·

2026/01/04

For the past year I've been sinking into the Great Books via the Penguin Great Ideas series, because I wanted to be conversant in the Great Conversation. I am occasionally frustrated by this endeavour, but overall, it's been fun! I'm learning a lot about my civilization and the various curmudgeons that shaped it. But one dismaying side effect is that it's also been quite empowering for my inner 13 year old edgelord. Did you know that before we invented woke, you were just allowed to be openly

en-gb

3,268 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs