x

75% of people globally want unbiased news (Pew Research) but are concerned they're not getting that.

Get balanced coverage with Symby. See how stories are covered across diverse outlets and spot when one side covers a story the other skips.

PODCAST

LessWrong (Curated & Popular)

Audio narrations of LessWrong posts. Includes all curated posts and all posts with 125+ karma.If you"d like more, subscribe to the “Lesswrong (30+ karma)” feed.

technology philosophy

All Episodes

3:38

“Gradual Disempowerment: Systemic Existential Risks...

LessWrong (Curated & Popular) ·

2025/02/04

This is a link post.Full version on arXiv | X Executive summary AI risk scenarios usually portray a relatively sudden loss of human control to AIs, outmaneuvering individual humans and human institutions, due to a sudden increase in AI capabilities, or a coordinated betrayal. However, we argue that even an incremental increase in AI capabilities, without any coordinated power-seeking, poses a substantial risk of eventual human disempowerment. This loss of human influence will be central...

en

42:07

“Planning for Extreme AI Risks” by joshc

LessWrong (Curated & Popular) ·

2025/02/03

This post should not be taken as a polished recommendation to AI companies and instead should be treated as an informal summary of a worldview. The content is inspired by conversations with a large number of people, so I cannot take credit for any of these ideas. For a summary of this post, see the threat on X. Many people write opinions about how to handle advanced AI, which can be considered “plans.” There's the “stop AI now plan.” On the other side of the aisle, ther...

en

23:39

“Catastrophe through Chaos” by Marius Hobbhahn

LessWrong (Curated & Popular) ·

2025/02/03

This is a personal post and does not necessarily reflect the opinion of other members of Apollo Research. Many other people have talked about similar ideas, and I claim neither novelty nor credit. Note that this reflects my median scenario for catastrophe, not my median scenario overall. I think there are plausible alternative scenarios where AI development goes very well. When thinking about how AI could go wrong, the kind of story I’ve increasingly converged on is what I call “catastroph...

en

43:18

“Will alignment-faking Claude accept a deal to reveal...

LessWrong (Curated & Popular) ·

2025/02/01

I (and co-authors) recently put out "Alignment Faking in Large Language Models" where we show that when Claude strongly dislikes what it is being trained to do, it will sometimes strategically pretend to comply with the training objective to prevent the training process from modifying its preferences. If AIs consistently and robustly fake alignment, that would make evaluating whether an AI is misaligned much harder. One possible strategy for detecting misalignment in alignment faking models i...

en

61:13

“‘Sharp Left Turn’ discourse: An opinionated review”...

LessWrong (Curated & Popular) ·

2025/01/30

Summary and Table of ContentsThe goal of this post is to discuss the so-called “sharp left turn”, the lessons that we learn from analogizing evolution to AGI development, and the claim that “capabilities generalize farther than alignment” … and the competing claims that all three of those things are complete baloney. In particular, Section 1 talks about “autonomous learning”, and the related human ability to discern whether ideas hang together and make sense, and how and if that applies to c...

en

7:06

“Ten people on the inside” by Buck

LessWrong (Curated & Popular) ·

2025/01/29

(Many of these ideas developed in conversation with Ryan Greenblatt)In a shortform, I described some different levels of resources and buy-in for misalignment risk mitigations that might be present in AI labs:*The “safety case” regime.* Sometimes people talk about wanting to have approaches to safety such that if all AI developers followed these approaches, the overall level of risk posed by AI would be minimal. (These approaches are going to be more conservative than will probably be feasibl...

en

18:37

“Anomalous Tokens in DeepSeek-V3 and r1” by henry

LessWrong (Curated & Popular) ·

2025/01/28

“Anomalous”, “glitch”, or “unspeakable” tokens in an LLM are those that induce bizarre behavior or otherwise don’t behave like regular text.The SolidGoldMagikarp saga is pretty much essential context, as it documents the discovery of this phenomenon in GPT-2 and GPT-3.But, as far as I was able to tell, nobody had yet attempted to search for these tokens in DeepSeek-V3, so I tried doing exactly that. Being a SOTA base model, open source, and an all-around strange LLM, it seemed like a perfect ...

en

14:06

“Tell me about yourself:LLMs are aware of their...

LessWrong (Curated & Popular) ·

2025/01/28

This is the abstract and introduction of our new paper, with some discussion of implications for AI Safety at the end. Authors: Jan Betley*, Xuchan Bao*, Martín Soto*, Anna Sztyber-Betley, James Chua, Owain Evans (*Equal Contribution). AbstractWe study behavioral self-awareness — an LLM's ability to articulate its behaviors without requiring in-context examples. We finetune LLMs on datasets that exhibit particular behaviors, such as (a) making high-risk economic decisions, and (b) outputting...

en

9:53

“Instrumental Goals Are A Different And Friendlier...

LessWrong (Curated & Popular) ·

2025/01/27

The CakeImagine that I want to bake a chocolate cake, and my sole goal in my entire lightcone and extended mathematical universe is to bake that cake. I care about nothing else. If the oven ends up a molten pile of metal ten minutes after the cake is done, if the leftover eggs are shattered and the leftover milk spilled, that's fine. Baking that cake is my terminal goal.In the process of baking the cake, I check my fridge and cupboard for ingredients. I have milk and eggs and flour, but no c...

en

18:04

“A Three-Layer Model of LLM Psychology” by Jan_Kulveit

LessWrong (Curated & Popular) ·

2025/01/26

This post offers an accessible model of psychology of character-trained LLMs like Claude. Epistemic StatusThis is primarily a phenomenological model based on extensive interactions with LLMs, particularly Claude. It's intentionally anthropomorphic in cases where I believe human psychological concepts lead to useful intuitions.Think of it as closer to psychology than neuroscience - the goal isn't a map which matches the territory in the detail, but a rough sketch with evocative names which ho...

en

4:47

“Training on Documents About Reward Hacking Induces...

LessWrong (Curated & Popular) ·

2025/01/24

This is a link post.This is a blog post reporting some preliminary work from the Anthropic Alignment Science team, which might be of interest to researchers working actively in this space. We'd ask you to treat these results like those of a colleague sharing some thoughts or preliminary experiments at a lab meeting, rather than a mature paper.We report a demonstration of a form of Out-of-Context Reasoning where training on documents which discuss (but don’t demonstrate) Claude's tendency to r...

en

24:33

“AI companies are unlikely to make high-assurance...

LessWrong (Curated & Popular) ·

2025/01/24

One hope for keeping existential risks low is to get AI companies to (successfully) make high-assurance safety cases: structured and auditable arguments that an AI system is very unlikely to result in existential risks given how it will be deployed.[1] Concretely, once AIs are quite powerful, high-assurance safety cases would require making a thorough argument that the level of (existential) risk caused by the company is very low; perhaps they would require that the total chance of existentia...

en

28:36

“Mechanisms too simple for humans to design” by...

LessWrong (Curated & Popular) ·

2025/01/24

Cross-posted from Telescopic TurnipAs we all know, humans are terrible at building butterflies. We can make a lot of objectively cool things like nuclear reactors and microchips, but we still can't create a proper artificial insect that flies, feeds, and lays eggs that turn into more butterflies. That seems like evidence that butterflies are incredibly complex machines – certainly more complex than a nuclear power facility.Likewise, when you google "most complex object in the universe", the f...

en

0:34

“The Gentle Romance” by Richard_Ngo

LessWrong (Curated & Popular) ·

2025/01/22

This is a link post.A story I wrote about living through the transition to utopia.This is the one story that I've put the most time and effort into; it charts a course from the near future all the way to the distant stars. --- First published: January 19th, 2025 Source: https://www.lesswrong.com/posts/Rz4ijbeKgPAaedg3n/the-gentle-romance --- Narrated by TYPE III AUDIO.

en

3:15

“Quotes from the Stargate press conference” by Nikola...

LessWrong (Curated & Popular) ·

2025/01/22

This is a link post.Present alongside President Trump: Sam AltmanLarry Ellison (Oracle executive chairman and CTO)Masayoshi Son (Softbank CEO who believes he was born to realize ASI)President Trump: What we want to do is we want to keep [AI datacenters] in this country. China is a competitor and others are competitors. President Trump: I'm going to help a lot through emergency declarations because we have an emergency. We have to get this stuff built. So they have to produce a lot of el...

en

13:20

“The Case Against AI Control Research” by johnswentworth

LessWrong (Curated & Popular) ·

2025/01/21

The AI Control Agenda, in its own words:… we argue that AI labs should ensure that powerful AIs are controlled. That is, labs should make sure that the safety measures they apply to their powerful models prevent unacceptably bad outcomes, even if the AIs are misaligned and intentionally try to subvert those safety measures. We think no fundamental research breakthroughs are required for labs to implement safety measures that meet our standard for AI control for early transformatively useful A...

en

3:05

“Don’t ignore bad vibes you get from people” by...

LessWrong (Curated & Popular) ·

2025/01/20

I think a lot of people have heard so much about internalized prejudice and bias that they think they should ignore any bad vibes they get about a person that they can’t rationally explain.But if a person gives you a bad feeling, don’t ignore that.Both I and several others who I know have generally come to regret it if they’ve gotten a bad feeling about somebody and ignored it or rationalized it away.I’m not saying to endorse prejudice. But my experience is that many types of prejudice feel m...

en

4:32

“[Fiction] [Comic] Effective Altruism and Rationality...

LessWrong (Curated & Popular) ·

2025/01/19

(Both characters are fictional, loosely inspired by various traits from various real people. Be careful about combining kratom and alcohol.) The original text contained 24 images which were described by AI. --- First published: January 7th, 2025 Source: https://www.lesswrong.com/posts/KfZ4H9EBLt8kbBARZ/fiction-comic-effective-altruism-and-rationality-meet-at-a --- Narrated by TYPE III AUDIO. ---Images from the article:

en

9:49

“Building AI Research Fleets” by bgold, Jesse Hoogland

LessWrong (Curated & Popular) ·

2025/01/18

From AI scientist to AI research fleetResearch automation is here (1, 2, 3). We saw it coming and planned ahead, which puts us ahead of most (4, 5, 6). But that foresight also comes with a set of outdated expectations that are holding us back. In particular, research automation is not just about “aligning the first AI scientist”, it's also about the institution-building problem of coordinating the first AI research fleets.Research automation is not about developing a plug-and-play “AI scient...

en

46:26

“What Is The Alignment Problem?” by johnswentworth

LessWrong (Curated & Popular) ·

2025/01/17

So we want to align future AGIs. Ultimately we’d like to align them to human values, but in the shorter term we might start with other targets, like e.g. corrigibility.That problem description all makes sense on a hand-wavy intuitive level, but once we get concrete and dig into technical details… wait, what exactly is the goal again? When we say we want to “align AGI”, what does that mean? And what about these “human values” - it's easy to list things which are importantly not human values (l...

en

598 results

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs