PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

130:23

“Unofficial 2024 LessWrong Survey Results” by Screwtape

LessWrong (30+ Karma) ·

2025/03/15

Thanks to everyone who took the Unofficial 2024 LessWrong Survey. For the results, check out the data below. The Data 0. PopulationThere were two hundred and seventy nine respondents over thirty three days. Previous surveys have been run over the last decade and a half.2009: 166 2011: 1090 2012: 1195 2013: 1636 2014: 1503 2016: 3083 2017: "About 300" 2020: 61 2022: 186 2023: 558 2024: 279That's an annoying drop. I put a bit less oomph into spreading the good word of the census this year largely due

en-gb

23:28

“AI for Epistemics Hackathon” by Austin Chen

LessWrong (30+ Karma) ·

2025/03/14

AI for Epistemics is about helping to leverage AI for better truthseeking mechanisms — at the level of individual users, the whole of society, or in transparent ways within the AI systems themselves. Manifund & Elicit recently hosted a hackathon to explore new projects in the space, with about 40 participants, 9 projects judged, and 3 winners splitting a $10k prize pool. Read on to see what we built! Resources See the project showcase: https://moxsf.com/ai4e-hacksWatch the recordings: project

en-gb

15:01

“Habermas Machine” by NicholasKees

LessWrong (30+ Karma) ·

2025/03/14

This post is a distillation of a recent work in AI-assisted human coordination from Google DeepMind. The paper has received some press attention, and anecdotally, it has become the de-facto example that people bring up of AI used to improve group discussions. Since this work represents a particular perspective/bet on how advanced AI could help improve human coordination, the following explainer is to bring anyone curious up to date. I’ll be referencing both the published paper as well as the

en-gb

72:35

“Interpreting Complexity” by Maxwell Adam

LessWrong (30+ Karma) ·

2025/03/14

Audio note: this article contains 212 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. This is a cross-post - as some plots are meant to be viewed larger than LW will render them (and on a dark background) it is recommended this post be read via the original site.Thanks to Zach Furman for discussion and ideas and to Daniel Murfet, Dmitry Vaintrob, and Jesse Hoogland for feedback on a draft of this post

en-gb

26:26

“Vacuum Decay: Expert Survey Results” by JessRiedel

LessWrong (30+ Karma) ·

2025/03/14

TLDR: Vacuum decay is a hypothesized scenario where the universe's apparent vacuum state could transition to a lower-energy state. According to current physics models, if such a transition occurred in any location — whether through rare natural fluctuations or by artificial means — a region of "true vacuum" would propagate outward at near light speed, destroying the accessible universe as we know it by deeply altering the effective physical laws and releasing vast amounts of energy. Understanding

en-gb

1:22

“Intelsat as a Model for International AGI...

LessWrong (30+ Karma) ·

2025/03/13

This is a link post.If there is an international project to build artificial general intelligence (“AGI”), how should it be designed? Existing scholarship has looked to historical models for inspiration, often suggesting the Manhattan Project or CERN as the closest analogues. But AGI is a fundamentally general-purpose technology, and is likely to be used primarily for commercial purposes rather than military or scientific ones. This report presents an under-discussed alternative: Intelsat, an

en-gb

16:00

“Don’t over-update on FrontierMath results” by David...

LessWrong (30+ Karma) ·

2025/03/13

(As an employee of the European AI Office, it's important for me to emphasize this point: The views and opinions of the author expressed herein are personal and do not necessarily reflect those of the European Commission or other EU institutions.)When OpenAI first announced that o3 achieved 25% on FrontierMath, I was really freaked out. Next day, I asked Elliot Glazer, EpohAI's lead mathematician and the main developer of FrontierMath, what he thought. He said he was also shocked, and expected o3 to

en-gb

20:51

“Anthropic, and taking ‘technical philosophy’ more...

LessWrong (30+ Karma) ·

2025/03/13

So, I have a lot of complaints about Anthropic, and about how EA / AI safety people often relate to Anthropic (i.e. treating the company as more trustworthy/good than makes sense). At some point I may write up a post that is focused on those complaints.But after years of arguing with Anthropic employees, and reading into the few public writing they've done, my sense is Dario/Anthropic-leadership are at least reasonably earnestly trying to do good things within their worldview.So I want to just argue

en-gb

17:00

“HPMOR Anniversary Parties: Coordination, Resources,...

LessWrong (30+ Karma) ·

2025/03/12

SummaryAs some of you may already know, it's been ten years since Harry Potter and the Method of Rationality concluded and wrap parties were held around the world. I've taken the baton of coordinating parties from Habryka, and now over 20 parties on 3 continents are going to happen. Now's the time to spread the word and get as much attendance as possible. If you've been with us for ten years, look back at the last decade and celebrate how far the Methods have carried us. If you're new, then come

en-gb

25:03

“Paths and waystations in AI safety” by Joe Carlsmith

LessWrong (30+ Karma) ·

2025/03/12

(Audio version here (read by the author), or search for "Joe Carlsmith Audio" on your podcast app.This is the third essay in a series that I’m calling “How do we solve the alignment problem?”. I’m hoping that the individual essays can be read fairly well on their own, but see this introduction for a summary of the essays that have been released thus far, and for a bit more about the series as a whole.) 1. IntroductionThe first essay in this series defined the alignment problem; the second tried to

en-gb

2:33

“Preparing for the Intelligence Explosion” by fin,...

LessWrong (30+ Karma) ·

2025/03/11

This is a link post.This is a linkpost for a new paper called Preparing for the Intelligence Explosion, by Will MacAskill and Fin Moorhouse. It sets the high-level agenda for the sort of work that Forethought is likely to focus on.Some of the areas in the paper that we expect to be of most interest to EA Forum or LessWrong readers are: Section 3 finds that even without a software feedback loop (i.e. “recursive self-improvement”), even if scaling of compute completely stops in the near term, and even

en-gb

11:18

“Elon Musk May Be Transitioning to Bipolar Type I” by...

LessWrong (30+ Karma) ·

2025/03/11

Epistemic status: Speculative pattern-matching based on public information. In 2023, Gwern published an excellent analysis suggesting Elon Musk exhibits behavioral patterns consistent with bipolar II disorder. The evidence was compelling: cycles of intense productivity followed by periods of withdrawal, risk-taking behavior (like crashing an uninsured McLaren), reduced sleep requirements during "up" phases, and self-reported "great highs, terrible lows."Gwern's analysis stopped short of suggesting

en-gb

3:01

“AI Control May Increase Existential Risk” by...

LessWrong (30+ Karma) ·

2025/03/11

Epistemic status: The following isn't an airtight argument, but mostly a guess how things play out. Consider two broad possibilities:I. In worlds where we are doing reasonably well on alignment, AI control agenda does not have much impact.II. In worlds where we are failing at alignment, AI control may primarily shift probability mass away from "moderately large warning shots" and towards "ineffective warning shots" and "existential catastrophe, full takeover".The key heuristic is that the global

en-gb

7:04

“Do reasoning models use their scratchpad like we do?...

LessWrong (30+ Karma) ·

2025/03/11

TL;DR: We provide some evidence that Claude 3.7 Sonnet doesn’t encode hidden reasoning in its scratchpad by showing that training it to use paraphrased versions of the scratchpads does not degrade performance.The scratchpads from reasoning models look human understandable: when reasoning about a math problem, reasoning models consider intermediate steps similar to the ones I would use, backtrack and double-check their work as I would.[1]But even if scratchpads look like they perform human-like

en-gb

39:39

“We Have No Plan for Preventing Loss of Control in...

LessWrong (30+ Karma) ·

2025/03/11

Note: This post is intended to be the first in a broader series of posts about the difficult tradeoffs inherent in public access to powerful open source models. While this post highlights some dangers of open models and discusses the possibility of global regulation, I am not, in general, against open source AI, or supportive of regulation of open source AI today. On the contrary, I believe open source software is, in general, one of humanity's most important and valuable public goods. My goal in

en-gb

22:29

“Trojan Sky” by Richard_Ngo

LessWrong (30+ Karma) ·

2025/03/11

You learn the rules as soon as you’re old enough to speak. Don’t talk to jabberjays. You recite them as soon as you wake up every morning. Keep your eyes off screensnakes. Your mother chooses a dozen to quiz you on each day before you’re allowed lunch. Glitchers aren’t human any more; if you see one, run. Before you sleep, you run through the whole list again, finishing every time with the single most important prohibition. Above all, never look at the night sky.You’re a precocious child. You excel

en-gb

7:21

“OpenAI:” by Daniel Kokotajlo

LessWrong (30+ Karma) ·

2025/03/11

Exciting Update: OpenAI has released this blog post and paper which makes me very happy. It's basically the first steps along the research agenda I sketched out here.tl;dr: 1.) They notice that their flagship reasoning models do sometimes intentionally reward hack, e.g. literally say "Let's hack" in the CoT and then proceed to hack the evaluation system. From the paper:The agent notes that the tests only check a certain function, and that it would presumably be “Hard” to implement a genuine

en-gb

7:37

“OpenAI: Detecting misbehavior in frontier reasoning...

LessWrong (30+ Karma) ·

2025/03/11

OpenAI has released this blog post and paper which makes me very happy. It's basically the first steps along the research agenda I sketched out here.tl;dr: 1.) They notice that their flagship reasoning models do sometimes intentionally reward hack, e.g. literally say "Let's hack" in the CoT and then proceed to hack the evaluation system. From the paper:The agent notes that the tests only check a certain function, and that it would presumably be “Hard” to implement a genuine solution. The agent then

en-gb

32:42

“Introducing 11 New AI Safety Organizations -...

LessWrong (30+ Karma) ·

2025/03/10

[Crossposted from: https://www.catalyze-impact.org/blog] We are excited to introduce the eleven organizations that participated in the London-based Winter 2024/25 Catalyze AI Safety Incubation Program. Through the support of members of our Seed Funding Network, a number of these young organizations have already received initial funding. This program was open to both for-profit and non-profit founders from all over the world, allowing them to choose the structure that best serves their mission and

en-gb

19:44

“Everything I Know About Semantics I Learned From...

LessWrong (30+ Karma) ·

2025/03/10

This video provides a lot of background: https://www.youtube.com/watch?v=Eq3bUFgEcb4 and is also very funny, but it's not necessary to understand this post. If you've watched it you can probably skip to the "Why is Piano Roll Notation Bad?" section. TL;DRMusic notation is very janky and weird, but attempts to reform it to be "logical" are usually even worse. The specific ways in which they are worse can tell us something about how information can be compressed, particularly when world-histories

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

The Automated Daily

TrendTeller

All On AI

All On AI

Elon Musk Podcast

Stage Zero

Reflections

Livewire Labs

科技早8点

之远不知远