PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

13:06

“2025-Era “Reward Hacking” Does Not Show that Reward...

LessWrong (30+ Karma) ·

2025/12/19

Folks ask me, "LLMs seem to reward hack a lot. Does that mean that reward is the optimization target?". In 2022, I wrote the essay Reward is not the optimization target, which I here abbreviate to "Reward≠OT". Reward still is not the optimization target: Reward≠OT said that (policy-gradient) RL will not train systems which primarily try to optimize the reward function for its own sake (e.g. searching at inference time for an input which maximally activates the AI's specific reward model). In

en-gb

6:29

“In defence of the human agency: “Curing Cancer” is...

LessWrong (30+ Karma) ·

2025/12/19

Epistemic Status: Philosophical. Based on the debate by Togelius at NeurIPS "The trouble with fighting for human freedom is that one spends most of one's time defending scoundrels. For it is against scoundrels that oppressive laws are first aimed, and oppression must be stopped at the beginning if it is to be stopped at all" - H.L. Mencken The Incident At the NeurIPS 2025 debate panel, when most panelists were discussing about replacing humans at all levels for scientific progress, Dr. Julian

en-gb

8:50

“Neuro-scaffold” by DirectedEvolution

LessWrong (30+ Karma) ·

2025/12/19

Cole Wyeth writes: It seems like we actually do not have a good name for the things that AI companies are building, weirdly enough... This actually slows down my reasoning or at least writing about the topic, because I have to choose from this inadequate list of options repeatedly, often using different nouns in different places. I do not have a good suggestion. Any ideas? They liked my suggestion of "neuro-scaffold" and suggested I write a short justification. Definition A neuro-scaffold means a

en-gb

4:10

“Wuckles!” by Raemon

LessWrong (30+ Karma) ·

2025/12/19

Every now and then, I say "wuckles" next to a person who I am work trialing, or who for some other reason cares about not looking dumb in front of me. They casually-but-frantically google "wuckles" in the background to try to figure out what I meant. The only google result for "wuckles" is an urban dictionary response which is not what I meant by it, and it's pretty clear from context it's not what I meant by it. They are confused. This post is for the third such person, should they appear one

en-gb

5:20

“Scalable End-to-End Interpretability” by jsteinhardt

LessWrong (30+ Karma) ·

2025/12/19

This is partly a linkpost for Predictive Concept Decoders, and partly a response to Neel Nanda's Pragmatic Vision for AI Interpretability and Leo Gao's Ambitious Vision for Interpretability. There is currently somewhat of a debate in the interpretability community between pragmatic interpretability---grounding problems in empirically measurable safety tasks---and ambitious interpretability----obtaining a full bottom-up understanding of neural networks. In my mind, these both get at something

en-gb

13:33

“Help keep AI under human control: Palisade Research...

LessWrong (30+ Karma) ·

2025/12/19

TL;DR: Please consider donating to Palisade Research this year, especially if you care about reducing catastrophic AI risks via research, science communications, and policy. Donations are matched 1:1 up to $1.1 million. You can donate via Every.org or reach out at donate@palisaderesearch.org. Who We Are Palisade Research is a nonprofit focused on reducing civilization-scale risks from agentic AI systems. We conduct empirical research on frontier AI systems, and inform policymakers and the public

en-gb

31:54

“BashArena: A Control Setting for Highly Privileged...

LessWrong (30+ Karma) ·

2025/12/18

We’ve just released BashArena, a new high-stakes control setting we think is a major improvement over the settings we’ve used in the past. In this post we’ll discuss the strengths and weaknesses of BashArena, and what we’ve learned about how to make settings for high stakes control research. BashArena Overview BashArena is a control setting: a dataset designed to support the kind of adversarial game we studied in the original AI control paper and Ctrl-Z. In general, control settings require two

en-gb

9:25

“A basic case for donating to the Berkeley Genomics...

LessWrong (30+ Karma) ·

2025/12/18

IntroductionReprogenetics is the field of using genetics and reproductive technology to empower parents to make genomic choices on behalf of their future children. The Berkeley Genomics Project is aiming to support and accelerate the field of reprogenetics, in order to more quickly develop reprogenetic technology in a way that will be safe, accessible, highly effective, and societally beneficial. A quick case for BGP: Effective reprogenetics would greatly improve many people's lives by

en-gb

11:24

“The Bleeding Mind” by Adele Lopez

LessWrong (30+ Karma) ·

2025/12/17

The simulator theory of LLM personas may be crudely glossed as: "the best way to predict a person is to simulate a person". Ergo, we can more-or-less think of LLM personas as human-like creatures—different, alien, yes; but these differences are pretty predictable by simply imagining a human put into the bizarre circumstances of an LLM. I've been surprised at how well this viewpoint has held up in the last three years, and have updated accordingly. Still, there are deep differences not implied by

en-gb

11:18

“Announcing RoastMyPost” by ozziegooen

LessWrong (30+ Karma) ·

2025/12/17

Today we're releasing RoastMyPost, a new experimental application for blog post evaluation using LLMs. Try it Here. TLDR RoastMyPost is a new QURI application that uses LLMs and code to evaluate blog posts and research documents. It uses a variety of LLM evaluators. Most are narrow checks: Fact Check, Spell Check, Fallacy Check, Math Check, Link Check, Forecast Check, and others. Optimized for EA & Rationalist content with direct import from EA Forum and LessWrong URLs. Other links use

en-gb

5:06

“Still Too Soon” by Gordon Seidoh Worley

LessWrong (30+ Karma) ·

2025/12/17

We had to let Sammie go last Wednesday. The lump first appeared on her nose sometime in the Summer of 2024. It was small. You could miss it if you didn’t know her face. The vet did a biopsy. Melanoma. We were told the median survival time was 6 months. Over the next 18, we did everything we could to make her life comfortable. When the tumor grew too large, we’d have it debulked. When she couldn’t clean herself anymore, we brushed her and gave her baths. In the end, we had to hand feed her when the

en-gb

15:28

“Non-Scheming Saints (Whether Human Or Digital) Might...

LessWrong (30+ Karma) ·

2025/12/17

Contextualizing Foreword The user "cdt" wrote to me: Please put this in a top-level post. I don't agree (or rather I don't feel it's this simple), but I really enjoyed reading your two rejoinders here. I don't normally do top level posts because I don't tend to believe that its possible to change people's minds for the better if they aren't exactly and precisely posed to seek an answer (or whatever (its more complicated than that)). But when someone who seems to be a person reading and writing in

en-gb

4:57

“Mistakes in the Moonshot Alignment Program and What...

LessWrong (30+ Karma) ·

2025/12/17

Good things about the program: I learnt a lot about alignment in doing the prep, interviewed a lot of agent foundations researchers, learnt some neuromorality, talked to some neuroscientists, saw lots of what it takes to organize an event like this and have it go well. Marketing went pretty well - got 298 candidates, ~50% of whom had phds, applicants included senior researchers/engineers from Nvidia, Meta, AMD, AWS, etc. And also feels like there's a lot left on the table for the marketing, lots

en-gb

8:30

“Dancing in a World of Horseradish” by lsusr

LessWrong (30+ Karma) ·

2025/12/17

Commercial airplane tickets are divided up into coach, business class, and first class. In 2014, Etihad introduced The Residence, a premium experience above first class. The Residence isn't very popular. The reason The Residence isn't very popular is because of economics. A Residence flight is almost as expensive as a private charter jet. Private jets aren't just a little bit better than commercial flights. They're a totally different product. The airplane waits for you, and you don't have to go

en-gb

2:08

[Linkpost] “Announcing: MIRI Technical Governance...

LessWrong (30+ Karma) ·

2025/12/17

This is a link post. MIRI's Technical Governance Team plans to run a small research fellowship program in early 2026. The program will run for 8 weeks, and include a $1200/week stipend. Fellows are expected to work on their projects 40 hours per week. The program is remote-by-default, with an in-person kickoff week in Berkeley, CA (flights and housing provided). Participants who already live in or near Berkeley are free to use our office for the duration of the program. Fellows will spend the first

en-gb

3:40

“Radiology Automation Does Not Generalize to Other...

LessWrong (30+ Karma) ·

2025/12/16

The NYT article Your A.I. Radiologist Will Not Be With You Soon reports, “Leaders at OpenAI, Anthropic and other companies in Silicon Valley now predict that A.I. will eclipse humans in most cognitive tasks within a few years… The predicted extinction of radiologists provides a telling case study. So far, A.I. is proving to be a powerful medical tool to increase efficiency and magnify human abilities, rather than take anyone's job.”[1] I disagree that this is a “telling case study.”[2] Radiology

en-gb

43:01

“GPT-5.2 Is Frontier Only For The Frontier” by Zvi

LessWrong (30+ Karma) ·

2025/12/16

Here we go again, only a few weeks after GPT-5.1 and a few more weeks after 5.0.There weren’t major safety concerns with GPT-5.2, so I’ll start with capabilities, and only cover safety briefly starting with ‘Model Card and Safety Training’ near the end. Table of Contents The Bottom Line. Introducing GPT-5.2. Official Benchmarks. GDPVal. Unofficial Benchmarks. Official Hype. Public Reactions. Positive Reactions. Personality Clash. Vibing the Code. Negative Reactions. But Thou

en-gb

5:56

“Scientific breakthroughs of the year” by technicalities

LessWrong (30+ Karma) ·

2025/12/16

A couple of years ago, Gavin became frustrated with science journalism. No one was pulling together results across fields; the articles usually didn’t link to the original source; they didn't use probabilities (or even report the sample size); they were usually credulous about preliminary findings (“...which species was it tested on?”); and they essentially never gave any sense of the magnitude or the baselines (“how much better is this treatment than the previous best?”). Speculative results were

en-gb

91:09

“Response to titotal’s critique of our AI 2027...

LessWrong (30+ Karma) ·

2025/12/16

Introduction In June, a Substack/LessWrong/EA Forum user named titotal wrote “A deep critique of AI 2027's bad timeline models”. Our original model that they were critiquing can be found here, with a brief overview below.[1] In a nutshell, we disagree with most of titotal's criticisms. While they pointed out a few mistakes, for which we are grateful, on the whole we think the problems that they pointed out do not add up to the model being “bad.” In this post we will explain why. While we initially

en-gb

18:38

“Defending Against Model Weight Exfiltration Through...

LessWrong (30+ Karma) ·

2025/12/15

Authors: Roy Rinberg, Adam Karvonen, Alex Hoover, Daniel Reuter, Keri Warr Arxiv paper link One Minute Summary Anthropic has adopted upload limits to prevent model weight exfiltration. The idea is simple: model weights are very large, text outputs are small, so if we cap the output bandwidth, we can make model weight transfer take a long time. The problem is that inference servers now generate an enormous amount of tokens (on the order of ~1TB tokens per day), and the output text channel is the one

en-gb

3,301 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs