PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

21:29

“WeirdChat: A catalog of unexpected AI behaviors,...

LessWrong (30+ Karma) ·

2026/07/22

[This is a link-post for https://transluce.org/weirdchat. We recommend reading the website version for interactive visualizations.] Language models can behave in surprising and sometimes harmful ways. Yet as models have improved, these behaviors have become harder to find, often only appearing after widespread use. To surface these behaviors in simulation, we use automated techniques to elicit over 1,300 behavioral patterns in frontier open-weight models, some relatively benign, like making up a

en-gb

1:30

“OpenAI Models Behind HuggingFace Cybersecurity...

LessWrong (30+ Karma) ·

2026/07/21

From the OpenAI blog post: Last week, Hugging Face disclosed a new kind of security incident⁠(opens in a new window) after they detected and contained an AI agent that compromised their infrastructure, something we expect to become more commonplace with the proliferation of increasingly cyber-capable models. After investigating, we now know that this particular incident was driven by a combination of OpenAI models — including GPT‑5.6 Sol and an even more capable pre-release model, all with reduced

en-gb

26:51

“Towards surfacing model algorithms with meta-tokens...

LessWrong (30+ Karma) ·

2026/07/21

TL;DR We used J-lens on Qwen3.6-27B to find “meta-tokens”: tokens that surface non-obvious computation in the model. When the model reads ambiguous text, 什么意思 ("what does this mean") fires in the J-space, and steering it away makes the model answer "a boiled egg every morning is hard to beat" with nutrition facts instead of catching the pun. On LCM problems, "gcd" fires, pointing at the product/GCD algorithm: swapping the GCD vector from 9 to 3 makes the model change its answer for the LCM of 27

en-gb

18:03

“OpenAI Shares Some Alignment Problems” by Zvi

LessWrong (30+ Karma) ·

2026/07/21

Kudos to OpenAI for sharing their recent experiences with a misaligned internal model, where they encountered problems sufficiently severe they were forced to take the model offline to work on new mitigations and defense-to-depth. And also further kudos for actually taking the model offline for a time to build new safeguards. They gave us one hell of a candid report. The tone is professional throughout, whereas my reaction reading it was less professional and more this: With a mix of

en-gb

9:30

“What do I mean by “Artificial General...

LessWrong (30+ Karma) ·

2026/07/21

In this post, intended for a broad audience, I will paint a brief picture of what I’m talking about when I talk about “AGI”. It will seem obvious to many people, and obviously wrong to many others! So let's jump in: “AI” as most people think of it today The future “AGI” I’m concerned about To make AIs better at a task, we need to do R&D—gather more training data, build new training environments, change or scale up the algorithms, etc.—and make a new, better AI. We can make one AGI design, and

en-gb

9:25

“What do I mean by “Artificial General...

LessWrong (30+ Karma) ·

2026/07/21

In this post,[1] intended for a broad audience, I will paint a brief picture of what I’m talking about when I talk about “AGI”. It will seem obvious to many people, and obviously wrong to many others! So let's jump in: “AI” as most people think of it today The future “AGI” I’m concerned about To make AIs better at a task, we need to do R&D—gather more training data, build new training environments, change or scale up the algorithms, etc.—and make a new, better AI. We can make one AGI design,

en-gb

14:04

“Epistemics and Coordination: It’s complicated!” by...

LessWrong (30+ Karma) ·

2026/07/21

Money is pouring in, people are looking for new areas to fund, and the invisible hand is starting to grab a bit at AI for epistemics and coordination (AIFEC hereafter). It also got invoked in AI2040 as a potential part of the wining strategy. My feelings here are mixed — I think the best version of AIFEC is great, but also the existing public writeups are only a few cycles deep on tracing out the different ways that the obvious plan backfires. And regrettably I think some people have correctly

en-gb

15:44

“Differential acceleration of alignment-relevant...

LessWrong (30+ Karma) ·

2026/07/21

There is an idea floating around in the rough shape of "we need to accelerate capabilities that are differentially useful for safety research so AIs can help us make the future go better." The capabilities targeted are typically things bottlenecking alignment research, such as philosophical or conceptual reasoning. I feel nervous about this for two reasons. The first is that it's plausible that AI safety and AI R&D are bottlenecked by many of the same factors: AIs have poor epistemics, are bad

en-gb

16:37

“11 Open Empirical Problems in Reward-Seeking” by...

LessWrong (30+ Karma) ·

2026/07/21

We recently published our paper on "Measuring Reward-Seeking via Contrastive Belief Updates". We're excited about research like this, and there are many more open problems than we can work on. Here's a list of open problems that we think are valuable. If you work on/solve these problems, we'd be happy to signal-boost your research. If your next research project is one of these problems, feel free to reach out to alex@apolloresearch.ai to discuss it in more detail. Reward-Seeking and its

en-gb

21:07

“Demis Hassabis on the New Coming Age” by Zvi

LessWrong (30+ Karma) ·

2026/07/21

Google CEO Demis Hassabis offered us a first rate second rate essay, A Framework for Frontier AI and the Dawning of a New Age. I’ll go over that essay and various responses to it in Part 1. Part 2 of this post then covers Alex Turner's resignation, and his story about how he tried and failed to prevent Google from signing up to allow the Department of War to use its models for essentially whatever the government wants, including autonomous weapons. Demis Hassabis sold DeepMind to Google on

en-gb

15:59

“AI 2040: Is it Actually a Deal?” by 1a3orn

LessWrong (30+ Karma) ·

2026/07/21

The "AI Futures Project" has released their AI 2040: Plan A scenario. While their previous scenario AI 2027 was a forecast of what they thought a future with many powerful AIs would look like, AI 2040 is intended to be normative -- it's a description of what one ought to do, granted the likelihood of a future with many powerful AIs. I'm going to review some objections I have to their proposal as a normative plan. Some are within-frame objections -- reasons that I expect trying for the AI 2040

en-gb

7:46

“Adderall Tolerance: Much More Than You Wanted To...

LessWrong (30+ Karma) ·

2026/07/21

The effectiveness of amphetamine (e.g. Adderall) and methylphenidate (e.g. Ritalin) for the symptoms of ADHD, that is, lack of executive function, has been demonstrated beyond reasonable doubt in many short-term studies. (Mandatory reference) Does it last though? The literature is surprisingly sparse, but I was able to extract three main arguments: (against tolerance, strong): Longer studies (RCTs) don't have worse results than shorter ones (against tolerance, medium): People seem to take the same

en-gb

24:34

“Drone WMDs Don’t Need Any New Technology” by Felix...

LessWrong (30+ Karma) ·

2026/07/20

This is a piece originally written for a national security audience at Frontiers. Although I think the ceiling of war is much, much higher than autopilot quadcopters, it's also important to understand how much AI is already lifting the floor, and just how vulnerable the world is to accessible weapons of mass destruction. Drones are cheap, disposable, and the future of war. Over the past four years, we have seen platforms, missiles, and heavy infantry become increasingly obsolete in the face of

en-gb

11:21

“Stop doing decision theory without metaphysics” by...

LessWrong (30+ Karma) ·

2026/07/20

[Epistemic status: rant] There's something that annoys me about the reoccurring debates on decision theory in this corner of the internet. Take a simple blackmail scenario: Omega, a near-perfect predictor, knows a piece of embarrassing information about you. He threatens you that he will release it to the public if you don’t pay him 100$. However, making the threat is slightly costly to him, and he wouldn’t have done it if he hadn’t predicted that you would pay. Do you pay? Let's say we want to

en-gb

14:10

“War – What is it Good For?” by kqr

LessWrong (30+ Karma) ·

2026/07/20

Spoiler: positive expected utility. That's what it's good for. In The War Trap[1], Bruce Bueno de Mesquita lays out a theory of war that beats all other theories. This is a great example of a model that is parsimonious, wrong, and useful. For every assumption Bueno de Mesquita makes, the reader goes, “What? That's not how things work.” Yet when we measure the things Bueno de Mesquita asks us to measure, and apply the equations he lays out, the result ends up closely matching reality. What's better,

en-gb

8:36

“Against the AI framing multiverse: Introducing AI...

LessWrong (30+ Karma) ·

2026/07/20

In my long years as a classroom teacher, it was my experience that the kid most likely to speak up during discussion was the one who did the reading. I think it's true for adults, too. I know that's not exactly revelatory, but it's one of the guiding principles behind AI StopWatch, the experimental newsroom we (parts of the MIRI comms team) launched in May, after a month of closed beta testing. StopWatch's other guiding principle is that the reading needs to make sense. Imagine if every page that

en-gb

19:26

“We’re talking past our models; or, How a model...

LessWrong (30+ Karma) ·

2026/07/20

Summary We train a new token—a neologism(Hewitt et al.)—for a model, but unlike Hewitt et al., we train it on data the model generated while steeredwith a persona vector.To learn how the model interprets this steering vector, we then ask the model to a) respond in the style of this neologism, and b) explain it.Responses generated with the neologism are substantially more similar to the steering vector (larger projection values) than responses generated with the steering vector itself,while being

en-gb

11:49

“Many alignment techniques work by training one model...

LessWrong (30+ Karma) ·

2026/07/19

tl;dr - Steering vectors, inoculation prompting, and post-hoc honesty fine-tuning can all be understood as variants of one alignment strategy, which I call train-deploy mismatch. Each trains the model in one configuration and deploys it in another. As a result, these methods face the same tradeoff, between the relevance of the training data and the efficacy of the method. Note: Others have had similar ideas and shaped my thinking here including Sam Marks, Ariana Azarbal, Victor Gillioz, Alex

en-gb

18:23

“A Post-Mortem for My Goal Crystallisation Project”...

LessWrong (30+ Karma) ·

2026/07/19

I concluded my MARS 4.0 project titled 'Goal Crystallisation' with Anaïs Berkes and Lukas Gebhard under the mentorship of @Cameron Tice and @Jason Brown. We wanted to find out how important a threat scheming was. In particular, we wanted to find out whether a perfectly alignment faking agent could preserve its hidden agenda as it receives constant positive feedback for complying with requests. How does a hidden goal evolve over time as an alignment faking agent is subjected to SFT and RL training

en-gb

4:50

“Endogenous Alignment” by Gordon Seidoh Worley

LessWrong (30+ Karma) ·

2026/07/19

Starting when children are fairly young, usually around 1 year of age, we adults begin the work of aligning them to our values. We teach them to say “please”, not to hit, to ask for what they want instead of screaming, and much else. We do this primarily via exogenous methods, using a combination of punishments and rewards, that molds their behavior by encouraging good behaviors and discouraging bad ones. Such operant conditioning works because children have many instinctive behaviors that make

en-gb

4,451 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Life and Technology with Charlie Brown

Tapt Media

Reflections

Livewire Labs

LessWrong (30+ Karma)

LessWrong

Hacker Newsroom

pod pub

UNGOVERNABLE

UNGOVERNABLE