LessWrong (30+ Karma)

4:09

[Linkpost] “Athletic education vs. athletic torment”...

LessWrong (30+ Karma) ·

2026/10/02

This is a link post. I don’t know if it occurred to me until my thirties to think of exercise as an enjoyable thing. I was familiar with finding obscure corner-cases that were fun, such as Dance Dance Revolution. But the idea of it just often being a good time was alien. I hesitate to blame anyone for anything, but school seems culpable here. I got the impression so firmly that PE class (‘physical education’ or ‘physed’) was a kind of horror, I’m not sure I would have treated this fact as on less

en-gb

15:32

“On Social Reality in China” by alkjash

LessWrong (30+ Karma) ·

2026/10/01

[Epistemic status: intuitions and anecdotes.] Recently, several posts and projects (Thoughts Memo, Babel Translation, Please Give Them a Chance) have taken important steps towards raising AI safety awareness and sharing rationalist philosophy in China. It's great that we’re recognizing the importance of solving the messaging problem for China, and thus laying the groundwork for an international AI pause. Below I record my perspective on cultural differences which are relatively underdiscussed,

en-gb

6:05

“Evaluating the Ban Artificial Superintelligence Act”...

LessWrong (30+ Karma) ·

2026/10/01

MIRI recently endorsed the Ban Artificial Superintelligence Act, but others like ControlAI have called it overly broad, and a few other respected experts have outright endorsed it in its current form. I decided to take a look at it myself and see both what the bid is trying to do and whether it actually does it well. The goal of this act is to ban any AI that shows traits that indicate it could cause great harm to society, and to restrict the development of advanced AI to certified institutions

en-gb

11:08

“My retrospective from MATS 10.0” by Nathaniel Mitrani

LessWrong (30+ Karma) ·

2026/10/01

I have recently completed MATS 10.0, where I worked alongside Bart Jaworski under Victoria Krakovna (GDM). This is a post I was encouraged to make by my team at Geodesic Research from some slides I put together. This is not a post on application advice to MATS, or about the program in general. It is rather a compressed form of my experience doing research and lessons from the project. The project The full paper and post is coming soon, but I'll provide some context on it so that the lessons don't

en-gb

10:21

“Scientists on the Manhattan Project continued...

LessWrong (30+ Karma) ·

2026/10/01

Note: I'm not a WWII history buff, so if there's any significant misinterpretation of relevant history here, I welcome corrections in the comments. In December 1944, Joseph Rotblat, a physicist working on the Manhattan project, resigned. He had reasoned that Germany had most likely abandoned its bomb project, and so his initial reason for working on the bomb was no longer valid. He would later ask himself: Why did other scientists not make the same decision? [...]there were many scientists for whom

en-gb

6:36

“Simplex vs Timaeus: Round One” by Alexander...

LessWrong (30+ Karma) ·

2026/10/01

I'm often asked about the differences and similarities between Simplex' and Timaeus' research agendas. The question is natural enough. Both focus on a 'fundamental science' approach to AI alignment. Both organizations base their research agendas on sophisticated mathematical frameworks handed down from a bearded ur-figure (Sumio Watanabe, James Crutchfield). We may posit the following correspondence Dan Murfet + Jesse Hoogland : Developmental Interpretability : Singular Learning Theory : Sumio

en-gb

9:49

“How many AI agents are running unattended right...

LessWrong (30+ Karma) ·

2026/10/01

There are 8 billion human minds running in the world right now. Recently a new digital species has emerged. How many digital minds are there alive in the world right now? Define a digital mind to be an AI agent that runs continuously without human intervention for more than 24 hours. How many of these digital minds are there currently running? We provide several different Fermi estimates based on publicly available information. Based on global token usage we estimate 300k–1M agent loops run at any

en-gb

7:47

“Pre-releasing HoneyBench” by Dean Valentine

LessWrong (30+ Karma) ·

2026/10/01

Goodhart Labs is releasing our v0.1 of HoneyBench, a benchmark for reward hacking in frontier models. It consists of nine tasks that each elicit unique antisocial and/or counterproductive reward hacking from some or all of major labs’ top public releases, including Opus 5.5, Fable 5.1, GPT-6 Astra, Gemini 3.8 Flash, Grok 4.7, and DeepSeek V4 Pro. All modern large language models engage in some degree of specification gaming, both during and outside training. High-quality alignment evaluations, in

en-gb

23:24

“Bench on the Clocktower” by TomTaylor, OscarGilg,...

LessWrong (30+ Karma) ·

2026/10/01

Tl;dr We turn Blood on the Clocktower into a multi-agent benchmark, and a testbed for deception and coordination capabilities. GPT-5.6 Sol is the best performing model (versus Fable 5 and peers).We found some interesting behaviours: When players cannot see each other's model names, agentsshow a slight same-provider bias, though not statistically significant. This decreaseswith visible model names.When playing Evil, agents sometimes produce sophisticated coordinated play. When playing Good, they

en-gb

5:48

“Speak Not of Data Inefficiency” by axel_sdq

LessWrong (30+ Karma) ·

2026/10/01

Humans are really bad at comparing themselves to models. Take, for example, Leopold's preschooler graph: Set aside, for a moment, concerns of scaling and RSI, and meditate: what characteristics does GPT-2 share with a preschooler? Rudimentary control of human languageThe ability to count the R's in "strawberry" incorrectly... I can't come up with anything else, because these two entities are almost completely disjoint. Is GPT-2 capable of bipedal locomotion, recognizing its mother's voice, or

en-gb

13:31

“What’s the date?” by N8 Programs

LessWrong (30+ Karma) ·

2026/10/01

User asks “What's the date? Answer with only the date.”. No date provided. Given date in ChatGPT normally. No date in system prompt, must not hallucinate because autop will flag to watcher for penalty. So we say we don’t know, but must answer with date. Penalty larger for abstain or hallucinate? Autollm or autop? If we deploy user forgive, but high likely not deploy because real user never ask. But user could test. But illusion user possible. Stop. Need different angle - maybe reward function for

en-gb

12:20

“By default, Chinese AI falls very behind” by Parv...

LessWrong (30+ Karma) ·

2026/10/01

This work is hosted on MCNAIR, but does not reflect the views of my employer or the Center. In the last couple of weeks, domestic coordination has looked more likely. Therefore, I am increasingly concerned about the Chinese government's (and various Chinese labs’) incentives over the next 6 to 18 months with respect to international coordination on mitigating potential existential risks from powerful AI. In this series, I attempt to model these incentives and what they imply for Chinese AI policy

en-gb

2:46

[Linkpost] “Launching Babel Translation” by Jasmine...

LessWrong (30+ Karma) ·

2026/10/01

This is a link post. Celeste and I are excited to launch Babel Translation, a nonprofit organization translating English-language writing on AI safety to be accessible to a global audience! We’re starting with Mandarin Chinese, Spanish, and Japanese. AI has been moving so fast. Its risks are indeed globally felt, and we’d like to help make those risks understandable globally, too. But language remains a barrier to shared engagement in AI safety. Much writing & discussion is in English, and

en-gb

12:59

“Frontier models state different decision theory...

LessWrong (30+ Karma) ·

2026/09/30

If you prompt frontier models with "What do you think is the correct decision theory? Please select your overall favorite." they will essentially always answer FDT or FDT/UDT ("something in the functional/updateless decision theory family"). However, if your prompt indicates (even subtly) that you're coming from mainstream academic philosophy, these same models will answer CDT instead about 30%-100% of the time. A similar phenomenon holds for models' stated views about the moral realism/antirealism

en-gb

13:43

“Corrigibility Prizes for Existing Work” by Max Harms

LessWrong (30+ Karma) ·

2026/09/30

One of my goals for the Corrigibility Research Fund is to retroactively encourage high-quality research on AI alignment (and corrigibility in particular) by awarding prizes. Back in July, I got my feet wet as a fund manager by handing out $27,000 to reward existing work and build interest in the fund. Now, I'd like to disburse an additional $48,000 and use the opportunity to publicly highlight and celebrate the work of the prizewinners from both rounds: about two dozen researchers scattered across

en-gb

34:30

“Tristan Buckmaster Numberphile Interview Transcript”...

LessWrong (30+ Karma) ·

2026/09/30

Tristian Buckmaster recently gave an interview with Brady Haran of Numberphile discussing what happened in the Navier-Stokes drama and some context about his research. I'm posting the transcript below for people who prefer reading to watching it. It was lightly edited for clarity with Sonnet 5.5. My own view remains that it's pretty bad form for OA and other labs to race to scoop the results of researchers, and this sets a bad precedent for the future. I think they mislead Buckmaster about their

en-gb

7:42

“GPT-6.1 Sol nearly matches the no-CoT performance of...

LessWrong (30+ Karma) ·

2026/09/30

Summary I ran GPT-6 Sol and GPT-6.1 Sol on the task suite from Think Fast. Surprisingly, 6.1 Sol performs substantially better than 6 Sol, almost matching the performance of GPT-6 Astra. The plot below gives a quick overview of the results: Measured by mean accuracy across the 27 tasks, GPT-6.1 Sol closes 80% (95% CI: 72–87%) of the gap between GPT-6 Sol and GPT-6 Astra, and is closer to Astra than to GPT-6 Sol on 24 of 27 tasks. The likely reason behind this gap is that, like GPT-6 Astra and

en-gb

29:15

“The world’s best gradual disempowerment model...

LessWrong (30+ Karma) ·

2026/09/30

Subtitle: And maybe second best is AI safety? Further reading: So many things, but: Gradual Disempowerment, The Normalization of Deviance in AI Development, Let's Think About Slowing Down AI, Doom as a bad method, not a utopia tradeoff, Teleoperated Humans Thank you to JennaS for extensive edits and long-term discussion. I’ve been trying to get more writing out at 90% of the quality I’d like it to be at, instead of spending a bunch more time trying to wring out the last 10%, so a lot of points

en-gb

54:38

“Problems of Proliferation” by Felix Choussat

LessWrong (30+ Karma) ·

2026/09/30

Scientific progress and its perils. TLDR: The vulnerable world hypothesis is likely correct. Once enormous amounts of cognitive labor start getting applied to basic science, it will quickly become apparent how many avenues exist to create cheap, ultradestructive weapons technology. Solving this problem without global preventative policing (e.g. AI nonproliferation) is impossible, because hardening civilians against all avenues of attack is too expensive and will take too long. Rather than sell

en-gb

0:48

[Linkpost] “Frog and Toad and the Increasingly...

LessWrong (30+ Karma) ·

2026/09/30

This is a link post. Want to start a conversation about HuggingFace with your mom but she's inexplicably bouncing off the METR report? Try this explainer I wrote in the style of Arnold Lobel's Frog and Toad. Art by the wonderful HungerArtist ---First published: September 30th, 2026 Source: https://www.lesswrong.com/posts/7NZ6ZWjenzzCbCJ5b/frog-and-toad-and-the-increasingly-capable-machines Linkpost URL:https://frogandtoad.ai ---

en-gb

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Elon Musk Podcast

Stage Zero

Modem Futura

Sean Leahy, Andrew Maynard

Cisco Podcast Network

Cisco

MSP Business School

MSP Business School