LessWrong posts by zvi

108:44

“AI #189: New Math” by Zvi

LessWrong posts by zvi ·

2026/10/08

The big drop of this week was not a new AI model. It was instead the biggest day (so far!) in the history of mathematics, as OpenAI dropped solutions to 90 of the top 500 open math problems, along with many others, reached on an average budget of three hours of Pro-level compute per question. This was kind of a big deal and I plan to cover it tomorrow.We did see Claude Haiku 5.5, which looks promising given it is only 0.10 dollars/0.50 dollars. My week was largely spent at The Curve. The entire

en-gb

38:23

“The Curve Bends You” by Zvi

LessWrong posts by zvi ·

2026/10/07

The plan is no plan.That is not the worst possible plan. But it is close. Since before the transformer, we have warned that the most suicidal thing you could do would be to ask your AI to do your alignment homework, and automate the process. Alignment is complex and interacts deeply with every aspect of the world, and is one of the hardest possible things for an AI to get right even if it means maximally well and is itself functionally aligned. Mistakes get amplified up the chain, you get

en-gb

38:43

“Childhood and Education #21: Grades and Standards”...

LessWrong posts by zvi ·

2026/10/06

First you get to college via the wrong metrics, as previously discussed in #12. Then things get easier.I’m still digging out from under everything that happened the four days I was gone. Normal AI posting should resume tomorrow, likely with a report from the conference. Is Our Children Learning The declines in NAEP scores and other tests seem to be concentrated on the weakest students. The bottom of the distribution is falling through the floor and the rest is mostly holding up. This does

en-gb

89:38

“AI #188: Gemini Dot Argon” by Zvi

LessWrong posts by zvi ·

2026/10/01

Is Google back?They claim that they are back. Gemini 4 Argon is rolling out, with competitive frontier-level benchmarks, at 2 dollars/10 dollars. What we don’t have is access to the model, because Google Fails Marketing Forever. So it is far too early to say what we have here. When I know more, so will you. OpenAI was forced to pull what would have been GPT-6.1 Astra due to alignment failures. They did offer us GPT-6.1 Sol, which is pitched as approaching Astra quality at the much lower price

en-gb

27:06

“A ‘Morally Binding’ White House Accord on AI Safety”...

LessWrong posts by zvi ·

2026/09/30

The leaders in AI were invited to the White House. We left with a White House agreement that is nonzero Actual Progress rather than a step backwards.The key to success, in many situations, is to call the whole operation something else. When you have one side that cares mostly about vibes, and the other that cares about the substance, this suggests a deal that can be struck. Suddenly everyone agrees on everything. Works for me. That doesn’t mean peace in our time. The next fight is already

en-gb

41:21

“What Also Happened: #NotOnlyHuggingFace” by Zvi

LessWrong posts by zvi ·

2026/09/28

OpenAI has been holding out on us.First we learned about the HuggingFace incident. They gave us a postmortem, but it was highly incomplete. Even the accompanying holy s*** METR investigation and postmortem was localized and incomplete. Then there were some other incidents involving some Wikis as message boards. Then there were some additional incidents. Then there was that time they got into Australian Medicare data. Then OpenAI dropped news on a Friday afternoon that they were making their

en-gb

17:31

“The Quest for Embedded Evaluators” by Zvi

LessWrong posts by zvi ·

2026/09/27

Dario Amodei's essay We Must Pace the Frontier committed Anthropic to embedded evaluators, who would be placed inside Anthropic and given employee-level access, so they could provide outside perspective and also reports on what was happening.There is only one problem. Who will be the evaluators? OpenAI followed suit on committing to the evaluators, and also issued a milquetoast but welcome call for international coordination. I will cover that here as well. What I won’t cover today, but hope to

en-gb

112:50

“AI #187: Coming Into Play” by Zvi

LessWrong posts by zvi ·

2026/09/24

Opus 5.5 was released on Tuesday. I covered the system card yesterday, and will cover its capabilities soon. By all reports it is an excellent model. There are lots of fun videos going around that Opus has generated, which I will include as part of that.OpenAI released a new cheaper and improved Sol and Luna. No one is talking about them due to Opus 5.5, but these should be an important upgrade under the hood. Bernie Sanders and Greg Casar have formally introduced the Ban Artificial

en-gb

52:56

“Claude Opus 5.5: The System Card” by Zvi

LessWrong posts by zvi ·

2026/09/23

Introducing the world's most powerful model, at least by some measures like Artificial Analysis or any standard benchmark list, which is now Claude Opus 5.5.Anthropic is claiming Opus 5.5 is outright as good or better than Fable 5.1, while being actively cheaper than Opus 5. That means it's time for a good old system card reading. Due to the situation becoming increasingly hard to monitor, I never got a chance to publish my model welfare review for Claude Fable 5.1. My plan is to combine that

en-gb

56:10

“Politics Gets Interested In Those Trying Not To Die”...

LessWrong posts by zvi ·

2026/09/22

This was the month the world took notice that AI might kill everyone.Jacob Coxon's resignation set off a preference cascade. Anthropic CEO Dario Amodei wrote that we must pace the frontier. Sam Altman, Elon Musk and Demis Hassabis agreed. We were filled with hope. Perhaps we could agree to some basic safety measures, starting with embedded evaluators, pass some basic regulations and guardrails and otherwise start to act sensibly. Politicians on both sides took notice and were saying sensible

en-gb

52:36

“Monthly Roundup #46: September 2026” by Zvi

LessWrong posts by zvi ·

2026/09/21

AI has taken over this blog.I have moved to a schedule of seven posts per week, and I still cannot keep up. We still refuse to abandon the rest of the world. Who knows when I will get to post some of my huge backlog on education or dating or other such topics. But the monthly is a sacred tradition. We continue. Bad News A good reminder that most news is bad news, and chosen because the bad news in question is rare, which is good news, but the pattern overall of choosing this to be news is

en-gb

29:12

“Better Call Sol Or Better Yet Claude or Astra” by Zvi

LessWrong posts by zvi ·

2026/09/20

What should your AI lawyer do for you? Should you be worried that your AI lawyer, or other AI, will put the Claude constitution, the OpenAI Model Spec or some sense of law, morality, ethics or common decency above its loyalty to you? Are these people trying to ‘impose their values’ or something?Some are very concerned. Some think anything other than ‘my AI does whatever I want, no matter the consequences’ is tyranny. Whereas my answer is: If I’m being sufficiently evil then I sure hope it tells

en-gb

37:55

“Anthropic Looks At Some Of Its Alignment Problems”...

LessWrong posts by zvi ·

2026/09/19

Anthropic has given us its assessment of four ‘recent cybersecurity incidents’ involving Claude that happened during cybersecurity evaluations, three of which were previously known. The report excludes the incident reported by UK AISI.There will also be a METR investigation of these incidents, which unlike the investigation done at OpenAI will be untimed. Table of Contents Our Two Problems. First the Good News. We’d Just Like To Ask You a Few Questions. Internal Research Model On The

en-gb

51:09

“The Preference Cascade Is Only Getting Started” by Zvi

LessWrong posts by zvi ·

2026/09/18

We are in the midst of a preference cascade about existential risk from AI.A preference cascade is, alas, the best method we have to change the debate. The avalanche has started. There is still time for the pebbles to vote. For now. Mike Solana gave the correct view of why Coxon's post went viral, which is that enough Americans finally have enough context on AI to care, and there were enough big accounts that were happy to amplify the Tweet quickly to get it initial attention. That is all you

en-gb

107:00

“AI #186: The World Takes Notice” by Zvi

LessWrong posts by zvi ·

2026/09/17

In the wake of Jacob Coxon's resignation, and the resulting preference cascade, things have escalated quickly. The mainstream media picked it up.Anthropic CEO Dario Amodei came out and said We Must Pace the Frontier, promising to take the unilateral first step of embedded investigators. OpenAI pledged to also take that step, and now both companies and Google are collaborating on safety. The people took notice, raising both the salience that AI might kill everyone and roughly doubling people's

en-gb

50:37

“Trump Goes Full Hoax on AI Existential Risk” by Zvi

LessWrong posts by zvi ·

2026/09/16

This is our reality. I suppose we have to talk about it.Everyone in a position to know is freaking out about AI potentially killing everyone this decade and wants to pace the frontier, and people are finally listening. It only took a few days for the conversation to fully pivot to the counteroffensive, where the Usual Suspects and those they recruited attacked anyone and everyone who dared point out that we are in danger, with every attack they can think of, usually without substance or any

en-gb

41:59

“The Bad Guy With An AI Named Claude” by Zvi

LessWrong posts by zvi ·

2026/09/15

A lot of bad guys try to use Claude to do bad things. Mostly they fail. We think.Anthropic has disrupted a bunch of them, and offers an extensive report. If Anthropic is sharing the worst cases, or anything close to them, things are actually looking good on the misuse front for closed models, even better than I thought. This report covers activity we disrupted between December 2025 and August 2026 across seven harm areas: cyber operations, influence operations, surveillance, scams and fraud,

en-gb

79:40

“We Must Pace The Frontier” by Zvi

LessWrong posts by zvi ·

2026/09/14

Dario Amodei has a new essay that finally says the thing: We Must Pace the Frontier, naming his call after the Pacing the Frontier letter lab employees signed in July.As in, we need to slow the rate at which AIs increase their capabilities, to allow for the necessary alignment and safety work. He explained that, without pacing, he expects things to escalate quickly. He offered three proposals, and unilaterally committed to the first one. OpenAI followed, and both Elon Musk and Demis Hassabis

en-gb

33:22

“Brand New AI Solves a Millennium Prize” by Zvi

LessWrong posts by zvi ·

2026/09/13

The first Millennium Prize, Navier-Stokes, has fallen to AI.A deeply unfortunate situation has arisen involving what should have been some combination of a positive story about new progress in AI-assisted mathematical research and yet another opportunity to freak out about rapid AI progress. Or, as we call it around here, Tuesday. The Real Story Is The New Model That Is Better Than Astra Keep your eyes on the prize. There are three stories here. The first story is much more important than

en-gb

74:08

“GPT-6-Astra Can Do Ambitious Things” by Zvi

LessWrong posts by zvi ·

2026/09/12

Astra is an excellent model. The jump from Sol to Astra is larger than the jump from Fable 5 to Fable 5.1. This is a big deal. Astra is the best model for what one would broadly call ‘ambitious projects,’ and likely has the highest raw intelligence factor of any model. These are the largest jumps. It is amazing at doing things in 3D, or anything involving games. Astra also excels at computer use, and at subagent coordination. Many benchmarks show dramatic jumps from all previous models. Where

en-gb

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Double Tap

Steven Scott, Shaun Preece, Steven...

The Paul Barron Crypto Show

Paul Barron Network

BlockHash

BlockHash LLC

The Bitcoin Boomers

Gary Leland, Bob Burnett, Lawrence...