PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

5:38

“Why care about AI personhood?” by Francis Rhys Ward

LessWrong (30+ Karma) ·

2025/01/27

In this new paper, I discuss what it would mean for AI systems to be persons — entities with properties like agency, theory-of-mind, and self-awareness — and why this is important for alignment. In this post, I say a little more about why you should care. The existential safety literature focuses on the problems of control and alignment, but these framings may be incomplete and/or untenable if AI systems are persons. The typical story is approximately as follows.AI x-safety problem (the usual

en-gb

8:09

“Brainrot” by Jesse Hoogland

LessWrong (30+ Karma) ·

2025/01/26

January: In early 2026, Meta launches a fleet of new AI influencers, targeting the massive audience displaced by the Xiaohongshu-TikTok wars. They are beautiful, funny, smart—whatever you want them to be. Equipped with the latest in online learning, the agents immediately begin adapting to social media trends as they occur. Engagement metrics reach new highs. Nobody questions whose engagement they're measuring. February: Fads pass faster now. The influencers are speed-running a revival movement of

en-gb

18:08

“Counterintuitive effects of minimum prices” by...

LessWrong (30+ Karma) ·

2025/01/26

The Attorney General of Massachusetts recently announced that drivers for ride-sharing companies must be paid at least $32.50 per hour.Now, if you’re a hardcore libertarian, then you probably hate with the minimum wage. You need no convincing and we can part now on good terms.But what if you’re part of the vast majority of people who like the minimum wage? What if you think the minimum wage is awesome and you want to expand it? That's fine. I won’t try to change your mind! But even so, there are

en-gb

5:36

“The Rising Sea” by Jesse Hoogland

LessWrong (30+ Karma) ·

2025/01/26

And then we hit a wall. Nobody expected it. Well... almost nobody. Yann LeCun posted his "I told you so's" all over X. Gary Marcus insisted he'd predicted this all along. Sam Altman pivoted, declaring o3 was actually already ASI. The first rumors of scaling laws breaking down were already circulating in late 2024. By late 2025, it was clear that test-time scaling was not coming to the rescue. Despite the scaling labs' best efforts, nobody could figure out how to generalize reasoning beyond the

en-gb

18:38

“Anomalous Tokens in DeepSeek-V3 and r1” by henry

LessWrong (30+ Karma) ·

2025/01/26

“Anomalous”, “glitch”, or “unspeakable” tokens in an LLM are those that induce bizarre behavior or otherwise don’t behave like regular text.The SolidGoldMagikarp saga is pretty much essential context, as it documents the discovery of this phenomenon in GPT-2 and GPT-3.But, as far as I was able to tell, nobody had yet attempted to search for these tokens in DeepSeek-V3, so I tried doing exactly that. Being a SOTA base model, open source, and an all-around strange LLM, it seemed like a perfect

en-gb

23:13

“On polytopes” by Dmitry Vaintrob

LessWrong (30+ Karma) ·

2025/01/25

[Epistemic status: slightly ranty. This is a lightly edited slack chat, and so may be lower-quality.]I am surprised by the perennial spikes in excitement about "polytopes" and "tropical geometry on activation space" in machine learning and interpretability[1]. I'm not going to discuss tropical geometry in this post in depth (and might save it for later -- it will have to wait until I'm in a less rant-y mood[2]). As I'll explain below, I think some interesting questions and insights are extractable

en-gb

8:51

“Attribution-based parameter decomposition” by Lucius...

LessWrong (30+ Karma) ·

2025/01/25

This is a linkpost for Apollo Research's new interpretability paper: "Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition".We introduce a new method for directly decomposing neural network parameters into mechanistic components. MotivationAt Apollo, we've spent a lot of time thinking about how the computations of neural networks might be structured, and how those computations might be embedded in networks' parameters. Our

en-gb

7:33

“Why Aligning an LLM is Hard, and How to Make it...

LessWrong (30+ Karma) ·

2025/01/25

Where the challenge of aligning an LLM-based AI comes from, and the obvious solution. Evolutionary Psychology is the Root CauseLLMs are pre-trained using stochastic gradient-descent on very large amounts of human-produced text, normally drawn from the web, books, journal articles and so forth. A pre-trained LLM has learned in detail how to simulate all the different human text-generation processes that produced this text — everything from a cooperatively edited wikipedia article to shit-postings. We

en-gb

4:12

“Yudkowsky on The Trajectory podcast” by Seth Herd

LessWrong (30+ Karma) ·

2025/01/25

This is a link post.I found this interview notable as the most useful public statement yet of Yudkowsky's views. I congratulate both him and the host, Dan Fagella, for strategically improving how they're communicating their ideas.The host, Dan Fagella is to be commended for asking the right questions and taking the right tone to get an extraordinarily concise statement of Yudkowsky's views on what we might do to survive and why. It also seemed likely that Yudkowsky has thought hard about his

en-gb

30:32

“Six Thoughts on AI Safety” by boazbarak

LessWrong (30+ Karma) ·

2025/01/25

[Crossposted from windowsontheory]The following statements seem to be both important for AI safety and are not widely agreed upon. These are my opinions, not those of my employer or colleagues. As is true for anything involving AI, there is significant uncertainty about everything written below. However, for readability, I present these points in their strongest form, without hedges and caveats. That said, it is essential not to be dogmatic, and I am open to changing my mind based on evidence. None

en-gb

9:54

“Instrumental Goals Are A Different And Friendlier...

LessWrong (30+ Karma) ·

2025/01/24

The CakeImagine that I want to bake a chocolate cake, and my sole goal in my entire lightcone and extended mathematical universe is to bake that cake. I care about nothing else. If the oven ends up a molten pile of metal ten minutes after the cake is done, if the leftover eggs are shattered and the leftover milk spilled, that's fine. Baking that cake is my terminal goal.In the process of baking the cake, I check my fridge and cupboard for ingredients. I have milk and eggs and flour, but no cocoa

en-gb

36:43

“Stargate AI-1” by Zvi

LessWrong (30+ Karma) ·

2025/01/24

There was a comedy routine a few years ago. I believe it was by Hannah Gadsby. She brought up a painting, and looked at some details. The details weren’t important in and of themselves. If an AI had randomly put them there, we wouldn’t care.Except an AI didn’t put them there. And they weren’t there at random. A human put them there. On purpose. Or, as she put it: THAT was a DECISION. This is the correct way to view decisions around a $500 billion AI infrastructure project, announced right after

en-gb

12:55

“Theory of Change for AI Safety Camp” by Linda Linsefors

LessWrong (30+ Karma) ·

2025/01/24

In a private discussion related to our fundraiser it was pointed out to me that AISC has not made clear enough what our actual theory of change is. Therefore this post.Some caveats/context: This is my personal viewpoint. Other organisers might disagree about what is central or not.I’ve co-organised AISC1, AISC8, AISC9, and now AISC10. Remmelt has co-organised all except AISC2. Robert recently joined for AISC10.I hope there will be an AISC11, but in either case, I will no longer be an organiser. This

en-gb

5:04

“Writing experiments and the banana escape valve” by...

LessWrong (30+ Karma) ·

2025/01/24

[Note: This is not alignment-related, but rather a spacefiller personal blog post.]I've been trying to write a public post every day of January. So far I’ve been enjoying it. I don’t think this approach works for everyone: in particular, I’ve also been hanging on by a thread to the schedule and to the ability to sleep. I publicly committed to write 3 posts a week, but my “secret” personal goal to write a post every day for the month. Not only in order to have more output, but because in the past

en-gb

24:34

“AI companies are unlikely to make high-assurance...

LessWrong (30+ Karma) ·

2025/01/23

One hope for keeping existential risks low is to get AI companies to (successfully) make high-assurance safety cases: structured and auditable arguments that an AI system is very unlikely to result in existential risks given how it will be deployed.[1] Concretely, once AIs are quite powerful, high-assurance safety cases would require making a thorough argument that the level of (existential) risk caused by the company is very low; perhaps they would require that the total chance of existential risk

en-gb

17:10

“MONA: Managed Myopia with Approval Feedback” by Seb...

LessWrong (30+ Karma) ·

2025/01/23

Blog post by Sebastian Farquhar, David Lindner, Rohin Shah. It discusses the paper MONA: Myopic Optimization with Non-myopic Approval Can Mitigate Multi-step Reward Hacking by Sebastian Farquhar, Vikrant Varma, David Lindner, David Elson, Caleb Biddulph, Ian Goodfellow, and Rohin Shah.Our paper tries to make agents that are safer in ways that we may not be able to evaluate through Myopic Optimization with Non-myopic Approval (MONA).Suppose we know that AI systems will do bad things but we also know

en-gb

4:46

“Tail SP 500 Call Options” by sapphire

LessWrong (30+ Karma) ·

2025/01/23

SPX is an index fund that tracks the SP500. Right now SPX is worth about 6100 per share. For individual stocks, even the large ones, you can usually only buy options through Dec 2027. However SPX has options that expire in Dec 2028 / 29 / 30[1]. Such options with strike prices of 10K, 11K, or 12K seem extremely lucrative to me. Max dated SPY calls are also a good option. The obvious motivation is AI progress. But the naive analysis suggests these are a good way to get exposure to something we want

en-gb

21:28

“Evolution and the Low Road to Nash” by Aydin...

LessWrong (30+ Karma) ·

2025/01/23

Solution concepts in game theory—like the Nash equilibrium and its refinements—are used in two key ways. Normatively, they proscribe how rational agents ought to behave. Descriptively, they propose how agents actually behave when interactions settle into equilibrium. The Nash equilibrium[1] underpins much of modern game theory and its applications in economics, political science, and evolutionary biology. Here, we focus on the descriptive use of the concept in game theory. To do so, our first

en-gb

16:12

“Detect Goodhart and shut down” by Jeremy Gillen

LessWrong (30+ Karma) ·

2025/01/23

A common failure of optimizers is Edge Instantiation. An optimizer often finds a weird or extreme solution to a problem when the optimization objective is imperfectly specified. For the purposes of this post, this is basically the same phenomenon as Goodhart's Law, especially Extremal and Causal Goodhart. With advanced AI, we are worried about plans created by optimizing over predicted consequences of the plan, potentially achieving the goal in an unexpected way.In this post, I want to draw an

en-gb

14:00

“Against blanket arguments against interpretability”...

LessWrong (30+ Karma) ·

2025/01/23

On blanket criticism and refutationIn his long post, Charbel-Raphaël's argues against theories of impacts of interpretability. I think it's a largely a good, well-argued post, and if the only thing you get out of it is reading that post, I'll be contributing to improving the discourse. There is other material with similar claims that I think are made with low context, and also I should say that I'm not very versed in the history and the various versions of the debate. At the same time, I disagree

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

The Automated Daily

TrendTeller

All On AI

All On AI

Elon Musk Podcast

Stage Zero

Reflections

Livewire Labs

科技早8点

之远不知远