PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

9:57

“Near-mode thinking on AI” by Olli Järviniemi

LessWrong (30+ Karma) ·

2024/08/05

There is a stark difference between rehearsing classical AI risk 101 arguments about instrumental convergence, and tackling problems like "Design and implement the exact evaluations we'll run on GPT-5 to determine whether it's capable enough that we should worry about it acting aligned until it can execute a takeover". And naturally, since I've started working on problems like the one above, I've noticed a large shift in my thinking on AI. I describe it as thinking about risks in near-mode, as

en-gb

10:53

“PIZZA: An Open Source Library for Closed LLM...

LessWrong (30+ Karma) ·

2024/08/04

From the research & engineering team at Leap Laboratories (incl. @Arush, @sebastian-sosa, @Robbie McCorkell), where we use AI interpretability to accelerate scientific discovery from data. This post is about our LLM attribution repo PIZZA: Prompt Input Z? Zonal Attribution. (In the grand scientific tradition we have tortured our acronym nearly to death. For the crimes of others see [1].) All examples in this post can be found in this notebook, which is also probably the easiest way to start

en-gb

8:44

“You don’t know how bad most things are nor precisely...

LessWrong (30+ Karma) ·

2024/08/04

TL;DR: Your discernment in a subject often improves as you dedicate time and attention to that subject. The space of possible subjects is huge, so on average your discernment is terrible, relative to what it could be. This is a serious problem if you create a machine that does everyone's job for them.See also: Reality has a surprising amount of detail. (You lack awareness of how bad your staircase is and precisely how your staircase is bad.) You don't know what you don't know. You forget your own

en-gb

6:17

“SRE’s review of Democracy” by Martin Sustrik

LessWrong (30+ Karma) ·

2024/08/04

Day OneWe've been handed this old legacy system called "Democracy". It's an emergency. The old maintainers are saying it has been misbehaving lately but they have no idea how to fix it. We've had a meeting with them to find out as much as possible about the system, but it turns out that all the original team members left the company long time ago. The current team doesn't have much understanding of the system beyond some basic operational knowledge.We've conducted a cursory code review, focusing

en-gb

5:21

“Some comments on intelligence” by Viliam

LessWrong (30+ Karma) ·

2024/08/03

After reading another article on IQ, there are a few things that I wish would become common knowledge to increase the quality of the debate. Posting them here:1)There is a difference between an abstract definition of intelligence such that it could also apply to aliens or AIs (something like "an agent able to optimize for outcomes in various environments") and the specific way the intelligence is implemented in human brains. Because of the implementation details, things can be true about human

en-gb

7:21

“The Bitter Lesson for AI Safety Research” by adamk,...

LessWrong (30+ Karma) ·

2024/08/03

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.Read the associated paper "Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?": https://arxiv.org/abs/2407.21792 Focus on safety problems that aren’t solved with scale.Benchmarks are crucial in ML to operationalize the properties we want models to have (knowledge, reasoning, ethics, calibration, truthfulness, etc.). They act as a criterion to judge the quality of models and drive implicit

en-gb

13:57

“A Simple Toy Coherence Theorem” by johnswentworth,...

LessWrong (30+ Karma) ·

2024/08/02

Audio note: this article contains 88 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. This post presents a simple toy coherence theorem, and then uses it to address various common confusions about coherence arguments. SettingDeterministic MDP. That means at each time _t_ there's a state _S[t]_[1], the agent/policy takes an action _A[t]_

en-gb

3:02

“AI Rights for Human Safety” by Simon Goldstein

LessWrong (30+ Karma) ·

2024/08/02

This is a link post.Just wanted to share a new paper on AI rights, co-authored with Peter Salib, that members of this community might be interested in. Here's the abstract:AI companies are racing to create artificial general intelligence, or “AGI.” If they succeed, the result will be human-level AI systems that can independently pursue high-level goals by formulating and executing long-term plans in the real world. Leading AI researchers agree that some of these systems will likely be

en-gb

30:17

“The ‘strong’ feature hypothesis could be wrong” by...

LessWrong (30+ Karma) ·

2024/08/02

NB. I am on the Google Deepmind language model interpretability team. But the arguments/views in this post are my own, and shouldn't be read as a team position. “It would be very convenient if the individual neurons of artificial neural networks corresponded to cleanly interpretable features of the input. For example, in an “ideal” ImageNet classifier, each neuron would fire only in the presence of a specific visual feature, such as the color red, a left-facing curve, or a dog snout” : Elhage et

en-gb

18:28

“The need for multi-agent experiments” by Martín Soto

LessWrong (30+ Karma) ·

2024/08/01

TL;DR: Let's start iterating on experiments that approximate real, society-scale multi-AI deploymentEpistemic status: These ideas seem like my most prominent delta with the average AI Safety researcher, have stood the test of time, and are shared by others I intellectually respect. Please attack them fiercely! Multi-polar risksSome authors have already written about multi-polar AI failure. I especially like how Andrew Critch has tried to sketch concrete stories for it.But, without even considering

en-gb

3:56

“Dragon Agnosticism” by jefftk

LessWrong (30+ Karma) ·

2024/08/01

I'm agnostic on the existence of dragons. I don't usually talk aboutthis, because people might misinterpret me as actually being a covert dragon-believer, but I wanted to give some background for why I disagree with calls for people to publicly assert the non-existence of dragons. Before I do that, though, it's clear that horrible acts have been committed in the name of dragons. Many dragon-believers publicly or privately endorse this reprehensible history. Regardless of whether dragons do

en-gb

133:39

“AI #75: Math is Easier” by Zvi

LessWrong (30+ Karma) ·

2024/08/01

Google DeepMind got a silver metal at the IMO, only one point short of the gold. That's really exciting.We continuously have people saying ‘AI progress is stalling, it's all a bubble’ and things like that, and I always find remarkable how little curiosity or patience such people are willing to exhibit. Meanwhile GPT-4o-Mini seems excellent, OpenAI is launching proper search integration, by far the best open weights model got released, we got an improved MidJourney 6.1, and that's all in the last

en-gb

135:49

“AI #75: Math is Easier” by Zvi

LessWrong (30+ Karma) ·

2024/08/01

Google DeepMind got a silver metal at the IMO, only one point short of the gold. That's really exciting. We continuously have people saying ‘AI progress is stalling, it's all a bubble’ and things like that, and I always find remarkable how little curiosity or patience such people are willing to exhibit. Meanwhile GPT-4o-Mini seems excellent, OpenAI is launching proper search integration, by far the best open weights model got released, we got an improved MidJourney 6.1, and that's all in the last

en-gb

22:24

“Recommendation: reports on the search for missing...

LessWrong (30+ Karma) ·

2024/08/01

This is a link post.Content warning: About an IRL death.Today's post isn’t so much an essay as a recommendation for two bodies of work on the same topic: Tom Mahood's blog posts and Adam “KarmaFrog1” Marsland's videos on the 2010 disappearance of Bill Ewasko, who went for a day hike in Joshua Tree National Park and dropped out of contact.2010 – Bill Ewasko goes missing Tom Mahood's writeups on the search [Blog post, website goes down sometimes so if the site doesn’t work, check the internet

en-gb

5:42

“Ambiguity in Prediction Market Resolution is Still...

LessWrong (30+ Karma) ·

2024/08/01

A brief followup to this post in light of recent events. Free and Fair ElectionsPolymarket has an open market 'Venezuela Presidential Election Winner'. Its description is as follows:The 2024 Venezuela presidential election is scheduled to take place on July 28, 2024. This market will resolve to "Yes" if Nicolás Maduro wins. Otherwise, this market will resolve to "No." This market includes any potential second round. If the result of this election isn't known by December 31, 2024, 11:59 PM ET, the

en-gb

3:40

“Twitter thread on open-source AI” by Richard_Ngo

LessWrong (30+ Karma) ·

2024/07/31

This is a link post.Some thoughts on open-source AI (copied over from a recent twitter thread):1. We should have a strong prior favoring open source. It's been a huge success driving tech progress over many decades. We forget how counterintuitive it was originally, and shouldn’t take it for granted.2. Open source has also been very valuable for alignment. It's key to progress on interpretability, as outlined here.3. I am concerned, however, that offense will heavily outpace defense in the long term

en-gb

5:22

“Twitter thread on AI takeover scenarios” by Richard_Ngo

LessWrong (30+ Karma) ·

2024/07/31

This is a link post.This is a slightly-edited version of a twitter thread I posted a few months ago about "internal deployment" threat models.My former colleague Leopold argues compellingly that society is nowhere near ready for AGI. But what might the large-scale alignment failures he mentions actually look like? Here's one scenario for how building misaligned AGI could lead to humanity losing control.Consider a scenario where human-level AI has been deployed across society to help with a wide

en-gb

2:47

“Twitter thread on politics of AI safety” by Richard_Ngo

LessWrong (30+ Karma) ·

2024/07/31

This is a link post.Some thoughts about the politics of AI safety, copied over (with slight modifications) from my recent twitter thread:Risks that seem speculative today will become common sense as AI advances. The pros and cons of different safety strategies will also become much clearer over time. So our main job now is to empower future common-sense decision-making. Understanding model cognition and behavior is crucial for making good decisions. But equally important is ensuring that key

en-gb

2:32

“If You Can Climb Up, You Can Climb Down” by jefftk

LessWrong (30+ Karma) ·

2024/07/31

A few weeks ago Julia wrote about how we approach kids climbing: The basics: Spot the child if they're doing something where a fall is likely. Don't encourage or help the child to climb something that's beyond their ability to do on their own. If they don't know how to get down, give advice rather than physically lifting them down. Don't allow climbing on some places that are too dangerous. I was thinking about this some when I was at the park with Nora (3y) a few days ago

en-gb

31:30

“Open Source Automated Interpretability for Sparse...

LessWrong (30+ Karma) ·

2024/07/31

This is a link post.Generated by Dalle BackgroundSparse autoencoders recover a diversity of interpretable, monosemantic features, but present an intractable problem of scale to human labelers. We investigate different techniques for generating and scoring text explanations of SAE features. Key Findings Open source models generate and evaluate text explanations of SAE features reasonably well, albeit somewhat worse than closed models like Claude 3.5 Sonnet.Explanations found by LLMs are similar to

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Reflections

Livewire Labs

ForGeeks

ForGeeks

FOX on Tech

FOX News Podcasts

The Linux Cast

The Linux Cast