PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

13:08

“Proveably Safe Self Driving Cars ” by Davidmanheim

LessWrong (30+ Karma) ·

2024/09/15

I’ve seen a fair amount of skepticism about the “Provably Safe AI” paradigm, but I think detractors give it too little credit. I suspect this is largely because of idea inoculation - people have heard an undeveloped or weak man version of the idea, for example, that we can use formal methods to state our goals and prove that an AI will do that, and have already dismissed it. (Not to pick on him at all, but see my question for Scott Aaronson here.)I will not argue that Guaranteed Safe AI solves AI

en-gb

2:30

“Not every accommodation is a Curb Cut Effect: The...

LessWrong (30+ Karma) ·

2024/09/15

This is a link post.In the fields of user experience and accessibility, everyone talks about the curb cut effect: Features that are added as accommodations for people with disabilities sometimes become widely useful and beloved. But not every accommodation becomes a "curb cut," and I've been thinking about other patterns that come up when accommodations intersect with wider society. [... Summary from the end of the post]First, there's a trio of scenarios in which an accommodation proves helpful to

en-gb

12:54

“Did Christopher Hitchens change his mind about...

LessWrong (30+ Karma) ·

2024/09/15

There's a popular story that goes like this: Christopher Hitchens used to be in favor of the US waterboarding terrorists because he though it's wasn't bad enough to be torture.. Then he had it tried on himself, and changed his mind, coming to believe it isn't torture.(Context for those unfamiliar: in the decade following 9/11, the US engaged in a lot of... questionable behavior to persecute the war on terror, and there was a big debate on whether waterboarding should be permitted. Many other public

en-gb

3:09

“OpenAI o1, Llama 4, and AlphaZero of LLMs ” by...

LessWrong (30+ Karma) ·

2024/09/15

GPT-4 level open weights models like Llama-3-405B don't seem capable of dangerous cognition. OpenAI o1 demonstrates that a GPT-4 level model can be post-trained into producing useful long horizon reasoning traces. AlphaZero shows how capabilities can be obtained from compute alone, with no additional data. If there is a way of bringing these together, the apparent helplessness of the current generation of open weights models might prove misleading. Post-training is currently a combination of

en-gb

51:45

“If-Then Commitments for AI Risk Reduction [by Holden...

LessWrong (30+ Karma) ·

2024/09/14

Holden just published this paper on the Carnegie Endowment website. I thought it was a decent reference, so I figured I would crosspost it.If-then commitments are an emerging framework for preparing for risks from AI without unnecessarily slowing the development of new technology. The more attention and interest there is these commitments, the faster a mature framework can progress. Introduction Artificial intelligence (AI) could pose a variety of catastrophic risks to international security in

en-gb

12:37

“Evidence against Learned Search in a Chess-Playing...

LessWrong (30+ Karma) ·

2024/09/14

IntroductionThere is a new paper and lesswrong post about "learned look-ahead in a chess-playing neural network". This has long been a research interest of mine for reasons that are well-stated in the paper: Can neural networks learn to use algorithms such as look-ahead or search internally? Or are they better thought of as vast collections of simple heuristics or memorized data? Answering this question might help us anticipate neural networks’ future capabilities and give us a better understanding

en-gb

47:54

“Estimating Tail Risk in Neural Networks ” by...

LessWrong (30+ Karma) ·

2024/09/13

Audio note: this article contains 292 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. Machine learning systems are typically trained to maximize average-case performance. However, this method of training can fail to meaningfully control the probability of tail events that might cause significant harm. For instance, while an artificial intelligence (AI) assistant may be generally safe, it would be catastrophic if

en-gb

1:14

“AI, centralization, and the One Ring ” by owencb

LessWrong (30+ Karma) ·

2024/09/13

This is a link post.People thinking about the future of AI sometimes talk about a single project ‘getting there first’ — achieving AGI, and leveraging this into a decisive strategic advantage over the rest of the world. That would represent a massive concentration of power.This post is a one-sided exploration of concerns with such concentration of power. They're conveyed in part via an extended analogy with The Lord of the Rings, in the hope that that could make it easier to keep track of the

en-gb

18:27

“The Great Data Integration Schlep ” by sarahconstantin

LessWrong (30+ Karma) ·

2024/09/13

Midjourney, “Fourth Industrial Revolution Digital Transformation”This is a little rant I like to give, because it's something I learned on the job that I’ve never seen written up explicitly.There are a bunch of buzzwords floating around regarding computer technology in an industrial or manufacturing context: “digital transformation”, “the Fourth Industrial Revolution”, “Industrial Internet of Things”.What do those things really mean?Do they mean anything at all?The answer is yes, and what they mean

en-gb

19:32

“Open Problems in AIXI Agent Foundations ” by Cole Wyeth

LessWrong (30+ Karma) ·

2024/09/13

Audio note: this article contains 42 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. I believe that the theoretical foundations of the AIXI agent and variations are a surprisingly neglected and high leverage approach to agent foundations research. Though discussion of AIXI is pretty ubiquitous in A.I. safety spaces, underscoring AIXI's usefulness as a model of superintelligence, this is usually limited to poorly

en-gb

9:55

“How to give in to threats (without incentivizing...

LessWrong (30+ Karma) ·

2024/09/13

TL;DR: using a simple mixed strategy, LDT can give in to threats, ultimatums, and commitments - while incentivizing cooperation and fair[1] splits instead.This strategy made it much more intuitive to many people I've talked to that smart agents probably won't do weird everyone's-utility-eating things like threatening each other or participating in commitment races. 1. The Ultimatum gameThis part is taken from planecrash[2][3].You're in the Ultimatum game. You're offered 0-10 dollars. You can accept

en-gb

18:19

“Contra papers claiming superhuman AI forecasting ”...

LessWrong (30+ Karma) ·

2024/09/12

[Conflict of interest disclaimer: We are FutureSearch, a company working on AI-powered forecasting and other types of quantitative reasoning. If thin LLM wrappers could achieve superhuman forecasting performance, this would obsolete a lot of our work.] Widespread, misleading claims about AI forecastingRecently we have seen a number of papers – (Schoenegger et al., 2024, Halawi et al., 2024, Phan et al., 2024, Hsieh et al., 2024) – with claims that boil down to “we built an LLM-powered forecaster

en-gb

0:23

“OpenAI o1 ” by Zach Stein-Perlman

LessWrong (30+ Karma) ·

2024/09/12

This is a link post. ---First published: September 12th, 2024 Source: https://www.lesswrong.com/posts/bhY5aE4MtwpGf3LCo/openai-o1 --- Narrated by TYPE III AUDIO.

en-gb

67:58

“AI #81: Alpha Proteo ” by Zvi

LessWrong (30+ Karma) ·

2024/09/12

Following up on Alpha Fold, DeepMind has moved on to Alpha Proteo. We also got a rather simple prompt that can create a remarkably not-bad superforecaster for at least some classes of medium term events. We did not get a new best open model, because that turned out to be a scam. And we don’t have Apple Intelligence, because it isn’t ready for prime time. We also got only one very brief mention of AI in the debate I felt compelled to watch. What about all the apps out there, that we haven’t

en-gb

25:22

“[Paper] Programming Refusal with Conditional...

LessWrong (30+ Karma) ·

2024/09/12

Audio note: this article contains 76 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. Refer to the arXiv preprint for full content. This post is a lighter, 15-minute version. Abstract Existing activation steering methods alter LLM behavior indiscriminately, limiting their practical applicability in settings where selective responses are essential, such as content moderation or domain-specific assistants.We propose

en-gb

6:25

“Refactoring cryonics as structural brain...

LessWrong (30+ Karma) ·

2024/09/11

I first learned about cryonics when I read Eliezer and Robin's posts about it on Overcoming Bias years ago. I got cryopilled. Somewhat amazingly to me, I'm now a researcher in this field. So I thought this community might be interested to know that I was one of several co-authors on a paper just published in Frontiers in Medical Technology, titled "Structural brain preservation: a potential bridge to future medical technologies".In this paper, we propose reframing cryonics as a type of structural

en-gb

5:22

“Reformative Hypocrisy, and Paying Close Enough...

LessWrong (30+ Karma) ·

2024/09/11

People often attack frontier AI labs for "hypocrisy" when the labs admit publicly that AI is an extinction threat to humanity. Often these attacks ignore the difference between various kinds of hypocrisy, some of which are good, including what I'll call "reformative hypocrisy". Attacking good kinds of hypocrisy can be actively harmful for humanity's ability to survive, and as far as I can tell we (humans) usually shouldn't do that when our survival is on the line. Arguably, reformative hypocrisy

en-gb

1:17

“Formalizing the Informal (event invite) ” by...

LessWrong (30+ Karma) ·

2024/09/11

Formalizing the InformalOne way to view MIRI's Agent Foundations research is that it saw the biggest problem in AI safety as "human preferences are informal, but we need to somehow get formal guarantees about them" -- and so, in response, it set out to make a formal-informal bridge.Recently, I’ve been thinking about how we might formally represent the difference between formal and informal. My prompt is something like: if we assume that classical probability theory applies to “fully formal”

en-gb

8:47

“The Best Lay Argument is not a Simple English Yud...

LessWrong (30+ Karma) ·

2024/09/10

Epistemic status: these are my own opinions on AI risk communication, based primarily on my own instincts on the subject and discussions with people less involved with rationality than myself. Communication is highly subjective and I have not rigorously A/B tested messaging. I am even less confident in the quality of my responses than in the correctness of my critique.If they turn out to be true, these thoughts can probably be applied to all sorts of communication beyond AI risk.Lots of work has

en-gb

11:09

“AI forecasting bots incoming ” by Dan H, Mantas Mazeika

LessWrong (30+ Karma) ·

2024/09/10

In a recent appearance on Conversations with Tyler, famed political forecaster Nate Silver expressed skepticism about AIs replacing human forecasters in the near future. When asked how long it might take for AIs to reach superhuman forecasting abilities, Silver replied: “15 or 20 [years].”In light of this, we are excited to announce “FiveThirtyNine,” an AI forecasting bot. Our bot, built on GPT-4o, provides probabilities for any user-entered query, including “Will Trump win the 2024 presidential

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

The Automated Daily

TrendTeller

Reflections

Livewire Labs

Eli the Computer Guy

Eli the Computer Guy

LessWrong (30+ Karma)

LessWrong

Superkraft KI News | by Jörg Schieb

Jörg Schieb