PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

12:59

“Logical Line-Of-Sight Makes Games Sequential or...

LessWrong (30+ Karma) ·

2024/01/19

In the last post, we talked about strategic time and the strategic time loops studied in open-source game theory. In that context, agents have logical line-of-sight to each other and the situation they're both facing, which creates a two-way information flow at the time each is making their decision. In this post I'll describe how agents in one context can use this logical line-of-sight to condition their behavior on how they behave in other contexts. This in turn makes those contexts strategically

en-gb

46:00

[Linkpost] “Manifund: 2023 in Review” by Austin Chen

LessWrong (30+ Karma) ·

2024/01/18

This is a linkpost for https://manifund.substack.com/p/manifund-2023-in-review Manifund is a new funding org that experiments with systems and software to support awesome projects. In 2023, we built a website (manifund.org) and donor ecosystem supporting three main programs: impact certificates, regranting, and an open call for applications. We allocated $2m across dozens of charitable projects, primarily in AI safety and effective altruism cause areas. Here's a breakdown of what Manifund

en-gb

1:04

[Linkpost] “Against Nonlinear (Thing Of Things)” by...

LessWrong (30+ Karma) ·

2024/01/18

This is a linkpost for https://thingofthings.substack.com/p/against-nonlinearAgainst Nonlinear is a followup that Ozy wrote to Practically A Book Review: Appendix to "Nonlinear's Evidence: Debunking False and Misleading Claims". In it, they lay out 4 trends in Effective Altruism where Nonlinear was especially bad: Do It For The Gram: Nonlinear was turning Effective Altruism into being Instagrammable instead of making the world betterSo-Called Top EAs: Nonlinear was ranking EAs based on their

en-gb

12:02

[Linkpost] “The True Story of How GPT-2 Became...

LessWrong (30+ Karma) ·

2024/01/18

This is a linkpost for https://youtu.be/qV_rOlHjvvsThis video recounts an incident that occurred at OpenAI in which flipping a single minus sign led the RLHF process to make GPT-2 only output sexually explicit continuations. The incident is described in OpenAI's paper "Fine-Tuning Language Models from Human Preferences" under section 4.4: "Bugs can optimize for bad behavior".The script has been written by Jai, with some significant input and rework by me. You can read it below.In 2019, one OpenAI

en-gb

72:22

“On the abolition of man” by Joe Carlsmith

LessWrong (30+ Karma) ·

2024/01/18

(Cross-posted from my website. Podcast version here, or search for "Joe Carlsmith Audio" on your podcast app. This essay is part of a series that I'm calling "Otherness and control in the age of AGI." I'm hoping that the individual essay can be read fairly well on their own, but see here for brief summaries of the essays that have been released thus far.) Earlier in this series, I discussed a certain kind of concern about the AI alignment discourse – namely, that it aspires to exert an inappropriate

en-gb

7:03

“Good job opportunities for helping with the most...

LessWrong (30+ Karma) ·

2024/01/18

Yes, this is my first post in almost a year. I’m no longer prioritizing this blog, but I will still occasionally post something. I wrote ~2 years ago that it was hard to point to concrete opportunities to help the most important century go well. That's changing. There are a good number of jobs available now that are both really promising opportunities to help (in my opinion) and are suitable for people without a lot of pre-existing knowledge of AI risk (or even AI). The jobs are demanding, but

en-gb

93:49

“AI #48: Exponentials in Geometry” by Zvi

LessWrong (30+ Karma) ·

2024/01/18

The biggest event of the week was the Sleeper Agents paper from Anthropic. I expect that to inform our thoughts for a while to come, and to lay foundation for additional work. We also had the first third of the IMO solved at almost gold metal level by DeepMind, discovering that math competition geometry is actually mostly composed of One Weird Trick. I knew that at the time I was doing it, though, and it was still really hard. As usual, there was also a bunch of other stuff. Tomorrow the 19th,

en-gb

5:20

“Does literacy remove your ability to be a bard as...

LessWrong (30+ Karma) ·

2024/01/18

Epistemic status: probably we did lose the ability to memorize long songs due to improper practice, but it may be possible to enjoy the benefits of literacy and epic memory simultaneously.Thanks to Niels uit de Bos for better links and editing, and to Ryan Kidd for encouragement to post.You probably know that Socrates thought writing was terrible and it would destroy people's ability to memorize things, because now they’re written down and don’t need to be memorized. I always thought that was a

en-gb

28:50

“Four visions of Transformative AI success” by Steven...

LessWrong (30+ Karma) ·

2024/01/17

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. Tl;drWhen people work towards making a good future in regards to Transformative AI (TAI), what's the vision of the future that they have in mind and are working towards?I’ll propose four (caricatured) answers that different people seem to give: (Vision 1) “Helper AIs”,(Vision 2) “Autonomous AIs”,(Vision 3) “Supercharged biological human brains”,(Vision 4) “Don’t build TAI”.For each of these four, I will go

en-gb

1:20

[Linkpost] “AlphaGeometry: An Olympiad-level AI...

LessWrong (30+ Karma) ·

2024/01/17

This is a linkpost for https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/[Published today by DeepMind]Our AI system surpasses the state-of-the-art approach for geometry problems, advancing AI reasoning in mathematicsReflecting the Olympic spirit of ancient Greece, the International Mathematical Olympiad is a modern-day arena for the world's brightest high-school mathematicians. The competition not only showcases young talent, but has emerged as a testing

en-gb

62:27

“On Anthropic’s Sleeper Agents Paper” by Zvi

LessWrong (30+ Karma) ·

2024/01/17

The recent paper from Anthropic is getting unusually high praise, much of it I think deserved. The title is: Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. Scott Alexander also covers this, offering an excellent high level explanation, of both the result and the arguments about whether it is meaningful. You could start with his write-up to get the gist, then return here if you still want more details, or you can read here knowing that everything he discusses is

en-gb

15:53

“An Introduction To The Mandelbrot Set That Doesn’t...

LessWrong (30+ Karma) ·

2024/01/17

Note: This post assumes you've heard of the Mandelbrot set before, and you want to know more about it, but that you find imaginary and complex numbers (e.g. the square root of negative one) a bit mystifying and counterintuitive. Instead of helping you understand the relevant math like a reasonable person would, I'm just going to pretend the concept doesn't exist, and try to explain how to generate the Mandelbrot set anyway.My goal is for this post to (theoretically) be acceptable to the historical

en-gb

54:05

“Medical Roundup #1” by Zvi

LessWrong (30+ Karma) ·

2024/01/16

Saving up medical and health related stories from several months allowed for much better organizing of them, so I am happy I split these off. I will still post anything more urgent on a faster basis. There's lots of things here that are fascinating and potentially very important, but I’ve had to prioritize and focus elsewhere, so I hope others pick up various torches. Vaccination Ho! We have a new malaria vaccine. That's great. WHO thinks this is not an especially urgent opportunity, or

en-gb

52:06

“Being nicer than Clippy” by Joe Carlsmith

LessWrong (30+ Karma) ·

2024/01/16

(Cross-posted from my website. Podcast version here, or search "Joe Carlsmith Audio" on your podcast app. This essay is part of a series I'm calling "Otherness and control in the age of AGI." I'm hoping that the individual essays can be read fairly well on their own, but see here for a summary of the essays that have been released thus far.) In my last essay, I discussed a certain kind of momentum, in some of the philosophical vibes underlying the AI risk discourse,[1] towards deeming more and more

en-gb

18:34

“Managing catastrophic misuse without robust AIs” by...

LessWrong (30+ Karma) ·

2024/01/16

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.Many people worry about catastrophic misuse of future AIs with highly dangerous capabilities. For instance, powerful AIs might substantially lower the bar to building bioweapons or allow for massively scaling up cybercrime. How could an AI lab serving AIs to customers manage catastrophic misuse? One approach would be to ensure that when future powerful AIs are asked to perform tasks in these problematic domains,

en-gb

11:31

“Why wasn’t preservation with the goal of potential...

LessWrong (30+ Karma) ·

2024/01/16

Cross-posted from my blog, Neurobiology Notes. John Hunter (1728–1793) did not have an especially promising start to his academic life. He was born the youngest of 10 children to a family living in the countryside near Glasgow. They lived in a two bedroom cottage and the children slept in box beds that were pulled out of the walls every night. He was stubborn, hated school, did not like to be taught reading or writing, would skip classes whenever he could, and quit formal education altogether at 13,

en-gb

24:23

“The impossible problem of due process” by mingyuan

LessWrong (30+ Karma) ·

2024/01/16

I wrote this entire post in February of 2023, during the fallout from the TIME article. I didn't post it at the time for multiple reasons: because I had no desire to get involved in all that nonsensebecause I was horribly burned out from my own community conflict investigation and couldn't stand the thought of engaging with people onlinebecause I generally think it's bad to post on the internet out of frustration or outrageBut after sitting on it for a full year, I still think it's worth posting, so

en-gb

37:38

“Sparse Autoencoders Work on Attention Layer Outputs”...

LessWrong (30+ Karma) ·

2024/01/16

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This post is the result of a 2 week research sprint project during the training phase of Neel Nanda's MATS stream. Executive Summary We replicate Anthropic's MLP Sparse Autoencoder (SAE) paper on attention outputs and it works well: the SAEs learn sparse, interpretable features, which gives us insight into what attention layers learn. We study the second attention layer of a two layer language model (with MLPs)

en-gb

14:15

“Goals selected from learned knowledge: an...

LessWrong (30+ Karma) ·

2024/01/15

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. Summary: Alignment work on network-based AGI focuses on reinforcement learning. There is an alternative approach that avoids some, but not all, of the difficulties of RL alignment. Instead of trying to build an adequate representation of the behavior and goals we want, by specifying rewards, we can choose its goals from the representations it has learned through any learning method. I give three examples of this

en-gb

6:43

“The case for training frontier AIs on Sumerian-only...

LessWrong (30+ Karma) ·

2024/01/15

Let your every day be full of joy, love the child that holds your hand, let your wife delight in your embrace, for these alone are the concerns of humanity.[1]— Epic of Gilgamesh - Tablet X Say we want to train a scientist AI to help in a precise, narrow field of science (e.g. medicine design) but prevent its power from being applied anywhere else (e.g. chatting with humans, designing bio-weapons, etc.) even if it has these abilities.Here's one safety layer one could implement: Train a scientist

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Reflections

Livewire Labs

ForGeeks

ForGeeks

FOX on Tech

FOX News Podcasts

The Linux Cast

The Linux Cast