PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

16:43

“Moderately Skeptical of ‘Risks of Mirror Biology’”...

LessWrong (30+ Karma) ·

2024/12/21

Last week, a report was released on the potential risks of mirror biology - that is, biological systems with reversed chirality. I had previously looked into the topic, and was skeptical of the viability of this as a global catastrophic or existential risk. This report was tremendously helpful in filling in details of a scenario where, if mirror bacteria are developed, they could cause a global catastrophe. I will assume that readers here have or will read at least a summary, if not the original

en-gb

7:45

“The nihilism of NeurIPS” by charlieoneill

LessWrong (30+ Karma) ·

2024/12/21

"What is the use of having developed a science well enough to make predictions if, in the end, all we're willing to do is stand around and wait for them to come true?" F. SHERWOOD HOWLAND in his speech accepting the Nobel Prize in Chemistry in 1995. "Once upon a time on Tralfamadore there were creatures who weren’t anything like machines. They weren’t dependable. They weren’t efficient. They weren’t predictable. They weren’t durable. And these poor creatures were obsessed by the idea that everything

en-gb

10:30

“Anthropic leadership conversation” by Zach...

LessWrong (30+ Karma) ·

2024/12/20

I quoted interesting-to-me parts and added some unjustified commentary. 20:00: Tom Brown says "the US treats the Constitution as like the holy document—which I think is just a big thing that strengthens the US, like we don't expect the US to go off the rails in part because just like every single person in the US is like The Constitution is a big deal, and if you tread on that, like, I'm mad. I think that the RSP, like, it holds that thing. It's like the holy document for Anthropic. So it's worth

en-gb

12:52

“You can validly be seen and validated by a chatbot”...

LessWrong (30+ Karma) ·

2024/12/20

There's a common sentiment saying that a chatbot can’t really make you feel seen or validated. As chatbots are (presumably) not sentient, they can’t see you and thus can’t make you seen either. Or if they do, it is somehow fake and it's bad that you feel that way.So let me tell you about ways in which Claude Sonnet makes me feel seen, and how I think those are valid. I was describing an essay idea to Claude. The essay is about something I call “psychological charge”, where the idea is that there

en-gb

18:38

“🇫🇷 Announcing CeSIA: The French Center for AI...

LessWrong (30+ Karma) ·

2024/12/20

Executive SummaryThe Centre pour la Sécurité de l'IA[1] (CeSIA, pronounced "seez-ya") or French Center for AI Safety is a new Paris-based organization dedicated to fostering a culture of AI safety at the French and European level. Our mission is to reduce AI risks through education and information about potential risks and their solutions.Our activities span three main areas: Education:We began our work by creating the ML4Good bootcamps, which have been replicated a dozen times worldwide. We also

en-gb

11:25

“A progress policy agenda” by jasoncrawford

LessWrong (30+ Karma) ·

2024/12/20

Elon Musk says that soon, builders “will be free to build” in America. If that promise is to be fulfilled, we have work to do.Here's my wishlist of policy goals to advance scientific, technological, and economic progress. I’m far from a policy wonk, so I’m mostly going to be referencing folks I trust, such as the RPI fellows, the Institute for Progress (IFP), or Eli Dourado at the Abundance Institute. (I’m sympathetic to most of what is linked below, and consider all of it interesting and

en-gb

0:47

“o3” by Zach Stein-Perlman

LessWrong (30+ Karma) ·

2024/12/20

I'm editing this post.OpenAI announced (but hasn't released) o3 (skipping o2 for trademark reasons).It gets 25% on FrontierMath, smashing the previous SoTA of 2%. (These are really hard math problems.) Wow.72% on SWE-bench Verified, beating o1's 49%.Also 88% on ARC-AGI. --- First published: December 20th, 2024 Source: https://www.lesswrong.com/posts/Ao4enANjWNsYiSFqc/o3 --- Narrated by TYPE III AUDIO.

en-gb

9:27

“What Goes Without Saying” by sarahconstantin

LessWrong (30+ Karma) ·

2024/12/20

There are people I can talk to, where all of the following statements are obvious. They go without saying. We can just “be reasonable” together, with the context taken for granted.And then there are people who…don’t seem to be on the same page at all. There's a real way to do anything, and a fake way; we need to make sure we’re doing the real version.Concepts like Goodhart's Law, cargo-culting, greenwashing, hype cycles, Sturgeon's Law, even bullshit jobs1 are all pointing at the basic understanding

en-gb

13:16

“Retrospective: PIBBSS Fellowship 2024” by...

LessWrong (30+ Karma) ·

2024/12/20

Between June and September 2024, we ran the third iteration of the PIBBSS Summer Research Fellowship. Here are our reflections on how the program went and what we learned. Apply for the 2025 program here! TLDR:The 2024 fellowship demonstrated continued success in attracting and developing senior academic talent while refining our approach to interdisciplinary AI safety research. Key outcomes include: Successfully facilitating the transition of multiple early-to-mid-career researchers into AI

en-gb

11:41

“‘Alignment Faking’ frame is somewhat fake” by...

LessWrong (30+ Karma) ·

2024/12/20

I like the research. I mostly trust the results. I dislike the 'Alignment Faking' name and frame, and I'm afraid it will stick and lead to more confusion. This post offers a different frame. The main way I think about the result is: it's about capability - the model exhibits strategic preference preservation behavior; also, harmlessness generalized better than honesty; and, the model does not have a clear strategy on how to deal with extrapolating conflicting values. What happened in this frame

en-gb

12:15

“Claude’s Constitutional Consequentialism?” by 1a3orn

LessWrong (30+ Karma) ·

2024/12/20

TLDR: Recent papers have shown that Claude will sometimes act to achieve long-term goods rather than be locally honest. I think this preference may follow naturally from the Constitutional principles by which Claude was trained, which often emphasize producing a particular outcome over adherence to deontological rules.Epistemic status: Fumbling in the darkness. Famished for lack of further information. Needing many more empirical facts known only to Anthropic and those within. The Puzzle Several

en-gb

22:02

“Measuring whether AIs can statelessly strategize to...

LessWrong (30+ Karma) ·

2024/12/20

(Work done at Redwood Research. Thanks to Misha Wagner, Fabien Roger, Tomek Korbak, Justis Mills, Samuel Marks, Charlie Griffin, Zach Stein-Perlman, and Aryan Bhatt for feedback on drafts.)One way to show that risk from deploying an AI system is small is by showing that the model is not capable of subverting security measures enough to cause substantial harm. In AI control research so far, control evaluations have measured whether a red-team-created attack policy can defeat control measures, but

en-gb

4:22

“How to replicate and extend our alignment faking...

LessWrong (30+ Karma) ·

2024/12/19

This is a link post.We recently released a paper presenting a demonstration of alignment faking where we do not give the model any explicit goal, and instead study whether alignment faking can emerge due to the model's prior harmlessness training.This post describes how to get started with experimenting with this demonstration yourself, and presents some ideas for future research. How to run your own experiments Running quick experimentsWe release a simple notebook with a minimal reproduction of our

en-gb

11:21

“When Is Insurance Worth It?” by kqr

LessWrong (30+ Karma) ·

2024/12/19

TL;DR: If you want to know whether getting insurance is worth it, use the Kelly Insurance Calculator. If you want to know why or how, read on.Note to LW readers: this is almost the entire article, except some additional maths that I couldn't figure out how to get right in the LW editor, and margin notes. If you're very curious, read the original article! Misunderstandings about insurancePeople online sometimes ask if they should get some insurance, and then other people say incorrect things,

en-gb

3:08

“A Solution for AGI/ASI Safety” by Weibing Wang

LessWrong (30+ Karma) ·

2024/12/19

I have a lot of ideas about AGI/ASI safety. I've written them down in a paper and I'm sharing the paper here, hoping it can be helpful. Title: A Comprehensive Solution for the Safety and Controllability of Artificial SuperintelligenceAbstract:As artificial intelligence technology rapidly advances, it is likely to implement Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI) in the future. The highly intelligent ASI systems could be manipulated by malicious humans or

en-gb

114:15

“Takes on ‘Alignment Faking in Large Language...

LessWrong (30+ Karma) ·

2024/12/18

(Cross-posted from my website. Audio version here, or search for "Joe Carlsmith Audio" on your podcast app.)Researchers at Redwood Research, Anthropic, and elsewhere recently released a paper documenting cases in which the production version of Claude 3 Opus fakes alignment with a training objective in order to avoid modification of its behavior outside of training – a pattern of behavior they call “alignment faking,” and which closely resembles a behavior I called “scheming” in a report I wrote

en-gb

1:53

“Preppers Are Too Negative on Objects” by jefftk

LessWrong (30+ Karma) ·

2024/12/18

Don't just buy some gear, throw it in a closet, pat yourself on theback, and move on. You are not prepared unless you practice with your supplies and plans. —The Prepared Preppers are often pretty negative on people who, after realizing disasters are worth taking seriously want to go out and buy a bunch of stuff. And it makes sense: there is a lot of marketing aimed at preppers, pushing all sorts of things that are marginally useful, and it's healthy to have some pushback. On the other

en-gb

1:10

“Review: Breaking Free with Dr. Stone” by TurnTrout

LessWrong (30+ Karma) ·

2024/12/18

This is a link post.Doctor Stone is an anime where everyone suddenly turns into a statue. Civilization falls apart. Thousands of years pass, and then a superhumanly knowledgeable teenager named Senku wakes up due to nitric acid dripping on him for long enough.He eventually deduces that the nitric acid caused him to depetrify and drags his petrified friend Taiju under a drip of the acid. Taiju wakes up. The two begin Senku's quest to rebuild civilization from scratch using the power and knowledge of

en-gb

16:32

“Careless thinking: A theory of bad thinking” by...

LessWrong (30+ Karma) ·

2024/12/18

Have you ever noticed how differently we approach buying a car versus choosing what to watch on Netflix? One might involve spreadsheets, research, and asking friends for advice. The other? We might just click on whatever catches our eye. I argue a lot of our thinking is more like the second than the first. This argument was originally made to me by Katja Grace, but if you don’t like it it's probably my fault.When people take something seriously – like their weight, their children's education, or a

en-gb

4:07

“AI Safety Seed Funding Network - Join as a Donor or...

LessWrong (30+ Karma) ·

2024/12/18

Summary You can now express interest in joining Catalyze Impact's seed funding networks for AI safety organizations. We offer a non-profit Seed Funding Circle for individuals and organizations with $15k+ annual donation capacity and an Investor Network for VCs and angel investors with $20k+ annual investment capacity. Joining does not create any obligation to invest or donate. For more information, please visit this page. Expression of interest form With the AI safety field still being remarkably

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

The Automated Daily

TrendTeller

Reflections

Livewire Labs

Eli the Computer Guy

Eli the Computer Guy

LessWrong (30+ Karma)

LessWrong

Superkraft KI News | by Jörg Schieb

Jörg Schieb