PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

28:37

“Mechanisms too simple for humans to design” by...

LessWrong (30+ Karma) ·

2025/01/22

Cross-posted from Telescopic TurnipAs we all know, humans are terrible at building butterflies. We can make a lot of objectively cool things like nuclear reactors and microchips, but we still can't create a proper artificial insect that flies, feeds, and lays eggs that turn into more butterflies. That seems like evidence that butterflies are incredibly complex machines – certainly more complex than a nuclear power facility.Likewise, when you google "most complex object in the universe", the first

en-gb

7:04

“Kitchen Air Purifier Comparison” by jefftk

LessWrong (30+ Karma) ·

2025/01/22

I make breakfast for the kids most mornings, and one thing I didn't realize before I started playing with an air quality monitor was how much this puts smoke in the air. It's not like cooking Naan or searing meat where if I don't put a fan in the window the smoke alarm will go off, but apparently it's quite a lot of particles: While I initially got interested in air quality from an infection reduction perspective, I'd also prefer not to be breathing smoke. How much does an air purifier

en-gb

14:07

“Tell me about yourself:LLMs are aware of their...

LessWrong (30+ Karma) ·

2025/01/22

This is the abstract and introduction of our new paper, with some discussion of implications for AI Safety at the end. Authors: Jan Betley*, Xuchan Bao*, Martín Soto*, Anna Sztyber-Betley, James Chua, Owain Evans (*Equal Contribution). AbstractWe study behavioral self-awareness — an LLM's ability to articulate its behaviors without requiring in-context examples. We finetune LLMs on datasets that exhibit particular behaviors, such as (a) making high-risk economic decisions, and (b) outputting

en-gb

3:16

“Quotes from the Stargate press conference” by Nikola...

LessWrong (30+ Karma) ·

2025/01/22

This is a link post.Present alongside President Trump: Sam AltmanLarry Ellison (Oracle executive chairman and CTO)Masayoshi Son (Softbank CEO who believes he was born to realize ASI)President Trump: What we want to do is we want to keep [AI datacenters] in this country. China is a competitor and others are competitors. President Trump: I'm going to help a lot through emergency declarations because we have an emergency. We have to get this stuff built. So they have to produce a lot of electricity and

en-gb

2:18

“Almost all growth is exponential growth” by lemonhope

LessWrong (30+ Karma) ·

2025/01/22

Why is almost everything either overwhelming or nonexistent? Topics are discussed either constantly or never. You bike 200 miles per week or never. You are probably cancer free or dead from cancer. Your old stocks and coins are either worthless or worth a fortune. Stuff from my old popular science magazines is either in the palm of my hand or nonexistent. I can't step outside without stepping on a pigeon, but I haven't seen a cardinal in years. The answer is what you would expect. Memes and bike

en-gb

4:48

“Training on Documents About Reward Hacking Induces...

LessWrong (30+ Karma) ·

2025/01/21

This is a link post.This is a blog post reporting some preliminary work from the Anthropic Alignment Science team, which might be of interest to researchers working actively in this space. We'd ask you to treat these results like those of a colleague sharing some thoughts or preliminary experiments at a lab meeting, rather than a mature paper.We report a demonstration of a form of Out-of-Context Reasoning where training on documents which discuss (but don’t demonstrate) Claude's tendency to reward

en-gb

4:20

“The Manhattan Trap: Why a Race to Artificial...

LessWrong (30+ Karma) ·

2025/01/21

This is a link post. Below is the executive summary of our new paper, The Manhattan Trap. Please visit the link above to see the full paper. We also encourage discussion and feedback in the comments here. This paper examines the strategic dynamics of international competition to develop Artificial Superintelligence (ASI). We argue that the same assumptions that might motivate the US to race to develop ASI also imply that such a race is extremely dangerous. A race to develop ASI is motivated by two

en-gb

13:21

“The Case Against AI Control Research” by johnswentworth

LessWrong (30+ Karma) ·

2025/01/21

The AI Control Agenda, in its own words:… we argue that AI labs should ensure that powerful AIs are controlled. That is, labs should make sure that the safety measures they apply to their powerful models prevent unacceptably bad outcomes, even if the AIs are misaligned and intentionally try to subvert those safety measures. We think no fundamental research breakthroughs are required for labs to implement safety measures that meet our standard for AI control for early transformatively useful AIs; we

en-gb

14:14

“Things I have been using LLMs for” by Kaj_Sotala

LessWrong (30+ Karma) ·

2025/01/21

There are quite a few different things you can use LLMs for, and I think we’re still only discovering most of them. Here are a few of the ones I’ve come up with. My favorite chatbot is Claude Sonnet. It does have a tendency for sycophancy – for example, it will go “what a fascinating/insightful/excellent/etc. question!” in response to most of the things you might ask it. Some people find this annoying, while my brain just filters it out automatically. If you don’t like it, you can put in a custom

en-gb

2:31

“We don’t want to post again ‘This might be the last...

LessWrong (30+ Karma) ·

2025/01/21

This is a link post.We still need more funding to be able to run another edition. Our fundraiser raised $6k as of now, and will end if it doesn't reach the $15k minimum, on February 1st. We need proactive donors.If we don't get funded for this time, there is a good chance we will move on to different work in AI Safety and new commitments. This would make it much harder to reassemble the team to run future AISCs, even if the funding situation improves. You can take a look at the track record section

en-gb

6:58

“Announcement: Learning Theory Online Course” by...

LessWrong (30+ Karma) ·

2025/01/21

Hey everyone! Gergely and Kōshin here from ALTER and Monastic Academy, respectively. We are excited to announce a 6-week, 4-hours-per-week mathematics course designed to introduce a few early topics in what we are calling mathematical AI alignment (roughly, classical learning theory up to embedded agency and infra-Bayesianism, though this course will cover just an initial portion of this) aimed at folks with the math proficiency level of a university math major (formal university degree not

en-gb

15:30

“Retrospective: 12 [sic] Months Since MIRI” by...

LessWrong (30+ Karma) ·

2025/01/21

it's now been 15 months since MIRI but I just remembered that three separate people have told me they liked this post despite my not cross-posting it, so I am now cross-posting it.Not written with the intention of being useful to any particular audience, just collecting my thoughts on this past year's work. September-December 2023: Orienting and Threat ModelingUntil September, I was contracting full time for a project at MIRI. When the MIRI project ended, I felt very confused about lots of things in

en-gb

42:20

“Tips and Code for Empirical Research Workflows” by...

LessWrong (30+ Karma) ·

2025/01/21

Our research is centered on empirical research with LLMs. If you are conducting similar research, these tips and tools may help streamline your workflow and increase experiment velocity. We are also releasing two repositories to promote sharing more tooling within the AI safety community.John Hughes is an independent alignment researcher working with Ethan Perez and was a MATS mentee in the Summer of 2023. In Ethan's previous writeup on research tips, he explains the criteria that strong

en-gb

15:17

“Worries about latent reasoning in LLMs” by CBiddulph

LessWrong (30+ Karma) ·

2025/01/20

When working through a problem, OpenAI's o1 model will write a chain-of-thought (CoT) in English. This CoT reasoning is human-interpretable by default, and I think that this is hugely valuable. Assuming we can ensure that these thoughts are faithful to the model's true reasoning, they could be very useful for scalable oversight and monitoring. I'm very excited about research to help guarantee chain-of-thought faithfulness.[1]However, there's a impending paradigm for LLM reasoning that could make the

en-gb

7:39

“Broader implications of the FrontierMath debacle” by...

LessWrong (30+ Karma) ·

2025/01/20

Recently, OpenAI announced their newest model, o3, achieving massive improvements over state-of-the-art on reasoning and math. The highlight of the announcement was the fact that o3 scored 25% on FrontierMath, a benchmark by Epoch AI ridiculously hard, unseen math problems of which previous models could only solve 2%. The events after the announcement, however, highlight that apart from OpenAI having the answer sheet before taking the exam, this was shady and lacked transparency in every possible

en-gb

21:54

“Logits, log-odds, and loss for parallel circuits” by...

LessWrong (30+ Karma) ·

2025/01/20

Audio note: this article contains 51 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. Today I’m going to discuss how to think about logits like a statistician, and what this implies about circuits. This post doesn’t have any prerequisites other than perhaps a very basic statistical background that can be adequately recovered from the AI-generated “glossary” to the right. I think the material here is good thing to

en-gb

9:15

“The Monster in Our Heads” by testingthewaters

LessWrong (30+ Karma) ·

2025/01/20

"He who fights with monsters might take care lest he thereby become a monster. And if you gaze for long into an abyss, the abyss gazes also into you." - Friedrich NietzscheI think it is not an exaggeration to say that many people I know in this community hate the idea of powerful, unaligned AI. They describe it in apocalyptic terms, equate its existence to the death of everything they love, treat every step towards it as evil in its purest form. We as a community are haunted by pictures of paperclip

en-gb

2:33

“Agent Foundations 2025 at CMU” by Alexander...

LessWrong (30+ Karma) ·

2025/01/20

We are opening applications to attend a 5 day agent foundations conference at Carnegie Mellon University. The program will include talks, breakout sessions, and other activities. Apply here by January 26Key Information March 3-7, 2025At Carnegie Mellon in Pittsburgh, PA30-40 attendeesApply by January 26AboutJoin us for the 2025 Agent Foundations Conference at Carnegie Mellon University, March 3-7. This five-day gathering brings together leading researchers in AI alignment to explore fundamental

en-gb

0:35

“The Gentle Romance” by Richard_Ngo

LessWrong (30+ Karma) ·

2025/01/19

This is a link post.A story I wrote about living through the transition to utopia.This is the one story that I've put the most time and effort into; it charts a course from the near future all the way to the distant stars. --- First published: January 19th, 2025 Source: https://www.lesswrong.com/posts/Rz4ijbeKgPAaedg3n/the-gentle-romance --- Narrated by TYPE III AUDIO.

en-gb

5:51

“What’s the Right Way to think about Information...

LessWrong (30+ Karma) ·

2025/01/19

Tl;dr, Neural networks are deterministic and sometimes even reversible, which causes Shannon information measures to degenerate. But information theory seems useful. How can we square this (if it's possible at all)? The attempts so far in the literature are unsatisfying.Here is a conceptual question: what is the Right Way to think about information theoretic quantities in neural network contexts?Example: I've been recently thinking about information bottleneck methods: given some data distribution

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

The Automated Daily

TrendTeller

All On AI

All On AI

Elon Musk Podcast

Stage Zero

Reflections

Livewire Labs

科技早8点

之远不知远