PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

6:29

“The Sweet Lesson: AI Safety Should Scale With...

LessWrong (30+ Karma) ·

2025/05/05

A corollary of Sutton's Bitter Lesson is that solutions to AI safety should scale with compute. Let me list a few examples of research directions that aim at this kind of solution: Deliberative Alignment: Combine chain-of-thought with Constitutional AI, so that safety improves with inference-time compute (see Guan et al. 2025, Figure 13). AI Control: Design control protocols that pit a red team against a blue team so that running the game for longer results in more reliable estimates of the

en-gb

17:15

“Interim Research Report: Mechanisms of Awareness” by...

LessWrong (30+ Karma) ·

2025/05/05

Summary Reproducing a result from recent work, we study a Gemma 3 12B instance trained to take risky or safe options; the model can then report its own risk tolerance. We find that: Applying LoRA to a single MLP is enough to reproduce the behavior The single LoRA layer learns a single additive steering vector. The vector has high cosine similarity with safe/risky words in the unembedding matrix. We can train just the steering vector, no LoRA needed. The steering vector has ~0.5 cosine sim with

en-gb

5:17

“Overview: AI Safety Outreach Grassroots Orgs” by...

LessWrong (30+ Karma) ·

2025/05/05

We’ve been looking for joinable endeavors in AI safety outreach over the past weeks and would like to share our findings with you. Let us know if we missed any and we’ll add them to the list. For comprehensive directories of AI safety communities spanning general interest, technical focus, and local chapters, check out https://www.aisafety.com/communities and https://www.aisafety.com/map. If you're uncertain where to start, https://aisafety.quest/ offers personalized guidance. ControlAI ControlAI

en-gb

4:51

“Notes on the Long Tasks METR paper, from a HCAST...

LessWrong (30+ Karma) ·

2025/05/05

I contributed one (1) task to HCAST, which was used in METR's Long Tasks paper. This gave me some thoughts I feel moved to share. Regarding Baselines and Estimates METR's tasks have two sources for how long they take humans: most of those used in the paper were Baselined using playtesters under persistent scrutiny, and some were Estimated by METR. I don’t quite trust the Baselines. Baseliners were allowed/incentivized to drop tasks they weren’t making progress with, and were – mostly, effectively,

en-gb

4:09

“Why I am not a successionist” by Nina Panickssery

LessWrong (30+ Karma) ·

2025/05/04

Utilitarianism implies that if we build an AI that successfully maximizes utility/value, we should be ok with it replacing us. Sensible people add caveats related to how hard it’ll be to determine the correct definition of value or check whether the AI is truly optimizing it. As someone who often passionately rants against the AI successionist line of thinking, the most common objection I hear is "why is your definition of value so arbitrary as to stipulate that biological meat-humans are

en-gb

24:36

“Updates from Comments on ‘AI 2027 is a Bet Against...

LessWrong (30+ Karma) ·

2025/05/04

AI 2027 is a Bet Against Amdahl's Law was my attempt to summarize and analyze "the key load-bearing arguments AI 2027 presents for short timelines". There were a lot of great comments – every time I post on LW is a learning experience. In this post, I'm going to summarize the comments and present some resulting updates to my previous analysis. I'm also using this post to address some comments that I didn't respond to in the original post, because the comment tree was becoming quite sprawling

en-gb

7:46

“‘Superhuman’ Isn’t Well Specified” by JustisMills

LessWrong (30+ Karma) ·

2025/05/04

Strength In 1997, with Deep Blue's defeat of Kasparov, computers surpassed human beings at chess. Other games have fallen in more recent years: Go, Starcraft, and League of Legends among them. AI is superhuman at these pursuits, and unassisted human beings will never catch up. The situation looks like this:[1]At chess, AI is much better than the very best humans The average serious chess player is pretty good (1500), the very best chess player is extremely good (2837), and the best AIs are way, way

en-gb

13:16

“Interpretability Will Not Reliably Find Deceptive...

LessWrong (30+ Karma) ·

2025/05/04

(Disclaimer: Post written in a personal capacity. These are personal hot takes and do not in any way represent my employer's views.)TL;DR: I do not think we will produce high reliability methods to evaluate or monitor the safety of superintelligent systems via current research paradigms, with interpretability or otherwise. Interpretability seems a valuable tool here and remains worth investing in, as it will hopefully increase the reliability we can achieve. However, interpretability should be

en-gb

1:16

“PSA: Before May 21 is a good time to sign up for...

LessWrong (30+ Karma) ·

2025/05/04

Cryonics Institute and Suspended Animation now have an arrangement where Suspended Animation will conduct a field cryopreservation before shipping the body to Cryonics Institute, thus decreasing tissue damage occuring in transit. They are raising their prices accordingly, but offering a discount from the new price for people who sign up by May 21 (and arrange funding within another 3 months). See https://cryonics.org/members/standby/suspended-animation-inc-standby-transport-services-option/ for

en-gb

9:00

“The Ukraine War and the Kill Market” by Martin Sustrik

LessWrong (30+ Karma) ·

2025/05/04

Politico writes: The [Ukrainian] program […] rewards soldiers with points if they upload videos proving their drones have hit Russian targets. It will soon be integrated with a new online marketplace called Brave 1 Market, which will allow troops to convert those points into new equipment for their units. [...] The program assigns points for each type of kill: 20 points for damaging and 40 for destroying a tank; up to 50 points for destroying a mobile rocket system, depending on the caliber; and

en-gb

15:36

“Navigating burnout” by gw

LessWrong (30+ Karma) ·

2025/05/04

Burnout. Burn out? Whatever, it sucks. Burnout is a pretty confusing thing made harder by our naive reactions being things like “just try harder” or “grit your teeth and push through”, which usually happen to be exactly the wrong things to do. Burnout also isn’t really just one thing, it's more like a collection of distinct problems that are clustered by similar symptoms. Something something intro, research context, this is what I’ve learned / synthesized blah blah blah. Read on! Models of burnout

en-gb

11:39

“Obstacles in ARC’s agenda: Low Probability...

LessWrong (30+ Karma) ·

2025/05/03

As an employee of the European AI Office, it's important for me to emphasize this point: The views and opinions of the author expressed herein are personal and do not necessarily reflect those of the European Commission or other EU institutions. Also, to stave off a common confusion: I worked at ARC Theory, which is now simply called ARC, on Paul Christiano's theoretical alignment agenda. The more famous ARC Evals was a different group working on evaluations, their work was completely separate from

en-gb

17:56

“Obstacles in ARC’s agenda: Mechanistic Anomaly...

LessWrong (30+ Karma) ·

2025/05/03

en-gb

16:12

“What’s going on with AI progress and trends? (As of...

LessWrong (30+ Karma) ·

2025/05/02

AI progress is driven by improved algorithms and additional compute for training runs. Understanding what is going on with these trends and how they are currently driving progress is helpful for understanding the future of AI. In this post, I'll share a wide range of general takes on this topic as well as open questions. Be warned that I'm quite uncertain about a bunch of this! This post will assume some familiarity with what is driving AI progress, specifically it will assume you understand the

en-gb

15:54

“RA x ControlAI video: What if AI just keeps getting...

LessWrong (30+ Karma) ·

2025/05/02

The video is about extrapolating the future of AI progress, following a timeline that starts from today's chatbots to future AI that's vastly smarter than all of humanity combined–with God-like capabilities. We argue that such AIs will pose a significant extinction risk to humanity. This video came out of a partnership between Rational Animations and ControlAI. The script was written by Arthur Frost (one of Rational Animations’ writers) with Andrea Miotti as an adaptation of key points from The

en-gb

17:09

“AI Governance to Avoid Extinction: The Strategic...

LessWrong (30+ Karma) ·

2025/05/02

We’re excited to release a new AI governance research agenda from the MIRI Technical Governance Team. With this research agenda, we have two main aims: to describe the strategic landscape of AI development and to catalog important governance research questions. We base the agenda around four high-level scenarios for the geopolitical response to advanced AI development. Our favored scenario involves building the technical, legal, and institutional infrastructure required to internationally restrict

en-gb

3:33

[Linkpost] “Don’t accuse your interlocutor of being...

LessWrong (30+ Karma) ·

2025/05/02

This is a link post. I argue that you shouldn't accuse your interlocutor of being insufficiently truth-seeking. This doesn't mean you can't internally model their level of truth-seeking and use that for your own decision-making. It just means you shouldn't come out and say "I think you are being insufficiently truth-seeking". What you should say instead Before I explain my reasoning, I'll start with what you should say instead: "You're wrong" People are wrong a lot. If you think they are wrong just

en-gb

6:40

“Anthropomorphizing AI might be good, actually” by...

LessWrong (30+ Karma) ·

2025/05/02

It is often noted that anthropomorphizing AI can be dangerous. People likely have prosocial instincts that AI systems lack (see below). Assuming AGI will be aligned because humans with similar behavior are usually mostly harmless is probably wrong and quite dangerous. I want to discuss a flip side of using humans as an intuition pump for thinking about AI. Humans have many of the properties we are worried about for truly dangerous AGI: Situational awareness Strong goal-directedness

en-gb

11:26

“Superhuman Coders in AI 2027 - Not So Fast” by...

LessWrong (30+ Karma) ·

2025/05/01

Thank you @elifland for reviewing this post. He and AI Futures are planning to publish updates to the AI 2027 Timeline Forecast soon. AI 2027 (also launched in this LW post) forecasts an R&D-based AI takeoff starting with the development of Superhuman Coders[1] within a frontier lab. FutureSearch co-authored the AI 2027 Timeline Forecast. We thought the other authors’ forecasts were excellently done, and as the piece says: All model-based forecasts have 2027 as one of the most likely years that

en-gb

11:34

“Slowdown After 2028: Compute, RLVR Uncertainty, MoE...

LessWrong (30+ Karma) ·

2025/05/01

It'll take until ~2050 to repeat the level of scaling that pretraining compute is experiencing this decade, as increasing funding can't sustain the current pace beyond ~2029 if AI doesn't deliver a transformative commercial success by then. Natural text data will also run out around that time, and there are signs that current methods of reasoning training might be mostly eliciting capabilities from the base model.If scaling of reasoning training doesn't bear out actual creation of new

en-gb

4,489 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Talking Tech w/ Wayne

Wayne Hunter

The Everything Feed - All Packet Pushers Pods

Packet Pushers

The Good Tech Companies

HackerNoon

Engadget News + Next

Engadget

Cybersecurity Tech Brief By HackerNoon

HackerNoon