x

75% of people globally want unbiased news (Pew Research) but are concerned they're not getting that.

Get balanced coverage with Symby. See how stories are covered across diverse outlets and spot when one side covers a story the other skips.

PODCAST

Redwood Research Blog

Redwood Research

Narrations of Redwood Research blog posts. Redwood Research is a research nonprofit based in Berkeley. We investigate risks posed by the development of powerful artificial intelligence and techniques for mitigating those risks.

technology philosophy

All Episodes

1:19

“Buck on the 80,000 Hours podcast” by Buck Shlegeris

Redwood Research Blog ·

2025/04/05

Buck on the 80,000 Hours podcast My podcast with Rob Wiblin from 80,000 Hours just came out. I’m really happy with how it turned out. I talked about a bunch of stuff on the podcast that I don’t think we’ve written up before. Transcript + links + summary here. You can also get it as a podcast: Spotify: Apple: Subscribe to Redwood Research blog Launched a year ago We research catastrophic AI risks and techniques that could be used to mitigate them. Discussion about this post ---Outline:(00:09)

en-gb

1:05

“Buck on the 80,000 Hours podcast” by Buck Shlegeris

Redwood Research Blog ·

2025/04/05

Buck on the 80,000 Hours podcast My podcast with Rob Wiblin from 80,000 Hours just came out. I’m really happy with how it turned out. I talked about a bunch of stuff on the podcast that I don’t think we’ve written up before. Transcript + links + summary here. You can also get it as a podcast: Spotify: Apple: Subscribe to Redwood Research blog Launched a year ago We research catastrophic AI risks and techniques that could be used to mitigate them. Discussion about this post ---Outline:(00:34)

en-gb

15:26

“Notes on countermeasures for exploration hacking...

Redwood Research Blog ·

2025/04/04

Subtitle: How can we prevent AIs from intentionally underperforming on our metrics?. If we naively apply RL to a scheming AI, the AI may be able to systematically get low reward/performance while simultaneously not having this behavior trained out because it intentionally never explores into better behavior. As in, it intentionally puts very low probability on (some) actions which would perform very well to prevent these actions from being sampled and then reinforced. We'll refer to this as

en-gb

32:52

“Notes on handling non-concentrated failures with AI...

Redwood Research Blog ·

2025/04/03

Subtitle: What are the methods and issues when failures occur diffusely over many actions?. In this post, I'll try to explain my current understanding of the high level methods for handling non-concentrated failures with control. I'll discuss the regimes produced by different methods and the failure modes of these different regimes. Non-concentrated failures are issues that arise from the AI doing a large number of problematic actions over a long period. (See here and here for more discussion of

en-gb

20:43

“Prioritizing threats for AI control” by Ryan Greenblatt

Redwood Research Blog ·

2025/03/19

Subtitle: What are the main threats and how should we prioritize them?. We often talk about ensuring control, which in the context of this doc refers to preventing AIs from being able to cause existential problems, even if the AIs attempt to subvert our countermeasures. To better contextualize control, I think it's useful to discuss the main threats. I'll focus my discussion on threats induced by misalignment which could plausibly increase existential risk. While control is often associated with

en-gb

72:10

“How might we safely pass the buck to AI?” by Josh...

Redwood Research Blog ·

2025/02/19

Subtitle: Developing AI employees that are safer than human ones. My goal as an AI safety researcher is to put myself out of a job. I don’t worry too much about how planet sized brains will shape galaxies in 100 years. That's something for AI systems to figure out. Instead, I worry about safely replacing human researchers with AI agents, at which point human researchers are “obsolete.” The situation is not necessarily fine after human obsolescence; however, the bulk of risks that are addressable

en-gb

7:39

“Takeaways from sketching a control safety case” by...

Redwood Research Blog ·

2025/01/30

Subtitle: Insights from a long technical paper compressed into a fun little commentary. Buck Shlegeris and I recently published a paper with UK AISI that sketches a safety case for “AI control” – measures that improve safety despite intentional subversion from AI systems. I would summarize this work as “turning crayon drawings of safety cases into blueprints.” It's a long and technical paper! So I’ve compiled my main takeaways in this blogpost. The main figure from the paper summarizes how to run

en-gb

7:36

“Takeaways from sketching a control safety case” by...

Redwood Research Blog ·

2025/01/30

Subtitle: Insights from a long technical paper compressed into a fun little commentary. Buck Shlegeris and I recently published a paper with UK AISI that sketches a safety case for “AI control” – measures that improve safety despite intentional subversion from AI systems. I would summarize this work as “turning crayon drawings of safety cases into blueprints.” It's a long and technical paper! So I’ve compiled my main takeaways in this blogpost. The main figure from the paper summarizes how to run

en-gb

45:52

“Planning for Extreme AI Risks” by Josh Clymer

Redwood Research Blog ·

2025/01/29

Subtitle: Are we ready for this?. This post should not be taken as a polished recommendation to AI companies and instead should be treated as an informal summary of a worldview. The content is inspired by conversations with a large number of people, so I cannot take credit for any of these ideas. For a summary of this post, see the thread on X. Many people write opinions about how to handle advanced AI, which can be considered “plans.” There's the “stop AI now plan.” On the other side of the

en-gb

7:27

“Ten people on the inside” by Buck Shlegeris

Redwood Research Blog ·

2025/01/28

Subtitle: A scary scenario that's worth planning for. (Many of these ideas developed in conversation with Ryan Greenblatt) In a shortform, I described some different levels of resources and buy-in for misalignment risk mitigations that might be present in AI labs: *The “safety case” regime.* Sometimes people talk about wanting to have approaches to safety such that if all AI developers followed these approaches, the overall level of risk posed by AI would be minimal. (These approaches are going to

en-gb

7:24

“Ten people on the inside” by Buck Shlegeris

Redwood Research Blog ·

2025/01/28

Subtitle: A scary scenario that's worth planning for. (Many of these ideas developed in conversation with Ryan Greenblatt) In a shortform, I described some different levels of resources and buy-in for misalignment risk mitigations that might be present in AI labs: *The “safety case” regime.* Sometimes people talk about wanting to have approaches to safety such that if all AI developers followed these approaches, the overall level of risk posed by AI would be minimal. (These approaches are going to

en-gb

42:52

“When does capability elicitation bound risk?” by...

Redwood Research Blog ·

2025/01/22

For a summary of this post see the thread on X. The assumptions behind and limitations of capability elicitation have been discussed in multiple places (e.g. here, here, here, etc); however, none of these match my current picture. For example, Evan Hubinger's “When can we trust model evaluations?” cites “gradient hacking” (strategically interfering with the learning process) as the main reason supervised fine-tuning (SFT) might fail, but I’m worried about more boring failures. For example, models

en-gb

43:38

“When does capability elicitation bound risk?” by...

Redwood Research Blog ·

2025/01/22

For a summary of this post see the thread on X. The assumptions behind and limitations of capability elicitation have been discussed in multiple places (e.g. here, here, here, etc); however, none of these match my current picture. For example, Evan Hubinger's “When can we trust model evaluations?” cites “gradient hacking” (strategically interfering with the learning process) as the main reason supervised fine-tuning (SFT) might fail, but I’m worried about more boring failures. For example, models

en-gb

82:20

“How will we update about scheming?” by Ryan Greenblatt

Redwood Research Blog ·

2025/01/19

Subtitle: A quantitative description of how I expect to change my mind.. [Cross-posted from LessWrong] I mostly work on risks from scheming (that is, misaligned, power-seeking AIs that plot against their creators such as by faking alignment). Recently, I (and co-authors) released "Alignment Faking in Large Language Models", which provides empirical evidence for some components of the scheming threat model. One question that's really important is how likely scheming is. But it's also really

en-gb

24:59

“Thoughts on the conservative assumptions in AI...

Redwood Research Blog ·

2025/01/17

Subtitle: Why are we so friendly to the red team?. Work that I’ve done on techniques for mitigating risk from misaligned AI often makes a number of conservative assumptions about the capabilities of the AIs we’re trying to control. (E.g. the original AI control paper, Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats, How to prevent collusion when using untrusted models to monitor each other.) For example: The AIs are consistently trying to subvert safety measures. They’re very

en-gb

25:00

“Thoughts on the conservative assumptions in AI...

Redwood Research Blog ·

2025/01/17

Subtitle: Why are we so friendly to the red team?. Work that I’ve done on techniques for mitigating risk from misaligned AI often makes a number of conservative assumptions about the capabilities of the AIs we’re trying to control. (E.g. the original AI control paper, Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats, How to prevent collusion when using untrusted models to monitor each other.) For example: The AIs are consistently trying to subvert safety measures. They’re very

en-gb

31:59

“Extending control evaluations to non-scheming...

Redwood Research Blog ·

2025/01/13

Buck Shlegeris and Ryan Greenblatt originally motivated control evaluations as a way to mitigate risks from ‘scheming’ AI models: models that consistently pursue power-seeking goals in a covert way; however, many adversarial model psychologies are not well described by the standard notion of scheming. For example, models might display scheming-like behavior inconsistently or acquire harmful goals through data poisoning, jailbreaks, etc.1 This post suggests a generalization of control evaluations

en-gb

30:30

“Extending control evaluations to non-scheming...

Redwood Research Blog ·

2025/01/13

Buck Shlegeris and Ryan Greenblatt originally motivated control evaluations as a way to mitigate risks from ‘scheming’ AI models: models that consistently pursue power-seeking goals in a covert way; however, many adversarial model psychologies are not well described by the standard notion of scheming. For example, models might display scheming-like behavior inconsistently or acquire harmful goals through data poisoning, jailbreaks, etc.1 This post suggests a generalization of control evaluations

en-gb

22:50

“Measuring whether AIs can statelessly strategize to...

Redwood Research Blog ·

2024/12/20

Subtitle: The complement to control evaluations. One way to show that risk from deploying an AI system is small is by showing that the model is not capable of subverting security measures enough to cause substantial harm. In AI control research so far, control evaluations have measured whether a red-team-created attack policy can defeat control measures, but they haven't measured how effectively models can generate such attack policies themselves. In particular, lack of a persistent memory poses a

en-gb

23:13

“Measuring whether AIs can statelessly strategize to...

Redwood Research Blog ·

2024/12/20

Subtitle: The complement to control evaluations. One way to show that risk from deploying an AI system is small is by showing that the model is not capable of subverting security measures enough to cause substantial harm. In AI control research so far, control evaluations have measured whether a red-team-created attack policy can defeat control measures, but they haven't measured how effectively models can generate such attack policies themselves. In particular, lack of a persistent memory poses a

en-gb

97 results

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs