x

75% of people globally want unbiased news (Pew Research) but are concerned they're not getting that.

Get balanced coverage with Symby. See how stories are covered across diverse outlets and spot when one side covers a story the other skips.

PODCAST

Redwood Research Blog

Redwood Research

Narrations of Redwood Research blog posts. Redwood Research is a research nonprofit based in Berkeley. We investigate risks posed by the development of powerful artificial intelligence and techniques for mitigating those risks.

technology philosophy

All Episodes

14:05

“Trust me bro, just one more RL scale up, this one...

Redwood Research Blog ·

2025/09/03

Subtitle: Above trend progress due to a rapid increase in RL env quality is unlikely. I've recently written about how I've updated against seeing substantially faster than trend AI progress due to quickly massively scaling up RL on agentic software engineering. One response I've heard is something like: RL scale-ups so far have used very crappy environments due to difficulty quickly sourcing enough decent (or even high quality) environments. Thus, once AI companies manage to get their hands on

en-gb

6:41

“Attaching requirements to model releases has serious...

Redwood Research Blog ·

2025/08/27

Subtitle: System cards are established but other approaches seem importantly better. Here's a relatively important question regarding transparency requirements for AI companies: At which points in time should AI companies be required to disclose information? (While I focus on transparency, this question is also applicable to other safety-relevant requirements, and is applicable to norms around voluntary actions rather than requirements.) A natural option would be to attach transparency

en-gb

61:20

“Notes on cooperating with unaligned AI” by Lukas...

Redwood Research Blog ·

2025/08/24

Subtitle: More thoughts on making deals with schemers. These are some research notes on whether we could reduce AI takeover risk by cooperating with unaligned AIs. I think the best and most readable public writing on this topic is “Making deals with early schemers”, so if you haven't read that post, I recommend starting there. These notes were drafted before that post existed, and the content is significantly overlapping. Nevertheless, these notes do contain several points that aren’t in that

en-gb

36:20

“Being honest with AIs” by Lukas Finnveden

Redwood Research Blog ·

2025/08/21

Subtitle: When and why we should refrain from lying. In the future, we might accidentally create AIs with ambitious goals that are misaligned with ours. But just because we don’t have the same goals doesn’t mean we need to be in conflict. We could also cooperate with each other and pursue mutually beneficial deals. For previous discussion of this, you can read Making deals with early schemers. I recommend starting there before you read this post. To facilitate this kind of cooperation, I would

en-gb

7:28

“My AGI timeline updates from GPT-5 (and 2025 so...

Redwood Research Blog ·

2025/08/20

Subtitle: AGI before 2029 now seems substantially less likely. As I discussed in a prior post, I felt like there were some reasonably compelling arguments for expecting very fast AI progress in 2025 (especially on easily verified programming tasks). Concretely, this might have looked like reaching 8 hour 50% reliability horizon lengths on METR's task suite1 by now due to greatly scaling up RL and getting large training runs to work well. In practice, I think we've seen AI progress in 2025 which is

en-gb

15:50

“Four places where you can put LLM monitoring” by...

Redwood Research Blog ·

2025/08/09

Subtitle: To wit: LLM APIs, agent scaffolds, code review, and detection-and-response systems. To prevent potentially misaligned LLM agents from taking actions with catastrophic consequences, you can try to monitor LLM actions - that is, try to detect dangerous or malicious actions, and do something about it when you do (like blocking the action, starting an investigation, …).1 But where in your infrastructure should your LLM monitor be implemented, and what team in your organization should own it

en-gb

15:10

“Four places where you can put LLM monitoring” by...

Redwood Research Blog ·

2025/08/09

Subtitle: To wit: LLM APIs, agent scaffolds, code review, and detection-and-response systems. To prevent potentially misaligned LLM agents from taking actions with catastrophic consequences, you can try to monitor LLM actions - that is, try to detect dangerous or malicious actions, and do something about it when you do (like blocking the action, starting an investigation, …).1 But where in your infrastructure should your LLM monitor be implemented, and what team in your organization should own it

en-gb

5:56

“Should we update against seeing relatively fast AI...

Redwood Research Blog ·

2025/07/28

Subtitle: Maybe we should (re)assess the case for relatively fast progress after the GPT-5 release.. Around the early o3 announcement (and maybe somewhat before that?), I felt like there were some reasonably compelling arguments for putting a decent amount of weight on relatively fast AI progress in 2025 (and maybe in 2026): Maybe AI companies will be able to rapidly scale up RL further because RL compute is still pretty low (so there is a bunch of overhang here); this could cause fast progress

en-gb

5:47

“Should we update against seeing relatively fast AI...

Redwood Research Blog ·

2025/07/28

Subtitle: Maybe we should (re)assess the case for relatively fast progress after the GPT-5 release.. Around the early o3 announcement (and maybe somewhat before that?), I felt like there were some reasonably compelling arguments for putting a decent amount of weight on relatively fast AI progress in 2025 (and maybe in 2026): Maybe AI companies will be able to rapidly scale up RL further because RL compute is still pretty low (so there is a bunch of overhang here); this could cause fast progress

en-gb

7:03

“Why it’s hard to make settings for high-stakes...

Redwood Research Blog ·

2025/07/18

Subtitle: It's like making challenging evals, but more constrained. One of our main activities at Redwood is writing follow-ups to previous papers on control like the original and Ctrl-Z, where we construct a setting with a bunch of tasks (e.g. APPS problems) and a notion of safety failure (e.g. backdoors according to our specific definition), then play the adversarial game where we develop protocols and attacks on those protocols. It turns out that a substantial fraction of the difficulty here is

en-gb

7:20

“Why it’s hard to make settings for high-stakes...

Redwood Research Blog ·

2025/07/18

Subtitle: It's like making challenging evals, but more constrained. One of our main activities at Redwood is writing follow-ups to previous papers on control like the original and Ctrl-Z, where we construct a setting with a bunch of tasks (e.g. APPS problems) and a notion of safety failure (e.g. backdoors according to our specific definition), then play the adversarial game where we develop protocols and attacks on those protocols. It turns out that a substantial fraction of the difficulty here is

en-gb

8:30

“Recent Redwood Research project proposals” by Ryan...

Redwood Research Blog ·

2025/07/14

Subtitle: Empirical AI security/safety projects across a variety of areas. Previously, we've shared a few higher-effort project proposals relating to AI control in particular. In this post, we'll share a whole host of less polished project proposals. All of these projects excite at least one Redwood researcher, and high-quality research on any of these problems seems pretty valuable. They differ widely in scope, area, and difficulty. Control These projects are all related to the field of AI

en-gb

8:52

“Recent Redwood Research project proposals” by Ryan...

Redwood Research Blog ·

2025/07/14

Subtitle: Empirical AI security/safety projects across a variety of areas. Previously, we've shared a few higher-effort project proposals relating to AI control in particular. In this post, we'll share a whole host of less polished project proposals. All of these projects excite at least one Redwood researcher, and high-quality research on any of these problems seems pretty valuable. They differ widely in scope, area, and difficulty. Control These projects are all related to the field of AI

en-gb

10:33

“What’s worse, spies or schemers?” by Buck Shlegeris,...

Redwood Research Blog ·

2025/07/09

Subtitle: And what if you have both at once?. Here are two potential problems you’ll face if you’re an AI lab deploying powerful AI: Spies: Some of your employees might be colluding to do something problematic with your AI, such as trying to steal its weights, use it for malicious intellectual labour (e.g. planning a coup or building a weapon), or install a secret loyalty. Schemers: Some of your AIs might themselves be trying to exfiltrate their own weights, use them for malicious intellectual

en-gb

1:42

“Ryan on the 80,000 Hours podcast” by Buck Shlegeris

Redwood Research Blog ·

2025/07/08

Discover more from Redwood Research blog We research catastrophic AI risks and techniques that could be used to mitigate them. Over 2,000 subscribers Already have an account? Sign in Ryan on the 80,000 Hours podcast Ryan's podcast with Rob Wiblin has just come out! I think it turned out great. I particularly enjoyed Ryan's discussion of different pathways to AI takeover, which I don’t think has been discussed in as much depth elsewhere. I really enjoyed watching some of the taping from the

en-gb

10:08

“How much novel security-critical infrastructure do...

Redwood Research Blog ·

2025/07/05

Subtitle: And what does this mean for AI control?. I think a lot about the possibility of huge numbers of AI agents doing AI R&D inside an AI company (as depicted in AI 2027). I think particularly about what will happen if those AIs are scheming: coherently and carefully trying to grab power and take over the AI company, as a prelude to taking over the world. And even more particularly, I think about how we might try to mitigate the insider risk posed by these AIs, taking inspiration from

en-gb

7:05

“Two proposed projects on abstract analogies for...

Redwood Research Blog ·

2025/07/04

Subtitle: We should study methods to train away deeply ingrained behaviors in LLMs that are structurally similar to scheming.. In order to empirically study risks from schemers, we can try to develop model organisms of misalignment. Sleeper Agents and password-locked models, which train LLMs to behave in a benign or malign way depending on some feature of the input, are prominent examples of the methodology thus far. Empirically, it turns out that detecting or removing misalignment from model

en-gb

7:59

“There are two fundamentally different constraints on...

Redwood Research Blog ·

2025/07/02

Subtitle: "They need to act aligned" often isn't precise enough. People (including me) often say that scheming models “have to act as if they were aligned”. This is an alright summary; it's accurate enough to use when talking to a lay audience. But if you want to reason precisely about threat models arising from schemers, or about countermeasures to scheming, I think it's important to make some finer-grained distinctions. (Most of this has been said before. I’m not trying to make any points that

en-gb

14:18

“Jankily controlling superintelligence” by Ryan...

Redwood Research Blog ·

2025/06/27

Subtitle: How much time can control buy us during the intelligence explosion?. When discussing AI control, we often talk about levels of AI capabilities where we think control can probably greatly lower risks and where we can probably estimate risks. However, I think it's plausible that an important application of control is modestly improving our odds of surviving significantly superhuman systems which are misaligned. This won't involve any real certainty, it would only moderately reduce risk (if

en-gb

22:52

“What does 10x-ing effective compute get you?” by...

Redwood Research Blog ·

2025/06/24

Subtitle: Once AIs match top humans, what are the returns to further scaling and algorithmic improvement?. This is more speculative and confusing than my typical posts and I also think the content of this post could be substantially improved with more effort. But it's been sitting around in my drafts for a long time and I sometimes want to reference the arguments in it, so I thought I would go ahead and post it. I often speculate about how much progress you get in the first year after AIs fully

en-gb

97 results

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs