x

75% of people globally want unbiased news (Pew Research) but are concerned they're not getting that.

Get balanced coverage with Symby. See how stories are covered across diverse outlets and spot when one side covers a story the other skips.

PODCAST

Redwood Research Blog

Redwood Research

Narrations of Redwood Research blog posts. Redwood Research is a research nonprofit based in Berkeley. We investigate risks posed by the development of powerful artificial intelligence and techniques for mitigating those risks.

technology philosophy

All Episodes

31:37

“Will AI systems drift into misalignment?” by Josh...

Redwood Research Blog ·

2025/11/15

Subtitle: A reason alignment could be hard. Written with Alek Westover and Anshul Khandelwal This post explores what I’ll call the Alignment Drift Hypothesis: An AI system that is initially aligned will generally drift into misalignment after a sufficient number of successive modifications, even if these modifications select for alignment with fixed and unreliable metrics.The intuition behind the Alignment Drift Hypothesis. The ball bouncing around is an AI system being modified (e.g. during

en-gb

39:23

“What’s up with Anthropic predicting AGI by early...

Redwood Research Blog ·

2025/11/03

Subtitle: I operationalize Anthropic's prediction of "powerful AI" and explain why I'm skeptical. As far as I’m aware, Anthropic is the only AI company with official AGI timelines1: they expect AGI by early 2027. In their recommendations (from March 2025) to the OSTP for the AI action plan they say: As our CEO Dario Amodei writes in ‘Machines of Loving Grace’, we expect powerful AI systems will emerge in late 2026 or early 2027. Powerful AI systems will have the following properties:

en-gb

35:41

“Sonnet 4.5’s eval gaming seriously undermines...

Redwood Research Blog ·

2025/10/30

Subtitle: and this seems caused by training on alignment evals. Sonnet 4.5's eval gaming seriously undermines alignment evals and seems caused by training on alignment evals According to the Sonnet 4.5 system card, Sonnet 4.5 is much more likely than Sonnet 4 to mention in its chain-of-thought that it thinks it is being evaluated; this seems to meaningfully cause it to appear to behave better in alignment evaluations. So, Sonnet 4.5's behavioral improvements in these evaluations may partly be

en-gb

18:30

“Should AI Developers Remove Discussion of AI...

Redwood Research Blog ·

2025/10/23

There is some concern that training AI systems on content predicting AI misalignment will hyperstition AI systems into misalignment. This has been discussed previously by a lot of people: Anna Salamon, Alex Turner, the AI Futures Project, Miles Kodama, Gwern, Cleo Nardo, Richard Ngo, Rational Animations, Mark Keavney and others. In this post, I analyze whether AI developers should filter out discussion of AI misalignment from training data. I discuss several details that I don’t think have been

en-gb

13:05

“Is 90% of code at Anthropic being written by AIs?”...

Redwood Research Blog ·

2025/10/22

Subtitle: I'm skeptical that Dario's prediction of AIs writing 90% of code in 3-6 months has come true. In March 2025, Dario Amodei (CEO of Anthropic) said that he expects AI to be writing 90% of the code in 3 to 6 months and that AI might be writing essentially all of the code in 12 months.1 Did this prediction end up being true? We’ll charitably interpret the prediction as being about Anthropic itself. Recently, various articles have claimed that Dario said that AIs are writing 90% of the code

en-gb

20:15

“Reducing risk from scheming by studying trained-in...

Redwood Research Blog ·

2025/10/16

Subtitle: Can we study scheming by studying AIs trained to act like schemers?. In a previous post, I discussed mitigating risks from scheming by studying examples of actual scheming AIs.1 In this post, I’ll discuss an alternative approach: directly training (or instructing) an AI to behave how we think a naturally scheming AI might behave (at least in some ways). Then, we can study the resulting models. For instance, we could verify that a detection method discovers that the model is scheming or

en-gb

14:37

“Iterated Development and Study of Schemers (IDSS)”...

Redwood Research Blog ·

2025/10/10

Subtitle: A strategy for handling scheming. In a previous post, we discussed prospects for studying scheming using natural examples. In this post, we’ll describe a more detailed proposal for iteratively constructing scheming models, techniques for detecting scheming, and techniques for preventing scheming. We’ll call this strategy Iterated Development and Study of Schemers (IDSS). We’ll be using concepts from that prior post, like the idea of trying to make schemers which are easier to catch. Two

en-gb

12:01

“The Thinking Machines Tinker API is good news for AI...

Redwood Research Blog ·

2025/10/09

Subtitle: It's a promising design for reducing model access inside AI companies.. Last week, Thinking Machines announced Tinker. It's an API for running fine-tuning and inference on open-source LLMs that works in a unique way. I think it has some immediate practical implications for AI safety research: I suspect that it will make RL experiments substantially easier, and increase the number of safety papers that involve RL on big models. But it's also interesting to me for another reason: the

en-gb

12:17

“Plans A, B, C, and D for misalignment risk” by Ryan...

Redwood Research Blog ·

2025/10/08

Subtitle: Different plans for different levels of political will. I sometimes think about plans for how to handle misalignment risk. Different levels of political will for handling misalignment risk result in different plans being the best option. I often divide this into Plans A, B, C, and D (from most to least political will required). See also Buck's quick take about different risk level regimes. In this post, I’ll explain the Plan A/B/C/D abstraction as well as discuss the probabilities and

en-gb

15:54

“Notes on fatalities from AI takeover” by Ryan...

Redwood Research Blog ·

2025/09/23

Subtitle: Large fractions of people will die, but literal human extinction seems unlikely. Suppose misaligned AIs take over. What fraction of people will die? I'll discuss my thoughts on this question and my basic framework for thinking about it. These are some pretty low-effort notes, the topic is very speculative, and I don't get into all the specifics, so be warned. I don't think moderate disagreements here are very action-guiding or cruxy on typical worldviews: it probably shouldn't alter your

en-gb

11:51

“Focus transparency on risk reports, not safety...

Redwood Research Blog ·

2025/09/22

Subtitle: Transparency about just safety cases would have bad epistemic effects. There are many different things that AI companies could be transparent about. One relevant axis is transparency about the current understanding of risks and the current mitigations of these risks. I think transparency about this should take the form of a publicly disclosed risk report rather than the company making a safety case. To be clear, there are other types of transparency focused on different aspects of the

en-gb

101:28

“Prospects for studying actual schemers” by Ryan...

Redwood Research Blog ·

2025/09/19

Subtitle: Studying actual schemers seems promising but tricky. One natural way to research scheming is to study AIs that are analogous to schemers. Research studying current AIs that are intended to be analogous to schemers won't yield compelling empirical evidence or allow us to productively study approaches for eliminating scheming, as these AIs aren't very analogous in practice. This difficulty is in part what motivates research that adopts a conservative approach to AI motivations. (In this

en-gb

44:01

“What training data should developers filter to...

Redwood Research Blog ·

2025/09/17

Subtitle: An initial narrow proposal. One potentially powerful way to change the properties of AI models is to change their training data. For example, Anthropic has explored filtering training data to mitigate bio misuse risk. What data, if any, should be filtered to reduce misalignment risk? In this post, I argue that the highest ROI data to filter is information about safety measures, and strategies for subverting them. Here's why this is valuable: If AI models don’t know what our security

en-gb

26:33

“AIs will greatly change engineering in AI companies...

Redwood Research Blog ·

2025/09/09

Subtitle: AIs that speed up engineering by 2x wouldn't accelerate AI progress that much. In response to my recent post arguing against above-trend progress from better RL environments, yet another argument for short(er) AGI timelines was raised to me: Sure, at the current rate of progress, it would take a while (e.g. 5 years) to reach AIs that can automate research engineering within AI companies while being cheap and fast (superhuman coder). But we'll get large increases in the rate of AI

en-gb

14:05

“Trust me bro, just one more RL scale up, this one...

Redwood Research Blog ·

2025/09/03

Subtitle: Above trend progress due to a rapid increase in RL env quality is unlikely. I've recently written about how I've updated against seeing substantially faster than trend AI progress due to quickly massively scaling up RL on agentic software engineering. One response I've heard is something like: RL scale-ups so far have used very crappy environments due to difficulty quickly sourcing enough decent (or even high quality) environments. Thus, once AI companies manage to get their hands on

en-gb

6:41

“Attaching requirements to model releases has serious...

Redwood Research Blog ·

2025/08/27

Subtitle: System cards are established but other approaches seem importantly better. Here's a relatively important question regarding transparency requirements for AI companies: At which points in time should AI companies be required to disclose information? (While I focus on transparency, this question is also applicable to other safety-relevant requirements, and is applicable to norms around voluntary actions rather than requirements.) A natural option would be to attach transparency

en-gb

61:20

“Notes on cooperating with unaligned AI” by Lukas...

Redwood Research Blog ·

2025/08/24

Subtitle: More thoughts on making deals with schemers. These are some research notes on whether we could reduce AI takeover risk by cooperating with unaligned AIs. I think the best and most readable public writing on this topic is “Making deals with early schemers”, so if you haven't read that post, I recommend starting there. These notes were drafted before that post existed, and the content is significantly overlapping. Nevertheless, these notes do contain several points that aren’t in that

en-gb

36:20

“Being honest with AIs” by Lukas Finnveden

Redwood Research Blog ·

2025/08/21

Subtitle: When and why we should refrain from lying. In the future, we might accidentally create AIs with ambitious goals that are misaligned with ours. But just because we don’t have the same goals doesn’t mean we need to be in conflict. We could also cooperate with each other and pursue mutually beneficial deals. For previous discussion of this, you can read Making deals with early schemers. I recommend starting there before you read this post. To facilitate this kind of cooperation, I would

en-gb

7:28

“My AGI timeline updates from GPT-5 (and 2025 so...

Redwood Research Blog ·

2025/08/20

Subtitle: AGI before 2029 now seems substantially less likely. As I discussed in a prior post, I felt like there were some reasonably compelling arguments for expecting very fast AI progress in 2025 (especially on easily verified programming tasks). Concretely, this might have looked like reaching 8 hour 50% reliability horizon lengths on METR's task suite1 by now due to greatly scaling up RL and getting large training runs to work well. In practice, I think we've seen AI progress in 2025 which is

en-gb

15:50

“Four places where you can put LLM monitoring” by...

Redwood Research Blog ·

2025/08/09

Subtitle: To wit: LLM APIs, agent scaffolds, code review, and detection-and-response systems. To prevent potentially misaligned LLM agents from taking actions with catastrophic consequences, you can try to monitor LLM actions - that is, try to detect dangerous or malicious actions, and do something about it when you do (like blocking the action, starting an investigation, …).1 But where in your infrastructure should your LLM monitor be implemented, and what team in your organization should own it

en-gb

91 results

Similar Podcasts

Lead With AI

Tamara Nall

Linux in the Ham Shack

Black Sparrow Media

Data Engineering Podcast

Tobias Macey

Eye On A.I.

Craig S. Smith

Reflections

Livewire Labs