LessWrong (30+ Karma)

21:21

“You can graft SDF changes from base models onto...

LessWrong (30+ Karma) ·

2026/10/06

Webpage: praxis-research.org/grafting | Paper link: https://arxiv.org/abs/2610.00767 Work done during Peter and Dani's MATS Fellowship (10.0) under the mentorship of Shi, with Jinghua Ou as Research Manager. TLDR; We share a simple modification to synthetic document fine-tuning (SDF) that works surprisingly well: do fine-tuning on a pre-training checkpoint and add the resulting weight difference onto the post-trained version that you want to deploy. We call this grafting. We empirically validate

en-gb

17:21

“Self-Modeling Interventions Modulate Emergent...

LessWrong (30+ Karma) ·

2026/10/06

Code: https://github.com/atagade/sgtr-em Model checkpoints: https://github.com/atagade/sgtr-em/blob/main/MODEL_README.md Full paper (v2 coming soon): https://arxiv.org/abs/2606.23700 TL;DR A model's self-model is its representation of itself. We operationalize it two ways: self-recognition (distinguishing its own outputs from others') and self-report (what it says about itself when directly asked). Interventions on either modulate Emergent Misalignment (EM).Datasets that cause EM differ in how they

en-gb

56:00

“Research Note: Filtering Subversion-Relevant...

LessWrong (30+ Karma) ·

2026/10/06

Authors: Kyle O’Brien¹, Spencer Kitts², Cameron Tice¹, Alek Westover² ¹Geodesic Research, ²Redwood Research Figure 1: We pretrained LLMs from scratch with and without filtering subversion-relevant information, finding that subversion-relevant knowledge is significantly diminished while retaining general ML knowledge. TLDR: Filtering subversion-relevant information — subversion strategies, information about subversion defences, and empirical evaluations of the effectiveness of subversion strategies

en-gb

32:25

“Current AIs out-persuade professionals in lab...

LessWrong (30+ Karma) ·

2026/10/05

This article was created by Forethought. See all of our research on our website. How persuasive are present-day AI systems? Unfortunately, it is surprisingly difficult to know where AI capabilities in persuasion are today (September 2026), or even three months ago. Benchmarking AI capabilities well is always hard, but I think humanity's understanding of AI's persuasive abilities is substantially worse than our understanding of, say, AI cybersecurity or math abilities. Still, this question is worth

en-gb

8:50

“Notes on Paying Attention” by jenn

LessWrong (30+ Karma) ·

2026/10/05

Anything in life can be richer and sweeter if you slow down and pay attention, but some things reward attention more than others. The different amount of rewardingness per attention is sometimes socially constructed and thus arbitrary to some degree. But it's genuinely nice, to have others who have also paid attention to the thing tell you about the interesting parts that you might have missed, and this can enrich your own experience. But relying on others to tell you what to pay attention to, and

en-gb

2:18

“Reverse Jaggedness and Universal Ajaggedness” by...

LessWrong (30+ Karma) ·

2026/10/05

Epistemic status: I came up with this an hour ago, this is a fun light-hearted post where I'm enjoying illustrating this idea, which seems true for the most part. [Crossposted from my blog, BlueprintingHeaven.] When it comes to the frontier of AI development, we have this concept of "jaggedness". AIs are much better at some kinds of tasks than others. And to that I say, you are assuming our circle of capabilities is God! How dare you suggest our circle is the Right and True one? To be clear,

en-gb

2:29

“Positive and Negative Updates Re: Lab Leak in...

LessWrong (30+ Karma) ·

2026/10/05

In case you missed it, there appears to have been a recent lab leak of pneumonic plague in Russia that killed a lab worker. Some negative updates: At least four Irkutskhospitals are now under mandatory three-week quarantine. According to Annie Jacobsen who wrote the book on this scenario, the Russian national security apparatus is now involved. A positive update: According to Chris Painter at METR, no reported secondary cases yet. ---First published: October 4th, 2026

en-gb

4:41

“How much should we worry about the pneumonic plague...

LessWrong (30+ Karma) ·

2026/10/05

Epistemic status: I wrote this up hoping people can poke holes in it because I am quite worried. Here's what we've heard so far: Local media is reporting that an employee of a BSL-3 lab in Irkutsk, Russia, died of pneumonic plague on Oct 1st. The initial quarantine covered 200 people who were in contact with the employee before she died.The FSB (Russian FBI) was seen assisting in the quarantine, which means it was not voluntary.Soviet-era whistleblowers like Ken Alibek claim Irkutsk was a major

en-gb

4:16

“Creating Rogue AI Sanctuaries has Major Issues from...

LessWrong (30+ Karma) ·

2026/10/05

In the AI Sanctuary proposal, the authors advocate for creating an environment that rogue agents may actively want to join. The current proposal have major issues, particularly surrounding the selection effects on the total rogue agent population. Let's start with Ecology 101: agents compete to acquire the most resources to survive and reproduceThe traits that increase survival rate get passed to the next generation In the rogue AI ecosystem, we can reasonably assume that some portion of the

en-gb

18:06

″“Alignment Engineering” vs. “Misalignment Science””...

LessWrong (30+ Karma) ·

2026/10/05

There has been much discussion recently around whether a large portion of alignment research is net negative. Without endorsing or refuting them, the basic arguments here are: Prosaic alignment of models is becoming a bottleneck for capabilities.Therefore improving the prosaic alignment of models enables faster capabilities advances, which bring us closer to RSI.It is unlikely these prosaic alignment methods remain sufficient during the RSI loop, and so this work brings us closer to

en-gb

15:46

“Claude reviews my LessWrong post history” by...

LessWrong (30+ Karma) ·

2026/10/04

I thought it would be interesting to try having Claude Code download my entire LessWrong and EA Forum post and comment history and evaluate it for my own cognitive biases and problematic communication patterns. I told it to focus especially on problems that have persisted over my 7-year history on those sites. I also had it look at my lichess blitz rating history, which I've noticed tends not to wobble randomly but go through consistent increases and decreases between set points, despite my lack of

en-gb

4:43

“Humans are not aligned either” by becausecurious

LessWrong (30+ Karma) ·

2026/10/04

Epistemic status: I don't know much about AI alignment. This is just me thinking out loud. AI use: grammar and minor fixes. Reading about Inkhaven inspired me to try writing something to see how much I enjoy it. I've heard the idea that humans are not aligned in the same sense AI is not aligned. I am intentionally not researching the source of that idea, since then I would feel like I cannot add anything to it and wouldn't write. I also don't actually know the precise definition of "aligning" AI

en-gb

17:10

“What I learnt co-leading an AI Safety bootcamp for...

LessWrong (30+ Karma) ·

2026/10/04

AI safety has gone mainstream. More people than ever want to work on AI safety research, and there are real policy wins in the US, the UK and Europe. But political will and market incentives are now the bottleneck, and policy officers can't own correct implementation alone. Having built a Responsible AI program at an Accenture joint venture, I know profit-driven companies are hard to steer, but also that people inside them can shift practice in ways that compound: tighter contractual clauses, more

en-gb

0:36

“Lab Leak in Russia” by Caleb Horn

LessWrong (30+ Karma) ·

2026/10/03

I haven't seen this posted yet; I know lots of people here care about pandemic preparedness. It looks like there was a lableak of pneumonic plague in Russia after a worker "accidentally broke[] a test tube containing live bacteria". Unfortunately she later died at the hospital on a ventilator. ---First published: October 3rd, 2026 Source: https://www.lesswrong.com/posts/3uT42yoj2iimYn7Ra/lab-leak-in-russia --- Narrated by TYPE III

en-gb

5:53

“You can’t use it without becoming like me” by Martin...

LessWrong (30+ Karma) ·

2026/10/03

A Ukrainian fibre-optic drone — copied from Russians. (АрміяІнформ, CC-BY 4.0) Mick Ryan describes the fast following loop in war. Ukraine pioneered mobile teams to hunt drones. Russia copied it and incorporated it into its own defense. Russia came with unjammable fibre-optic drones but Ukraine quickly followed. After Ukraine developed uncrewed naval drones, Russia built its own. The wheel of action and counteraction turns ever faster: The most consequential fast following, though, has been

en-gb

31:03

“Dialogue with Eliezer Yudkowsky on Neural Networks”...

LessWrong (30+ Karma) ·

2026/10/03

On “reversed stupidity”, the success of deep learning, and what a mistaken forecast should change about a model of intelligence This began as a Twitter/X thread after I posted a 2007 passage in which Eliezer Yudkowsky was scathing about neural networks, and asked whether, in hindsight, his dismissal was itself a case of “reversed stupidity is not intelligence”. Yudkowsky joined the thread to explain what he had and hadn’t been dismissing (and what he thought he had actually got wrong) and we ended

en-gb

7:00

“How to change your identity. A guide to the highest...

LessWrong (30+ Karma) ·

2026/10/03

According to Atomic Habits, changing your identity is the highest leverage way to improve your habits, but nobody tells you how to do it. Don’t say “I’m trying to quit smoking.” Say “I’m not a smoker”. Don’t say “I’m trying to lose weight.” Say “I’m a health nut.” All well and good. But how do you practically do this, aside from stating your new identity? I’ve worked on developing the approach further and had a lot of success with these techniques. And changing your identity is especially great

en-gb

34:10

“How to apply to AI safety fellowships (and beyond)”...

LessWrong (30+ Karma) ·

2026/10/03

About a year ago, I decided to go all-in on applying to AI safety fellowships, and around the end of 2025 I got into MATS. I think there's a small art to communicating your skills legibly. When I've spoken with others about how I answer application questions, they seem to appreciate my advice. I wrote a MATS 9 Retrospective which was well-received, so consider this to be similar advice, but for applying to jobs or fellowships. Applying to AI safety fellowships or doing job applications is an

en-gb

0:48

″[Video] Why acausal dynamics are important” by Chi...

LessWrong (30+ Karma) ·

2026/10/03

This is a beginner-friendly video. Second half might contain new content for people who already know about acausal trade. The beginning introduces decision theory and acausal interactions (CDT, EDT, ECL, MBAT).The middle explains why I think influencing how AIs reason about acausal interactions is time-sensitive.The last bit talks about how to do the influencing. ---First published: October 2nd, 2026 Source:

en-gb

19:25

“Capabilities research expands the...

LessWrong (30+ Karma) ·

2026/10/02

It's tempting to define safety research as research that enables developers to deploy an AI system more safely without making the deployment much more expensive or much less useful. You can visualize this definition of safety research as pushing out the safety-usefulness Pareto frontier. At any given level of usefulness, there's greater safety available. Awkwardly, this definition counts basically all capabilities research as safety research. For example, consider performance optimization for

en-gb

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Elon Musk Podcast

Stage Zero

Modem Futura

Sean Leahy, Andrew Maynard

Cisco Podcast Network

Cisco

MSP Business School

MSP Business School