PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

113:23

“39 - Evan Hubinger on Model Organisms of...

LessWrong (30+ Karma) ·

2024/12/02

YouTube link The ‘model organisms of misalignment’ line of research creates AI models that exhibit various types of misalignment, and studies them to try to understand how the misalignment occurs and whether it can be somehow removed. In this episode, Evan Hubinger talks about two papers he's worked on at Anthropic under this agenda: “Sleeper Agents” and “Sycophancy to Subterfuge”. Topics we discuss: Model organisms and stress-testing Sleeper Agents Do ‘sleeper agents’ properly model deceptive

en-gb

63:16

“(The) Lightcone is nothing without its people: LW +...

LessWrong (30+ Karma) ·

2024/11/30

TLDR: LessWrong + Lighthaven need about $3M for the next 12 months. Donate here, or send me an email, DM or signal message (+1 510 944 3235) if you want to support what we do. Donations are tax-deductible in the US. Reach out for other countries, we can likely figure something out. We have big plans for the next year, and due to a shifting funding landscape we need support from a broader community more than in any previous year.I've been running LessWrong/Lightcone Infrastructure for the last 7

en-gb

11:16

“You should consider applying to PhDs soon.” by...

LessWrong (30+ Karma) ·

2024/11/29

TLDR: In this post, I argue that if you are a junior AI safety researcher, you should consider applying to PhD programs in ML soon, especially if you have recently participated in an AI safety upskilling or research program like MATS or ARENA, might be interested in working on AI safety long term, but don't have immediate career plans. It is relatively cheap to apply, and provides good future option value. I don’t argue that you should necessarily do a PhD, but some other posts do. PhD application

en-gb

10:26

“Causal inference for the home gardener” by braces

LessWrong (30+ Karma) ·

2024/11/29

Note: This is meant to be an accessible introduction to causal inference. Comments appreciated.Let's say you buy a basil plant and put it on the counter in your kitchen. Unfortunately, it dies in a week.So the next week you buy another basil plant and feed it a special powder, Vitality Plus. This second plant lives. Does that mean Vitality Plus worked? Not necessarily! Maybe the second week was a lot sunnier, you were better about watering, or you didn’t grab a few leaves for a pasta. In other

en-gb

87:57

“The Big Nonprofits Post” by Zvi

LessWrong (30+ Karma) ·

2024/11/29

There are lots of great charitable giving opportunities out there right now.The first time that I served as a recommender in the Survival and Flourishing Fund (SFF) was back in 2021. I wrote in detail about my experiences then. At the time, I did not see many great opportunities, and was able to give out as much money as I found good places to do so. How the world has changed in three years. I recently had the opportunity to be an SFF recommender for the second time. This time I found an

en-gb

18:02

“Estimates of GPU or equivalent resources of large AI...

LessWrong (30+ Karma) ·

2024/11/29

AI infrastructure numbers are hard to find with any precision. There are many reported numbers of “[company] spending Xbn on infrastructure this quarter” and “[company] has bought 100k H100s or “has a cluster of 100k H100s” but when I went looking for an estimate of how much compute a given company had access to, I could not find consistent numbers available. Here I’ve tried to pull together information from a variety of sources to get ballpark estimates of (i) as of EOY 2024, who do we expect to

en-gb

29:35

“ARENA 4.0 Impact Report” by Chloe Li, JamesH, James Fox

LessWrong (30+ Karma) ·

2024/11/28

If you're interested in helping to run the ARENA program, note that we're currently hiring for an operations lead! For more details, and to apply, see here. SummaryThe purpose of this report is to evaluate ARENA 4.0's impact according to our four success criteria: Source high-quality participantsUpskill these talented participants in ML skills for AI safety workIntegrate participants with the existing AI safety community and legitimise AI safety as a compelling field to work inAccelerate

en-gb

82:20

“AI #92: Behind the Curve” by Zvi

LessWrong (30+ Karma) ·

2024/11/28

People don’t give thanks enough, and it's actual Thanksgiving, so here goes.Thank you for continuing to take this journey with me every week. It's a lot of words. Even if you pick and choose, and you probably should, it's a lot of words. You don’t have many slots to spend on things like this. I appreciate it. Thanks in particular for those who are actually thinking about all this, and taking it seriously, and forming their own opinions. It is the only way. To everyone who is standing up, peacefully

en-gb

4:48

“‘Map of AI Futures’ - An interactive flowchart” by...

LessWrong (30+ Karma) ·

2024/11/28

This is a link post.Note: The linked site is a personal project, and all views expressed here are my own.TL;DR I created an interactive flowchart about various scenarios how the future of AI might play out.By setting various conditional probabilities you can see charts of what the resulting estimated probabilities for good, ambiguous, and existentially bad outcomes are.Use the site as a conversation starter and tool for reflection. You can share your results either as images of the resulting outcome

en-gb

9:39

“Dave Kasten’s AGI-by-2027 vignette” by davekasten

LessWrong (30+ Karma) ·

2024/11/28

(Epistemic status: Very loosely held and generated in a 90-minute workshop led by @Daniel Kokotajlo, @Thomas Larsen, @elifland, and Jonas Vollmer at The Curve Conference; explores how it might happen, if it happens soon. I expect there to be at least one "duh, that makes no sense" discovered with any significant level of attention that would require me to rethink some of this.) Recently, at The Curve conference, I participated in a session that helped facilitate us writing AGI vignettes --

en-gb

2:25

“New o1-like model (QwQ) beats Claude 3.5 Sonnet with...

LessWrong (30+ Karma) ·

2024/11/27

This is a link post.A new o1-like model based on Qwen-2.5-32B reportedly beats Claude 3.5 Sonnet[1] on a bunch of difficult reasoning benchmarks. A new regime dawns.The blog post reveals nothing but the most inane slop ever sampled:What does it mean to think, to question, to understand? These are the deep waters that QwQ (Qwen with Questions) wades into. Like an eternal student of wisdom, it approaches every problem - be it mathematics, code, or knowledge of our world - with genuine wonder and

en-gb

20:42

“38.2 - Jesse Hoogland on Singular Learning Theory”...

LessWrong (30+ Karma) ·

2024/11/27

YouTube linkYou may have heard of singular learning theory, and its “local learning coefficient”, or LLC - but have you heard of the refined LLC? In this episode, I chat with Jesse Hoogland about his work on SLT, and using the refined LLC to find a new circuit in language models. Topics we discuss: About Jesse The Alignment Workshop About Timaeus SLT that isn’t developmental interpretability The refined local learning coefficient Finding the multigram circuit Daniel Filan (00:09): Hello,

en-gb

4:35

“Call for evaluators: Participate in the European AI...

LessWrong (30+ Karma) ·

2024/11/27

I am sharing this call from the EU AI Office for organizations involved in evaluation. Please take a close look: among the selection criteria, organizations must be based in Europe, or their leader must be European. If these criteria pose challenges for some of you, feel free to reach out to me at tom@prism-eval.ai. We can explore potential ways to collaborate through PRISM Eval. I believe it's crucial that we support one another on these complex and impactful issues. The AI office is collecting

en-gb

23:02

“Hierarchical Agency: A Missing Piece in AI...

LessWrong (30+ Karma) ·

2024/11/27

The most ambitious direction I've worked on in the past few years is a theory of hierarchical agency (as mentioned in the ACS announcement). Recently, I've become worried that the inferential distance between "the median of current AI safety discourse" and "what I and my colleagues at ACS work on" has become too large. After multiple attempts to write a good introduction to hierarchical agency ended up in perpetual draft form, I decided to try something else: explain it to Claude. This is a lightly

en-gb

3:57

“Locally optimal psychology” by Chipmonk

LessWrong (30+ Karma) ·

2024/11/26

This is a link post.Thanks to Kaj Sotala, Brian Toomey, Stag Lynn, Ethan Kuntz, and Anna Salamon.There's no way that chronic depression, self-loathing, poor agency, or muscle tension could be optimal… right? Jake was depressed for 6 months. He also felt horrible every time he interacted with other people because of his emotional insecurities.So without knowing how to outgrow his insecurities, his system basically had two options: Interact with other people —and constantly feel horribleDon’t interact

en-gb

23:37

“You are not too ‘irrational’ to know your...

LessWrong (30+ Karma) ·

2024/11/26

Epistemic Status: 13 years working as a therapist for a wide variety of populations, 5 of them working with rationalists and EA clients. 7 years teaching and directing at over 20 rationality camps and workshops. This is an extremely short and colloquially written form of points that could be expanded on to fill a book, and there is plenty of nuance to practically everything here, but I am extremely confident of the core points in this frame, and have used it to help many people break out of or avoid

en-gb

62:01

“Counting AGIs” by cash, Will Taylor

LessWrong (30+ Karma) ·

2024/11/26

“The resources used to train the model can be repurposed to run millions of instances of it (this matches projected cluster sizes by ~2027), and the model can absorb information and generate actions at roughly 10x-100x human speed. … We could summarize this as a ‘country of geniuses in a datacenter’.”Dario Amodei, CEO of Anthropic, Machines of Loving Grace“Let's say each copy of GPT-4 is producing 10 words per second. It turns out they would be able to run something like 300,000 copies of GPT-4 in

en-gb

18:36

“a space habitat design” by bhauth

LessWrong (30+ Karma) ·

2024/11/25

This post is mainly about a design concept for far-future large space habitats. some proposed designsAs you can see on Wikipedia, many space habitat designs have been proposed. Below are some that I thought were worth mentioning. current space stationsObviously, space stations with long-term occupants have already been made, the biggest being the ISS.issue: small modulesEach launch lifts a complete cylindrical module, and then the modules are assembled. This limits module diameter, which has some

en-gb

57:26

“Passages I Highlighted in The Letters of...

LessWrong (30+ Karma) ·

2024/11/25

All quotes, unless otherwise marked, are Tolkien's words as printed in The Letters of J.R.R.Tolkien: Revised and Expanded Edition. All emphases mine. Machinery is Power is EvilWriting to his son Michael in the RAF:[here is] the tragedy and despair of all machinery laid bare. Unlike art which is content to create a new secondary world in the mind, it attempts to actualize desire, and so to create power in this World; and that cannot really be done with any real satisfaction. Labour-saving machinery

en-gb

6:34

“Perils of Generalizing from One’s Social Group” by...

LessWrong (30+ Karma) ·

2024/11/25

I see people make statements of the form, "In my experience with people I encounter, X is correlated with ...". The problem is, there's an excellent chance that the people they deal with are very unrepresentative of the population they want to generalize about, and I rarely see them show awareness of the possibility that selection bias has created the effect they're describing.Scott has written about the strength of social group filter bubbles. But there's a systematic effect I want to highlight:

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

The Automated Daily

TrendTeller

Reflections

Livewire Labs

Eli the Computer Guy

Eli the Computer Guy

LessWrong (30+ Karma)

LessWrong

Superkraft KI News | by Jörg Schieb

Jörg Schieb