PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

26:49

“Why Don’t We Just... Shoggoth+Face+Paraphraser?” by...

LessWrong (30+ Karma) ·

2024/11/19

Daniel KokotajloHere's a fairly concrete AGI safety proposal: Default AGI design: Let's suppose we are starting with a pretrained LLM 'base model' and then we are going to do a ton of additional RL ('agency training') to turn it into a general-purpose autonomous agent. So, during training it'll do lots of CoT of 'reasoning' (think like how o1 does it) and then it'll output some text that the user or some external interface sees (e.g. typing into a browser, or a chat window), and then maybe it'll get

en-gb

0:58

“U.S.-China Economic and Security Review Commission...

LessWrong (30+ Karma) ·

2024/11/19

https://x.com/hamandcheese/status/1858897287268725080"The annual report of the US-China Economic and Security Review Commission is now live. 🚨Its top recommendation is for Congress and the DoD to fund a Manhattan Project-like program to race to AGI." The original text contained 1 image which was described by AI. ---First published: November 19th, 2024 Source:

en-gb

1:10

“Looking back on the Future of Humanity Institute -...

LessWrong (30+ Karma) ·

2024/11/19

Asterisk's latest is a retrospective on FHI by Tom Ough. Including (stealing from Tom here:)☛ Its apparent origin: a candlelit nerdsnipe ☛ When Bostrom and Anders Sandberg lodged with Derek Parfit ☛ The 'whiteboard panopticon' ☛ FHI's profound influence on the development of AI ☛ The ingredients of Bostrom's 'elixir' ☛ How the relationship with the university went downhill ☛ The attempts to save FHI, inc. a mooted transfer to the Department of Physics ☛ The moment the university swung the axe ☛

en-gb

20:35

“How to use bright light to improve your life.” by...

LessWrong (30+ Karma) ·

2024/11/19

You may have heard that you 'shouldn't use screens late in the evening' and maybe even that 'it's good for you to get exposure to sunshine as soon as possible after waking'. For the majority of people, these are generally beneficial heuristics. They are also the extent of most people's knowledge about how light affects their wellbeing.The multiple mechanisms through which light affects our physiology make it hard to provide generalisable guidance. Among other things, the time of day, your genetics,

en-gb

63:07

“OpenAI Email Archives” by habryka

LessWrong (30+ Karma) ·

2024/11/19

As part of the court case between Elon Musk and Sam Altman, a substantial number of emails between Elon, Sam Altman, Ilya Sutskever, and Greg Brockman have been released as part of the court proceedings. I have found reading through these really valuable, and I haven't found an online source that compiles all of them in an easy to read format. So I made one.I used AI assistance to generate this, which might have introduced errors. Check the original source to make sure it's accurate before you quote

en-gb

58:35

“Training AI agents to solve hard problems could lead...

LessWrong (30+ Karma) ·

2024/11/19

TLDR: We want to describe a concrete and plausible story for how AI models could become schemers. We aim to base this story on what seems like a plausible continuation of the current paradigm. Future AI models will be asked to solve hard tasks. We expect that solving hard tasks requires some sort of goal-directed, self-guided, outcome-based, online learning procedure, which we call the “science loop”, where the AI makes incremental progress toward its high-level goal. We think this “science loop”

en-gb

97:17

“Monthly Roundup #24: November 2024” by Zvi

LessWrong (30+ Karma) ·

2024/11/18

This is your monthly roundup. Let's get right to it. Young People are Young and Stupid As a reminder that yes college students are often young and stupid and wrong about everything, remember the time they were behind a ban on paid public toilets? This is a central case of the kind of logic that often gets applied by college students. No One Voted for This HR and Title IX training seems like it's going a lot of compelled speech in the form of ‘agree with us or you can’t complete your

en-gb

8:55

“Why imperfect adversarial robustness doesn’t doom AI...

LessWrong (30+ Karma) ·

2024/11/18

(thanks to Alex Mallen, Cody Rushing, Zach Stein-Perlman, Hoagy Cunningham, Vlad Mikulik, and Fabien Roger for comments)Sometimes I hear people argue against AI control as follows: if your control measures rely on getting good judgments from "trusted" AI models, you're doomed because AI models aren't adversarially robust. I agree that better adversarial robustness would definitely help with control, but I think this argument misses several important points; my overall take is that current levels of

en-gb

21:26

“A Qualitative Case for LTFF: Filling Critical...

LessWrong (30+ Karma) ·

2024/11/18

The longtermist funding ecosystem needs certain functions to exist at a reasonable scale. I argue LTFF should continue to be funded because we're currently one of the only organizations comprehensively serving these functions. Specifically, we: Fund early-stage technical AI safety researchers working outside major labs Help talented people transition into existential risk work Provide independent funding voices to balance out AI company influence Move quickly to fund promising work in emerging

en-gb

4:59

“‘It’s a 10% chance which I did 10 times, so it...

LessWrong (30+ Karma) ·

2024/11/18

Audio note: this article contains 33 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. Many of you readers may instinctively know that this is wrong. If you flip a coin (50% chance) twice, you are not guaranteed to get heads. The odds of getting a heads are 75%. However you may be surprised to learn that there is some truth to this statement; modifying the statement just slightly will yield not just a true

en-gb

10:26

“College technical AI safety hackathon retrospective...

LessWrong (30+ Karma) ·

2024/11/18

TLDR: the AI safety initiative at Georgia Tech recently hosted an AI safety focused track at the college's flagship AI hackathon. In this post I share how it went and some of our thoughts. OverviewHey! I’m Yixiong, co-director of Georgia Tech's AI safety student group. We recently hosted an AI safety focused track at Georgia Tech's biggest AI hackathon, AI ATL. I’m writing this retrospective because I think this could be a useful data point to update on for fellow AIS groups thinking about hosting

en-gb

21:03

“‘The Solomonoff Prior is Malign’ is a special case...

LessWrong (30+ Karma) ·

2024/11/18

[Warning: This post is probably only worth reading if you already have opinions on the Solomonoff induction being malign, or at least heard of the concept and want to understand it better.] IntroductionI recently reread the classic argument from Paul Christiano about the Solomonoff prior being malign, and Mark Xu's write-up on it. I believe that the part of the argument about the Solomonoff induction is not particularly load-bearing, and can be replaced by a more general argument that I think is

en-gb

3:09

“Announcing turntrout.com, my new digital home” by...

LessWrong (30+ Karma) ·

2024/11/17

This is a link post.For months, I have built a new home for my online content: www.turntrout.com. I brooked no compromises. Over 2,220 commits later, I’m ready to publicize.Welcome to The Pond! I commissioned this GIF for $270.94. I paid a bit extra to ensure the goose honks twice.I don’t want to be on LessWrong anymore. Briefly, the site—and parts of the rationality community—don’t meet my standards for discourse, truth-seeking, charity, and community health. For the most part, I’ll elaborate my

en-gb

29:49

“Cross-context abduction: LLMs make inferences about...

LessWrong (30+ Karma) ·

2024/11/17

Epistemic status: These are first positive results. I have not yet run extensive tests to verify repeatability, so take them with a grain of salt. This post is meant to disseminate early results and collect ideas for further experiments to concretise these findings. Tldr:I study whether LLMs understand their training data and can use that understanding to make inferences about later training data. Specifically, I measure whether LLMs can infer which declarative facts in their training data are

en-gb

9:55

“Which evals resources would be good?” by Marius...

LessWrong (30+ Karma) ·

2024/11/16

I want to make a serious effort to create a bigger evals field. I’m very interested in which resources you think would be most helpful. I’m also looking for others to contribute and potentially some funding. Which resources would be most helpful? Suggestions include: Open Problems in Evals list: A long list of open relevant problems & projects in evals.More Inspect Tutorials/Examples:Specifically, examples with deep explanations, examples of complex agent setups, and more detailed examples of

en-gb

63:45

“OpenAI Email Archives (from Musk v. Altman)” by habryka

LessWrong (30+ Karma) ·

2024/11/16

en-gb

9:03

“Ayn Rand’s model of ‘living money’; and an upside of...

LessWrong (30+ Karma) ·

2024/11/16

Epistemic status: Toy model. Oversimplified, but has been anecdotally useful to at least a couple people, and I like it as a metaphor. IntroductionI’d like to share a toy model of willpower: your psyche's conscious verbal planner “earns” willpower (earns a certain amount of trust with the rest of your psyche) by choosing actions that nourish your fundamental, bottom-up processes in the long run. For example, your verbal planner might expend willpower dragging you to disappointing first dates, then

en-gb

13:39

“Win/continue/lose scenarios and...

LessWrong (30+ Karma) ·

2024/11/15

In this post, I’ll make a technical point that comes up when thinking about risks from scheming AIs from a control perspective.In brief: Consider a deployment of an AI in a setting where it's going to be given a sequence of tasks, and you’re worried about a safety failure that can happen suddenly. Every time the AI tries to attack, one of three things happens: we win, we lose, or the deployment continues (and the AI probably eventually attempts to attack again). So there's two importantly different

en-gb

14:21

“Making a conservative case for alignment” by Cameron...

LessWrong (30+ Karma) ·

2024/11/15

Trump and the Republican party will yield broad governmental control during what will almost certainly be a critical period for AGI development. In this post, we want to briefly share various frames and ideas we’ve been thinking through and actively pitching to Republican lawmakers over the past months in preparation for this possibility.Why are we sharing this here? Given that >98% of the EAs and alignment researchers we surveyed earlier this year identified as

en-gb

9:07

“The Third Fundamental Question” by Screwtape

LessWrong (30+ Karma) ·

2024/11/15

I.Epistemic status: I’m making a big claim based on little more than how useful it's been to me personally. Pushback is encouraged. That said, I’m also putting this forward as one tool; if your brain is like my brain then you may find this a very useful tool indeed, while other people may not find it's something they use.It has been written that the fundamental question of rationality is “What do you think you know and how do you think you know it?” There is a further claim that a question of equal

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

The Automated Daily

TrendTeller

Reflections

Livewire Labs

Eli the Computer Guy

Eli the Computer Guy

LessWrong (30+ Karma)

LessWrong

Superkraft KI News | by Jörg Schieb

Jörg Schieb