PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

8:56

“Covert Malicious Finetuning” by Tony Wang, dannyhalawi

LessWrong (30+ Karma) ·

2024/07/02

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This post discusses our recent paper Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation and comments on its implications for AI safety. What is Covert Malicious Finetuning?Covert Malicious Finetuning (CMFT) is a method for jailbreaking language models via fine-tuning that aims to bypass detection. The following diagram gives an overview of what CMFT accomplishes:To unpack the diagram: An

en-gb

15:16

“In Defense of Lawyers Playing Their Part” by Isaac King

LessWrong (30+ Karma) ·

2024/07/02

This is a linkpost for In Defense of Lawyers Playing Their Part.Michael Huemer writes about why he believes it's wrong for lawyers to pursue unjust legal outcomes.It's a good article, and one of the best defenses of this position I've seen. Still, I think this argument is mistaken. The reason why we require lawyers to fight for "their side" even if they believe they're in the wrong is to minimize the opportunity for bias.Imagine if all trials were bench trials, decided by only one person as the

en-gb

2:00

“Important open problems in voting” by Closed...

LessWrong (30+ Karma) ·

2024/07/02

Strategy-resistanceIdentify, or prove impossibility, of a voting system which incentivizes— A strictly sincere ranking of all candidates in the zero-information setting, where it implements a "good" social choice rule such as the relative (normalized) utilitarian rule, a Condorcet social choice rule, or the Borda rule.In a Poisson game or similar setting: a unique semi-sincere Nash equilibrium that elects the Condorcet winner (if one exists), similar to those shown for approval voting by Myerson and

en-gb

19:58

“Interpreting Preference Models w/ Sparse...

LessWrong (30+ Karma) ·

2024/07/02

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is the real reward output for an OS preference model. The bottom "jailbreak" completion was manually created by looking at reward-relevant SAE features.Preference Models (PMs) are trained to imitate human preferences and are used when training with RLHF (reinforcement learning from human feedback); however, we don't know what features the PM is using when outputting reward. For example, maybe curse words make

en-gb

3:08

“New Executive Team & Board — PIBBSS ” by Nora_Ammann

LessWrong (30+ Karma) ·

2024/07/01

TLDR: PIBBSS is changing our core team. Nora is stepping down as director due to joining ARIA, and Lucas Teixeira and Dusan Nesic are taking over her leadership role. Nora joins the board, alongside Tan Zhi Xuan, Alexander Gietelink Oldenziel, Ben Goldhaber and Gabriel Weil.***I (Nora) have recently accepted an offer to join ARIA's Safeguarded AI Programme as Technical Specialist under davidad. As such, I am stepping back as Director at PIBBSS, after co-founding and leading PIBBSS since 2021. It

en-gb

12:44

“Datasets that change the odds you exist” by dynomight

LessWrong (30+ Karma) ·

2024/06/30

This is a link post. 1.It's October 1962. The Cuban missile crisis just happened, thankfully without apocalyptic nuclear war. But still: Apocalyptic nuclear war easily could have happened.Crises as serious as the Cuban missile crisis clearly aren’t that rare, since one just happened.You estimate (like President Kennedy) that there was a 25% chance the Cuban missile crisis could have escalated to nuclear war. And you estimate that there's a 4% chance of an equally severe crisis happening each year

en-gb

10:28

“Contra Acemoglu on AI” by Maxwell Tabarrok

LessWrong (30+ Karma) ·

2024/06/29

This is a link post.The Simple Macroeconomics of AI is a 2024 working paper by Daron Acemoglu which models the economic growth effects of AI and predicts them to be small: About a .06% increase in TFP growth annually. This stands in contrast to many predictions which forecast immense impacts on economic growth from AI, including many from other academic economists. Why does Acemoglu come to such a different conclusion than his colleagues and who is right?First, Acemoglu divides up the ways AI could

en-gb

14:22

“The Incredible Fentanyl-Detecting Machine” by...

LessWrong (30+ Karma) ·

2024/06/29

An NII machine in Nogales, AZ. (Image source)There's bound to be a lot of discussion of the Biden-Trump presidential debates last night, but I want to skip all the political prognostication and talk about the real issue: fentanyl-detecting machines.Joe Biden says: And I wanted to make sure we use the machinery that can detect fentanyl, these big machines that roll over everything that comes across the border, and it costs a lot of money. That was part of this deal we put together, this bipartisan

en-gb

10:44

“How a chip is designed” by YM

LessWrong (30+ Karma) ·

2024/06/28

Disclaimer: This is highly incomplete. I am not an expert in the field. There might be some unfamiliar terms. While I will try to explain things, explaining every single term would be beyond this post. You will usually be able to get a sufficient understanding by clicking the links or googling it. Introduction Introduction . I think everyone, if they read about the chip industry long enough, has a moment where they have to put down a book or pause a podcast and simply remain stunned at the fact

en-gb

9:29

“how birds sense magnetic fields” by bhauth

LessWrong (30+ Karma) ·

2024/06/28

This is a link post.introduction introduction . It is known that many birds are able to sense the direction of Earth's magnetic field. Here's a wikipedia page on that general phenomenon. There have been 2 main theories of how that works. One theory is that birds have magnets in their beak that act like a compass. We know this is the correct theory because: Small magnetite crystals have been found in bird beaks. Anaesthesia of bird beaks seems to affect their magnetic sense, sometimes. The

en-gb

45:47

“Childhood and Education Roundup #6: College Edition”...

LessWrong (30+ Karma) ·

2024/06/28

Childhood roundup #5 excluded all developments around college. So this time around is all about issues related to college or graduate school, including admissions. Tuition and Costs What went wrong with federal student loans? Exactly what you would expect when you don’t check who is a good credit risk. From a performance perspective, the federal government offered loans to often-unqualified students to attend poor-performing, low-value institutions. Those students then did not earn much

en-gb

18:57

“Corrigibility = Tool-ness?” by johnswentworth, David...

LessWrong (30+ Karma) ·

2024/06/28

Goal of This PostI have never seen anyone give a satisfying intuitive explanation of what corrigibility (in roughly Eliezer's sense of the word) is. There's lists of desiderata, but they sound like scattered wishlists which don’t obviously point to a unified underlying concept at all. There's also Eliezer's extremely meta pointer:We can imagine, e.g., the AI imagining itself building a sub-AI while being prone to various sorts of errors, asking how it (the AI) would want the sub-AI to behave in

en-gb

2:53

“Secondary forces of debt” by KatjaGrace

LessWrong (30+ Karma) ·

2024/06/28

A general thing I hadn’t noticed about debts until lately:Whenever Bob owes Alice, then Alice has reason to look after Bob, to the extent that increases the chance he satisfies the debt. Yet at the same time, Bob has an incentive for Alice to disappear, insofar as it would relieve him. These might be tiny incentives, and not overwhelm for instance Bob's many reasons for not wanting Alice to disappear. But the bigger the owing, the more relevant the incentives. When big enough, the former

en-gb

85:28

“AI #70: A Beautiful Sonnet” by Zvi

LessWrong (30+ Karma) ·

2024/06/28

They said it couldn’t be done. No, not Claude Sonnet 3.5 becoming the clear best model. No, not the Claude-Sonnet-empowered automatic meme generators. Those were whipped together in five minutes. They said I would never get quiet time and catch up. Well, I showed them! That's right. Yes, there is a new best model, but otherwise it was a quiet week. I got a chance to incorporate the remaining biggest backlog topics. The RAND report is covered under Thirty Eight Ways to Steal Your Model

en-gb

14:07

“Detecting Genetically Engineered Viruses With...

LessWrong (30+ Karma) ·

2024/06/27

This is a link post. This represents work from several people at the NAO. Thanks especially to Dan Rice for implementing the duplicate junction detection, and to @Will Bradshaw and @mike_mclaren for editorial feedback.Summary Summary . If someone were to intentionally cause a stealth pandemic today, one of the ways they might do it is by modifying an existing virus. Over the past few months we’ve been working on building a computational pipeline that could flag evidence of this kind of genetic

en-gb

10:27

“Schelling points in the AGI policy space” by...

LessWrong (30+ Karma) ·

2024/06/27

I've been thinking about memetically fit Schelling points in the AGI policy space. I'll describe four such "Schelling policies", and use them as pedagogical examples.Shut it all down MIRI's new stated objective is the clearest example of a Schelling policy: "Shut it all down". MIRI states that they want governments to coordinate to pause all AI research that involves smarter-than-human systems. Laypeople will find this policy easy to understand, since they can rely on the shared cultural knowledge

en-gb

18:00

“Live Theory Part 0: Taking Intelligence Seriously”...

LessWrong (30+ Karma) ·

2024/06/27

AcknowledgementsThe vision here was midwifed originally in the wild and gentle radiance that is Abram's company (though essentially none of the content is explicitly his). The PIBBSS-spirit has been infused in this work from before it began (may it infuse us all), as have meetings with the Agent Foundations team at MIRI over the past ~2 years. More recently, everyone who has been loving the High Actuation project into form (very often spontaneously and without being encumbered by self-consciousness

en-gb

2:55

“Progress Conference 2024: Toward Abundant Futures”...

LessWrong (30+ Karma) ·

2024/06/26

This is a link post.The progress movement has grown a lot in the last few years. We now have progress journals, think tanks, and fellowships. The progress idea has spread and evolved into the “abundance agenda”, “techno-optimism”, “supply-side progressivism”, “American dynamism”. All of us want to see more scientific, technological, and economic progress for the good of humanity, and envision a bold, ambitious, flourishing future.What we haven’t had so far is a regular gathering of the

en-gb

11:17

“What is a Tool?” by johnswentworth, David Lorell

LessWrong (30+ Karma) ·

2024/06/26

Throughout this post, we’re going to follow the Cognition -> Convergence -> Consequences methodology[1]. That means we’ll tackle tool-ness in three main stages, each building on the previous: Cognition: What does it mean, cognitively, to view or model something as a tool?Convergence: Insofar as different minds (e.g. different humans) tend to convergently model thesame things as tools, what are the “real patterns” in the environment which give rise to that convergence?Consequences: Having

en-gb

2:45

“In favour of exploring nagging doubts about x-risk”...

LessWrong (30+ Karma) ·

2024/06/26

I was chatting recently to someone who had difficulty knowing how to orient to x-risk work (despite being a respected professional working in that field). They expressed that they didn't find it motivating at a gut level in the same way they did with poverty or animal stuff; and relatedly that they felt some disconnect between the arguments that they intellectually believed, and what they felt to be important. I think that existential security (or something like it) should be one of the top

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Reflections

Livewire Labs

ForGeeks

ForGeeks

FOX on Tech

FOX News Podcasts

The Linux Cast

The Linux Cast