PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

6:47

“Anthropic announces interpretability advances. How...

LessWrong (30+ Karma) ·

2024/05/22

This is a link post.Anthropic just published a pretty impressive set of results in interpretability. This raises for me, once again the question and concern: Interpretability isn't alignment, right? It seems to me as though the vast bulk of alignment funding is now going to interpretability. Who is thinking about how to leverage interpretability into alignment? It intuitively seems as though we are better off the more we understand the cognition of foundation models. I think this is true, but there

en-gb

6:43

“EIS XIII: Reflections on Anthropic’s SAE Research...

LessWrong (30+ Karma) ·

2024/05/21

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.Part 13 of 12 in the Engineer's Interpretability Sequence. TL;DROn May 5, 2024, I made a set of 10 predictions about what the next sparse autoencoder (SAE) paper from Anthropic would and wouldn’t do. Today's new SAE paper from Anthropic was full of brilliant experiments and interesting insights, but it ultimately underperformed my expectations. I am beginning to be concerned that Anthropic's recent approach to

en-gb

38:35

“On Dwarkesh’s Podcast with OpenAI’s John Schulman”...

LessWrong (30+ Karma) ·

2024/05/21

Dwarkesh Patel recorded a Podcast with John Schulman, cofounder of OpenAI and at the time their head of current model post-training. Transcript here. John's job at the time was to make the current AIs do what OpenAI wanted them to do. That is an important task, but one that employs techniques that their at-the-time head of alignment, Jan Leike, made clear we should not expect to work on future more capable systems. I strongly agree with Leike on that. Then Sutskever left and Leike resigned, and

en-gb

11:50

“The Problem With the Word ‘Alignment’” by...

LessWrong (30+ Karma) ·

2024/05/21

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This post was written by Peli Grietzer, inspired by internal writings by TJ (tushant jha), for AOI[1]. The original post, published on Feb 5, 2024, can be found here: https://ai.objectives.institute/blog/the-problem-with-alignment.The purpose of our work at the AI Objectives Institute (AOI) is to direct the impact of AI towards human autonomy and human flourishing. In the course of articulating our mission and

en-gb

11:24

“New voluntary commitments (AI Seoul Summit)” by Zach...

LessWrong (30+ Karma) ·

2024/05/21

The UK and Republic of Korea governments announced that the following organisations have agreed to the Frontier AI Safety Commitments: AmazonAnthropicCohereGoogleG42IBMInflection AIMetaMicrosoftMistral AINaverOpenAISamsung ElectronicsTechnology Innovation InstitutexAIZhipu.aiThe above organisations, in furtherance of safe and trustworthy AI, undertake to develop and deploy their frontier AI models and systems[1] responsibly, in accordance with the following voluntary commitments, and to demonstrate

en-gb

6:31

“What’s Going on With OpenAI’s Messaging?” by ozziegooen

LessWrong (30+ Karma) ·

2024/05/21

This is a quickly-written opinion piece, of what I understand about OpenAI. I first posted it to Facebook, where it had some discussion. Some arguments that OpenAI is making, simultaneously: OpenAI will likely reach and own transformative AI (useful for attracting talent to work there). OpenAI cares a lot about safety (good for public PR and government regulations). OpenAI isn’t making anything dangerous and is unlikely to do so in the future (good for public PR and government regulations)

en-gb

20:03

“Anthropic: Reflections on our Responsible Scaling...

LessWrong (30+ Karma) ·

2024/05/20

This is a link post.Last September we published our first Responsible Scaling Policy (RSP) [LW discussion], which focuses on addressing catastrophic safety failures and misuse of frontier models. In adopting this policy, our primary goal is to help turn high-level safety concepts into practical guidelines for fast-moving technical organizations and demonstrate their viability as possible standards. As we operationalize the policy, we expect to learn a great deal and plan to share our findings. This

en-gb

84:45

“OpenAI: Exodus” by Zvi

LessWrong (30+ Karma) ·

2024/05/20

Previously: OpenAI: Facts From a Weekend, OpenAI: The Battle of the Board, OpenAI: Leaks Confirm the Story, OpenAI: Altman Returns, OpenAI: The Board Expands. Ilya Sutskever and Jan Leike have left OpenAI. This is almost exactly six months after Altman's temporary firing and The Battle of the Board, the day after the release of GPT-4o, and soon after a number of other recent safety-related OpenAI departures. Many others working on safety have also left recently. This is part of a longstanding

en-gb

0:52

“Jaan Tallinn’s 2023 Philanthropy Overview” by jaan

LessWrong (30+ Karma) ·

2024/05/20

This is a link post.to follow up my philantropic pledge from 2020, i've updated my philanthropy page with 2023 results.in 2023 my donations funded $44M worth of endpoint grants ($43.2M excluding software development and admin costs) — exceeding my commitment of $23.8M (20k times $1190.03 — the minimum price of ETH in 2023). --- First published: May 20th, 2024 Source:

en-gb

30:33

“International Scientific Report on the Safety of...

LessWrong (30+ Karma) ·

2024/05/19

I thought that the recently released International Scientific Report on the Safety of Advanced AI seemed like a pretty good summary of the state of the field on AI risks, in addition to being about as close to a statement of expert consensus as we're likely to get at this point. I noticed that each section of the report has a useful "Key Information" bit with a bunch of bullet points summarizing that section.So for my own use as well as perhaps the use of others, and because I like bullet-point

en-gb

10:45

“Fund me please - I Work so Hard that my Feet start...

LessWrong (30+ Karma) ·

2024/05/18

Bleeding Feet and DedicationDuring AISC 2024 I was working with somebody about how to use binary search to approximate a hull that would contain a set of points, only to knock off a glass of my table. It splintered into a thousand pieces all over my floor.A normal person might stop and remove all the glass splinters. I just spent 10 seconds picking up some of the largest pieces and then decided that it would be better to push on the train of thought without interruption.Some time later I forgot

en-gb

3:57

“‘If we go extinct due to misaligned AI, at least...

LessWrong (30+ Karma) ·

2024/05/18

[memetic status: stating directly despite it being a clear consequence of core AI risk knowledge because many people have "but nature will survive us" antibodies to other classes of doom and misapply them here.]Unfortunately, no.[1]Technically, “Nature”, meaning the fundamental physical laws, will continue. However, people usually mean forests, oceans, fungi, bacteria, and generally biological life when they say “nature”, and those would not have much chance competing against a misaligned

en-gb

7:21

“DeepMind’s ’Frontier Safety Framework’ is weak and...

LessWrong (30+ Karma) ·

2024/05/18

FSF blogpost. Full document (just 6 pages; you should read it). Compare to Anthropic's RSP, OpenAI's RSP ("PF"), and METR's Key Components of an RSP.DeepMind's FSF has three steps: Create model evals for warning signs of "Critical Capability Levels" Evals should have a "safety buffer" of at least 6x effective compute so that CCLs will not be reached between evalsThey list 7 CCLs across "Autonomy, Biosecurity, Cybersecurity, and Machine Learning R&D" E.g. "Autonomy level 1: Capable of expanding

en-gb

10:50

“The Dunning-Kruger of disproving Dunning-Kruger” by...

LessWrong (30+ Karma) ·

2024/05/18

In an online discussion elsewhere today someone linked this article which in turn linked the paper Gignac & Zajenkowski, The Dunning-Kruger effect is (mostly) a statistical artefact: Valid approaches to testing the hypothesis with individual differences data (PDF) (ironically hosted on @gwern's site).And I just don't understand what they were thinking.Let's look at their methodology real quick in section 2.2 (emphasis added):2.2.1. Subjectively assessed intelligence Participants assessed their

en-gb

10:07

“Is There Really a Child Penalty in the Long Run?” by...

LessWrong (30+ Karma) ·

2024/05/17

This is a link post.A couple of weeks ago three European economists published this paper studying the female income penalty after childbirth. The surprising headline result: there is no penalty. Setting and MethodologyThe paper uses Danish data that tracks IVF treatments as well as a bunch of demographic factors and economic outcomes over 25 years. Lundborg et al identify the causal effect of childbirth on female income using the success or failure of the first attempt at IVF as an instrument for

en-gb

4:59

“DeepMind: Frontier Safety Framework” by Zach...

LessWrong (30+ Karma) ·

2024/05/17

This is a link post.DeepMind's RSP is here. Excerpt:Today, we are introducing our Frontier Safety Framework - a set of protocols for proactively identifying future AI capabilities that could cause severe harm and putting in place mechanisms to detect and mitigate them. Our Framework focuses on severe risks resulting from powerful capabilities at the model level, such as exceptional agency or sophisticated cyber capabilities. It is designed to complement our alignment research, which trains models to

en-gb

29:06

“Language Models Model Us” by eggsyntax

LessWrong (30+ Karma) ·

2024/05/17

Produced as part of the MATS Winter 2023-4 program, under the mentorship of @Jessica RumbelowOne-sentence summary: On a dataset of human-written essays, we find that gpt-3.5-turbo can accurately infer demographic information about the authors from just the essay text, and suspect it's inferring much more. Introduction. Every time we sit down in front of an LLM like GPT-4, it starts with a blank slate. It knows nothing[1] about who we are, other than what it knows about users in general. But with

en-gb

5:17

“Towards Guaranteed Safe AI: A Framework for Ensuring...

LessWrong (30+ Karma) ·

2024/05/17

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.I want to draw attention to a new paper, written by myself, David "davidad" Dalrymple, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark Barrett, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing, and Joshua Tenenbaum.In this paper we introduce the concept of "guaranteed safe (GS) AI", which is a broad research

en-gb

9:41

“Identifying Functionally Important Features with...

LessWrong (30+ Karma) ·

2024/05/17

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.A short summary of the paper is presented below.This work was produced by Apollo Research in collaboration with Jordan Taylor (MATS + University of Queensland) .TL;DR: We propose end-to-end (e2e) sparse dictionary learning, a method for training SAEs that ensures the features learned are functionally important by minimizing the KL divergence between the output distributions of the original model

en-gb

3:22

“AISafety.com – Resources for AI Safety” by Søren...

LessWrong (30+ Karma) ·

2024/05/17

There are many resources for those who wish to contribute to AI Safety, such as courses, communities, projects, jobs, events and training programs, funders and organizations. However, we often hear from people that they have trouble finding the right resources. To address this, we've built AISafety.com as a central hub—a list-of-lists—where community members maintain and curate these resources to increase their visibility and accessibility.In addition to presenting resources, the website is

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Reflections

Livewire Labs

ForGeeks

ForGeeks

FOX on Tech

FOX News Podcasts

The Linux Cast

The Linux Cast