PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

45:22

“A ‘Bitter Lesson’ Approach to Aligning AGI and ASI”...

LessWrong (30+ Karma) ·

2024/07/07

TL;DR: I discuss the challenge of aligning AGI/ASI, and outline an extremely simple approach to aligning an LLM: train entirely on a synthetic dataset that always shows the AI acting aligned (even when the humans behave badly), and use a conditional training/inference-time technique to lock the LLM into the AI role.Epistemic status: To me, this looks like an obvious thing to try. It's conceptually very simple: a vast amount of work is required to actually create the synthetic dataset, but the great

en-gb

7:43

“An AI Manhattan Project is Not Inevitable” by...

LessWrong (30+ Karma) ·

2024/07/07

This is a link post.Early last month, Leopold Aschenbrenner released a long essay and podcast outlining his projections for the future of AI. Both of these sources are full of interesting arguments and evidence, for a comprehensive summary see Zvi's post here. Rather than going point by point I will instead accept the major premises of Leopold's essay but contest some of his conclusions.So what are the major premises of his piece? There will be several orders of magnitude increase in investment into

en-gb

12:30

“AI Alignment Research Engineer Accelerator (ARENA):...

LessWrong (30+ Karma) ·

2024/07/07

TL;DRWe are excited to announce the fourth iteration of ARENA (Alignment Research Engineer Accelerator), a 4-5 week ML bootcamp with a focus on AI safety! ARENA's mission is to provide talented individuals with the skills, tools, and environment necessary for upskilling in ML engineering, for the purpose of contributing directly to AI alignment in technical roles. ARENA will be running in-person from LISA from 2nd September - 4th October (the first week is an optional review of the fundamentals of

en-gb

5:32

“Scalable oversight as a quantitative rather than...

LessWrong (30+ Karma) ·

2024/07/06

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.[Many of these ideas were developed in conversation with Ryan Greenblatt and Ansh Radhakrishnan; a lot of this isn’t original but I haven’t seen it written up]A lot of the time when people talk about scalable oversight, which I'll define for the moment as "the problem of developing techniques that allow you to assess whether an action taken or proposed by an AI is good (oversee it), in a way that works even if the

en-gb

15:20

“Finding the Wisdom to Build Safe AI” by Gordon...

LessWrong (30+ Karma) ·

2024/07/06

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.We may soon build superintelligent AI. Such AI poses an existential threat to humanity, and all life on Earth, if it is not aligned with our flourishing. Aligning superintelligent AI is likely to be difficult because smarts and values are mostly orthogonal and because Goodhart effects are robust, so we can neither rely on AI to naturally decide to be safe on its own nor can we expect to train it to stay safe. We

en-gb

6:11

“Musings on LLM Scale (Jul 2024)” by Vladimir_Nesov

LessWrong (30+ Karma) ·

2024/07/06

In a recent interview, Dario Amodei claimed that cost of training is (starting with models already available) Right now, $100 million. There are models in training today that are more like a $1 billion. I think if we go to $10 or a $100 billion, and I think that will happen in 2025-2026, maybe 2027, ... (Epistemic status: Fermi estimates, 8 is approximately 10 which is greater than 9.) Assuming $40,000 per H100 and associated infrastructure in a datacenter, $1 billion gives 25K H100s, which

en-gb

25:10

“[Interim research report] Activation plateaus &...

LessWrong (30+ Karma) ·

2024/07/05

This part-report / part-proposal describes ongoing research, but I'd like to share early results for feedback. I am especially interested in any comment finding mistakes or trivial explanations for these results. I will work on this proposal with a LASR Labs team over the next 3 months. If you are working (or want to work) on something similar I would love to chat! Experiments and write-up by Stefan, with substantial inspiration and advice from Jake (who doesn’t necessarily endorse every sloppy

en-gb

12:19

“Doomsday Argument and the False Dilemma of Anthropic...

LessWrong (30+ Karma) ·

2024/07/05

Doomsday InferenceCan we use probability theory to estimate how many people there will be throughout the whole human history? Sure. We can build a probability model, that takes into account birth rates, possible existential hazards, ways to mitigate them and multiple other factors. Such models tend not to be very precise, so we would have pretty large confidence intervals but, we would still have some estimate.Hmm... this sounds like a lot of work for not much of a result. Can't we just use the

en-gb

69:36

“AI #71: Farewell to Chevron” by Zvi

LessWrong (30+ Karma) ·

2024/07/04

Chevron deference is no more. How will this impact AI regulation?The obvious answer is it is now much harder for us to ‘muddle through via existing laws and regulations until we learn more,’ because the court narrowed our affordances to do that. And similarly, if and when Congress does pass bills regulating AI, they are going to need to ‘lock in’ more decisions and grant more explicit authority, to avoid court challenges. The argument against state regulations is similarly weaker now. Similar

en-gb

6:02

“Static Analysis As A Lifestyle” by adamShimi

LessWrong (30+ Karma) ·

2024/07/04

This is a link post.I’ve been watching French Top Chef (the best Top Chef, fight me) with my wife again, and I’m always impressed by how often the mentoring chefs, all with multiple michelin stars and years of experience, can just guess that a dish will work or that it will be missing something.By far, whenever a chef points to an error (not a risk, an error), it's then immediately validated experimentally: either the candidate corrected it and the jury comments positively on that aspect of the

en-gb

6:09

“When Are Results from Computational Complexity Not...

LessWrong (30+ Karma) ·

2024/07/04

Tl;dr, While an algorithm's computational complexity may be exponential in general (worst-case), it is often possible to stratify its input via some dimension _k_ that makes it polynomial for a fixed _k_, and only exponential in _k_. Conceptually, this quantity captures the core aspect of a problem's structure that makes specific instances of it 'harder' than others, often with intuitive interpretations. Example: Bayesian

en-gb

11:39

“Introduction to French AI Policy” by Lucie Philippon

LessWrong (30+ Karma) ·

2024/07/04

This post was written as part of the AI Governance Fundamentals course by BlueDot. I thank Charles Beasley and the students from my cohort for their feedback and encouragements.Disclaimer: The French policy landscape is in rapid flux, after president Macron called for a snap election on 1st and 7th July. The situation is still unfolding, and the state of French AI policy may be significantly altered.At various AI governance events, I noticed that most people had a very unclear vision of what was

en-gb

3:33

“Open Sourcing Metaculus” by ChristianWilliams

LessWrong (30+ Karma) ·

2024/07/03

This is a link post. This is a linkpost for an announcement by Metaculus CEO, Deger Turan, originally published June 5th, 2024. In addition to spreading the word here, we are interested to gather feedback and ideas from the EA community. We make reasoning and coordination tools for the public benefit, and that requires the trust and participation of the wider forecasting community. That's why we’re going open source. GitHub issues, pull requests, and community-sourced forecasts will dictate much of

en-gb

12:44

“80,000 hours should remove OpenAI from the Job Board...

LessWrong (30+ Karma) ·

2024/07/03

I haven't shared this post with other relevant parties – my experience has been that private discussion of this sort of thing is more paralyzing than helpful. I might change my mind in the resulting discussion, but, I prefer that discussion to be public. I think 80,000 hours should remove OpenAI from its job board, and similar EA job placement services should do the same. (I personally believe 80k shouldn't advertise Anthropic jobs either, but I think the case for that is somewhat less clear) I

en-gb

15:04

“3C’s: A Recipe For Mathing Concepts” by...

LessWrong (30+ Karma) ·

2024/07/03

Opening Example: TeleologyWhen people say “the heart's purpose is to pump blood” or “a pencil's function is to write”, what does that mean physically? What are “purpose” or “function”, not merely in intuitive terms, but in terms of math and physics? That's the core question of what philosophers call teleology - the study of “telos”, i.e. purpose or function or goal.This post is about a particular way of approaching conceptual/philosophical questions, especially for finding “True Names” - i.e

en-gb

5:07

“List of Collective Intelligence Projects” by Chipmonk

LessWrong (30+ Karma) ·

2024/07/03

This is a link post.During the last Foresight Intelligent Cooperation Workshop I got very curious about what collective intelligence tools currently exist. A list: Pol.is: "Input Crowd, Output Meaning" InspiredTwitter/X community notesPeople: Colin Megill, et al.Collective Intelligence Project vibe: democratic AI,“How AI and Democracy Can Fix Each Other”People: Divya Siddharth, Saffron Huang, et al.AI Objectives Institute Talk to the City: "an open-source LLM interface for improving collective

en-gb

45:20

“Economics Roundup #2” by Zvi

LessWrong (30+ Karma) ·

2024/07/03

Previously: Economics Roundup #1Let's take advantage of the normality while we have it. In all senses. Insane Tax Proposals There is Trump's proposal to replace income taxes with tariffs, but he is not alone. So here is your periodic reminder, since this is not actually new at core: Biden's proposed budgets include completely insane tax regimes that would cripple our economic dynamism and growth if enacted. As in for high net worth individuals, taking unrealized capital gains at

en-gb

18:48

“OthelloGPT learned a bag of heuristics” by jylin04,...

LessWrong (30+ Karma) ·

2024/07/02

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.Work performed as a part of Neel Nanda's MATS 6.0 (Summer 2024) training program. TLDRThis is an interim report on reverse-engineering Othello-GPT, an 8-layer transformer trained to take sequences of Othello moves and predict legal moves. We find evidence that Othello-GPT learns to compute the board state using many independent decision rules that are localized to small parts of the board. Though we cannot rule out

en-gb

23:43

“An AI Race With China Can Be Better Than Not Racing”...

LessWrong (30+ Karma) ·

2024/07/02

Frustrated by all your bad takes, I write a Monte-Carlo analysis of whether a transformative-AI-race between the PRC and the USA would be good. To my surprise, I find that it is better than not racing. Advocating for an international project to build TAI instead of racing turns out to be good if the probability of such advocacy succeeding is ≥20%. A common scheme for a conversation about pausing the development of transformative AI goes like this: Abdullah: "I think we should pause the development

en-gb

30:22

“Decomposing the QK circuit with Bilinear Sparse...

LessWrong (30+ Karma) ·

2024/07/02

This work was produced as part of Lee Sharkey's stream in the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort Intro and MotivationSparse dictionary learning (SDL) has attracted a lot of attention recently as a method for interpreting transformer activations. They demonstrate that model activations can often be explained using a sparsely-activating, overcomplete set of human-interpretable directions. However, despite its success for explaining many components, applying SDL to

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Reflections

Livewire Labs

ForGeeks

ForGeeks

FOX on Tech

FOX News Podcasts

The Linux Cast

The Linux Cast