PODCAST

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

technology philosophy

All Episodes

2:27

“What distinguishes ‘early’, ‘mid’ and ‘end’ games?”...

LessWrong (30+ Karma) ·

2024/06/21

Recently William_S posted:In my mental model, we're still in the mid-game, not yet in the end-game.I replied:A thing I've been thinking about lately is "what does it mean to shift from the early-to-mid-to-late game". In strategy board games, there's an explicit shift from "early game, it's worth spending the effort to build a longterm engine. At some point, you want to start spending your resources on victory points." And a lens I'm thinking through is "how long does it keep making sense to invest

en-gb

39:49

“Attention Output SAEs Improve Circuit Analysis” by...

LessWrong (30+ Karma) ·

2024/06/21

This is the final post of our Alignment Forum sequence produced as part of the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort. Executive Summary In a previous post we trainedAttention Output SAEs onevery layer of GPT-2 Small.Following that work, we wanted to stress-test that Attention SAEs were genuinely helpful for circuit analysis research. This would both validate SAEs as a useful tool, and provide evidence that they are identifying the real variables of the model's

en-gb

17:57

“Connecting the Dots: LLMs can Infer & Verbalize...

LessWrong (30+ Karma) ·

2024/06/21

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.TL;DR: We published a new paper on out-of-context reasoning in LLMs. We show that LLMs can infer latent information from training data and use this information for downstream tasks, without any in-context learning or CoT. For instance, we finetune GPT-3.5 on pairs (x,f(x)) for some unknown function f. We find that the LLM can (a) define f in Python, (b) invert f, (c) compose f with other

en-gb

9:45

“Interpreting and Steering Features in Images” by...

LessWrong (30+ Karma) ·

2024/06/21

We trained a SAE to find sparse features in image embeddings. We found many meaningful, interpretable, and steerable features. We find that steering image diffusion works surprisingly well and yields predictable and high-quality generations.You can see the feature library here. We also have an intervention playground you can try. Key Results We can extract interpretable features from CLIP image embeddings.We observe a diverse set of features, e.g. golden retrievers, there being two of something,

en-gb

1:06

“Case studies on social-welfare-based standards in...

LessWrong (30+ Karma) ·

2024/06/21

Last year, I posted a call for case studies on social-welfare-based standards for companies and products (including standards imposed by regulation). The goal was to gain general context on standards to inform work on possible standards and/or regulation for AI. This resulted[1] in several dozen case studies that I found informative and helpful. I've been hoping to write up my reflections after reading them all, but it's taken me long enough to get to this that for now I am just publishing a public

en-gb

0:57

“Claude 3.5 Sonnet” by Zach Stein-Perlman

LessWrong (30+ Karma) ·

2024/06/20

This is a link post.we’ll be releasing Claude 3.5 Haiku and Claude 3.5 Opus later this year.They made a mini model card. Notably:The UK AISI also conducted pre-deployment testing of a near-final model, and shared their results with the US AI Safety Institute . . . . Additionally, METR did an initial exploration of the model's autonomy-relevant capabilities. ---First published: June 20th, 2024 Source:

en-gb

3:45

“Jailbreak steering generalization” by Nina Rimsky,...

LessWrong (30+ Karma) ·

2024/06/20

This is a link post.This work was performed as part of SPARWe use activation steering (Turner et al., 2023; Rimsky et al., 2023) to investigate whether different types of jailbreaks operate via similar internal mechanisms. We find preliminary evidence that they may.Our analysis includes a wide range of jailbreaks such as harmful prompts developed in Wei et al. 2024, the universal jailbreak in Zou et al. (2023b), and the payload split jailbreak in Kang et al. (2023). For all our experiments we use

en-gb

96:18

“AI #69: Nice” by Zvi

LessWrong (30+ Karma) ·

2024/06/20

Nice job breaking it, hero, unfortunately. Ilya Sutskever, despite what I sincerely believe are the best of intentions, has decided to be the latest to do The Worst Possible Thing, founding a new AI company explicitly looking to build ASI (superintelligence). The twists are zero products with a ‘cracked’ small team, which I suppose is an improvement, and calling it Safe Superintelligence, which I do not suppose is an improvement. How is he going to make it safe? His statements tell us nothing

en-gb

4:21

“Actually, Power Plants May Be an AI Training...

LessWrong (30+ Karma) ·

2024/06/20

There have been presistent rumors that electricity generation was somehow bottlenecking new data centers. This claim was recently repeated by Donald Trump, who implied that San Francisco donors requested the construction of new power plants for powering new AI data centers in the US. While this may sound unlikely, my research suggests it's actually quite plausible.Electricity generation, capacity, and sales in the United States - U.S. Energy Information Administration (EIA)Electricity generation,

en-gb

1:53

“Beyond the Board: Exploring AI Robustness Through...

LessWrong (30+ Karma) ·

2024/06/19

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.Last year, we showed that supposedly superhuman Go AIs can be beaten by human amateurs playing specific “cyclic” patterns on the board. Vulnerabilities have previously been observed in a wide variety of sub- or near-human AI systems, but this result demonstrates that even far superhuman AI systems can fail catastrophically in surprising ways. This lack of robustness poses a critical challenge

en-gb

19:13

“Surviving Seveneves” by Yair Halberstadt

LessWrong (30+ Karma) ·

2024/06/19

Contains spoilers for the first couple of chapters of SevenevesHighly speculative on my part, I know very little about most of these topicsIn Seveneves Neal Stephenson does the classic sci-fi trick of assuming that exactly one thing in the universe is different, and seeing where that takes us. In his case that one thing is the moon has somehow exploded.And where that takes us is the complete destruction of the earth. As the initially huge chunks of moon rock hit into each other they break into

en-gb

2:07

“Ilya Sutskever created a new AGI startup” by harfe

LessWrong (30+ Karma) ·

2024/06/19

This is a link post.[copy of the whole text of the announcement]Safe Superintelligence Inc. Superintelligence is within reach. Building safe superintelligence (SSI) is the most important technical problem of our time. We have started the world's first straight-shot SSI lab, with one goal and one product: a safe superintelligence. It's called Safe Superintelligence Inc. SSI is our mission, our name, and our entire product roadmap, because it is our sole focus. Our team, investors, and business

en-gb

10:40

“Suffering Is Not Pain” by jbkjr

LessWrong (30+ Karma) ·

2024/06/19

This is a link post.“Pain is inevitable; suffering is optional.”The motivation of this post is to address the persistent conflation between suffering and pain I have observed from members of the EA community, even amongst those who purport to be “suffering-focused” in their ethical motivations. In order to best address the problem of suffering, it is necessary to be clear about the difference between suffering and mere pain or ordinary displeasure. The parable of the second arrowIn the Buddhist

en-gb

64:23

“Loving a world you don’t trust” by Joe Carlsmith

LessWrong (30+ Karma) ·

2024/06/18

(Cross-posted from my website. Audio version here, or search for "Joe Carlsmith Audio" on your podcast app.)This is the final essay in a series that I'm calling "Otherness and control in the age of AGI." I'm hoping that the individual essays can be read fairly well on their own, but see here for a brief summary of the series as a whole. There's also a PDF of the whole series here. Warning: spoilers for Angels in America; and moderate spoilers for Harry Potter and the Methods of Rationality.) "I

en-gb

63:54

“Loving a world you don’t trust” by Joe Carlsmith

LessWrong (30+ Karma) ·

2024/06/18

en-gb

3:00

“Boycott OpenAI” by PeterMcCluskey

LessWrong (30+ Karma) ·

2024/06/18

This is a link post.I have canceled my OpenAI subscription in protest over OpenAI's lack of ethics. In particular, I object to: threats to confiscate departing employees' equity unless those employees signed a life-long non-disparagement contract Sam Altman's pattern of lying about important topics I'm trying to hold AI companies to higher standards than I use for typical companies, due to the risk that AI companies will exert unusual power. A boycott of OpenAI subscriptions seems unlikely to

en-gb

16:11

“On DeepMind’s Frontier Safety Framework” by Zvi

LessWrong (30+ Karma) ·

2024/06/18

On DeepMind's Frontier Safety Framework Previously: On OpenAI's Preparedness Framework, On RSPs. The First Two Frameworks To first update on Anthropic and OpenAI's situation here: Anthropic's RSP continues to miss the definitions of the all-important later levels, in addition to other issues, although it is otherwise promising. It has now been a number of months, and it is starting to be concerning that nothing has changed. They are due for an update. OpenAI also has not updated its

en-gb

11:34

“D&D.Sci Alchemy: Archmage Anachronos and the Supply...

LessWrong (30+ Karma) ·

2024/06/18

This is a follow-up to last week's D&D.Sci scenario: if you intend to play that, and haven't done so yet, you should do so now before spoiling yourself.There is a web interactive here you can use to test your answer, and generation code available here if you're interested, or you can read on for the ruleset and scores. RULESETThere are two steps to brewing a potion: STEP 1: MAGICAL POTENCYAny ingredient that doesn't exist in the mundane world is Magical, while any ingredient that exists in the

en-gb

3:01

“Fat Tails Discourage Compromise” by niplav

LessWrong (30+ Karma) ·

2024/06/17

Say that we have a set of options, such as (for example) wild animal welfare interventions. Say also that you have two axes along which you can score those interventions: popularity (how much people will like your intervention) and effectiveness (how much the intervention actually helps wild animals). Assume that we (for some reason) can't convert between and compare those two properties. Should you then pick an intervention that is a compromise on the two axes—that is, it scores decently well on

en-gb

15:38

“Sycophancy to subterfuge: Investigating reward...

LessWrong (30+ Karma) ·

2024/06/17

Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.New Anthropic model organisms research paper led by Carson Denison from the Alignment Stress-Testing Team demonstrating that large language models can generalize zero-shot from simple reward-hacks (sycophancy) to more complex reward tampering (subterfuge). Our results suggest that accidentally incentivizing simple reward-hacks such as sycophancy can have dramatic and very difficult to reverse

en-gb

4,491 results

LessWrong (30+ Karma)

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Reflections

Livewire Labs

ForGeeks

ForGeeks

FOX on Tech

FOX News Podcasts

The Linux Cast

The Linux Cast