[Biden] What's up, Barack?
[Trump] Hi there, Sleepy Joe.
[Obama] For our viewers, let's explain what we're doing.
[Obama] Eliezer Yudkowsky recently wrote a Time article about how AI will kill everyone unless we shut down the expensive training runs.
[Obama] I read this article, and though I found it hyperbolic, I thought it made concerning points.
[Obama] After consulting more with experts in and around the field, I concluded that Eliezer's stance is reasonable.
[Obama] I agree.
[Obama] We should shut down AI development.
[Trump] Bigly.
[Trump] I mean, I thought this sounded kind of bogus, to be honest, and I never thought I'd be agreeing with Barack Hussein Obama, but this issue is important.
[Biden] I also...
[Trump] Shut up, Joe.
[Trump] No one cares.
[Biden] Will you shut up, man?
[Biden] This is so unpresidential.
[Obama] Let's put aside these petty disputes.
[Obama] Remember what we're here to do.
[Biden] Fine.
[Biden] As I was saying, I also found this article concerning.
[Obama] So, we asked to see what we can do.
[Obama] 80,000 Hours said that direct work probably isn't our comparative advantage, but apparently there aren't enough grant evaluators.
[Obama] Since we aren't technical researchers ourselves, 80,000 Hours concluded that our time is not valuable, and they made us evaluate research areas.
[Trump] Can you get to the point, Obama?
[Obama] We're ranking alignment agendas today in a tier list.
[Obama] We've all done a lot of reading, so let's share our thoughts.
[Obama] Let's start off with something I think we can agree on, mechanistic interpretability.
[Obama] Easy S tier?
[Trump] Are you another interpretability simp, Barack?
[Trump] Ever since King Eliezer started drooling over it, people just trip over themselves.
[Trump] You wouldn't believe it.
[Trump] They absolutely trip over themselves to show how much they love interpretability.
[Obama] I notice you haven't answered my question.
[Trump] Can we be specific, at least?
[Trump] There's lots of kinds of interpretability work.
[Obama] Let's start off with Anthropic's agenda, tackling superposition.
[Biden] I think they do good, decent work.
[Biden] God, I wish they'd pump the brakes on their capabilities progress, though.
[Trump] Great idea, Joe.
[Trump] Amazing.
[Trump] That way we can be caught with our pants down when AGI comes, and we don't know shit about how to apply alignment techniques at scale.
[Biden] Aren't you in court right now because you got caught with your pants down?
[Biden] Don't you have other things to be doing, Donald?
[Obama] Seriously, Joe?
[Obama] Anyways, I think their superposition work has a solid chance of tackling a core interpretability challenge.
[Obama] We can visualize attention patterns all day long, but to truly understand realistic transformer circuits, we've got to tackle the MLP layers.
[Obama] After reading the toy models of superposition posts, I'm convinced that superposition is a substantial roadblock.
[Biden] Yeah, sure, but I'm not too sold.
[Biden] They do a good job, but you can never be sure with these things.
[Biden] Appealingsounding alignment theorizing often seems useless in retrospect.
[Biden] I vote for A tier.
[Trump] You're one to talk about useless theory, Joe.
[Trump] You're the one who keeps Discord messaging me these insane theories from the most abstract alignment agendas.
[Trump] Like I really had no idea you could become that detached from reality.
[Trump] I really didn't.
[Trump] At least Anthropic runs experiments and interprets real models.
[Trump] I mean, to be honest, guys, I think this superposition stuff isn't going to save us.
[Trump] We solve superposition—fine.
[Trump] We can now understand whatever tiny network their interp team is messing around with next year.
[Obama] Donald, I think many researchers would disagree with you.
[Obama] I've spoken to a few interpretability researchers who consider superposition to be a more fundamental challenge than scaling interpretability to realistic models.
[Trump] You know what?
[Trump] Okay, I'll go along with a high tier ranking for this one.
[Trump] Superposition is interesting and concrete and at least has a shot of being important, you know?
[Obama] I'm putting this in A tier.
[Obama] Donald, you made a good point earlier.
[Obama] There are several kinds of interpretability.
[Obama] Let's talk about circuits interpretability now.
[Biden] Why is that different?
[Obama] It's true that Anthropic superposition agenda aims to enable circuit research by demystifying MLP layers.
[Obama] However, solving superposition is its own problem and may yield benefits beyond just improving our ability to interpret circuits.
[Trump] Circuit interpretability, for real?
[Trump] Listen, guys, I know we like huffing hopium over here, but I just can't believe it.
[Trump] I don't know what era these guys are living in.
[Trump] Do they think GPT-4 is made out of, like, 10 circuits?
[Trump] How are we possibly going to understand what any realistic model is doing?
[Obama] Donald, I think the hope is to learn general facts about how models learn to make decisions to get faster at finding circuits.
[Obama] Maybe we can even automate it.
[Obama] I think that's a reasonable possibility and very much on the table.
[Trump] Oh, yeah, bigly.
[Trump] That's bigly going to happen.
[Trump] We'll just automate it.
[Trump] What do we have, like, four years left until this A.I. absolutely fucks us?
[Trump] How are we going to automate this in time?
[Obama] We can talk about causal scrubbing and other ideas for automating alignment research, but the real answer is that I don't know.
[Obama] The situation looks pretty grim.
[Obama] How are we going to do anything in time?
[Biden] We should judge agendas by their relative merits, obviously.
[Biden] Seems like we're leaning towards B tier.
[Biden] That's hogwash.
[Biden] Bigly Don pointed out that we aren't fast at it right now.
[Biden] Fine. But circuits interpretability is our source of ground truth on how A.I. cognition develops.
[Biden] That's like saying, uh, I'm not good at breathing right now, so breathing isn't important.
[Biden] I think it's true that we're slow at interpretability and also true that it's extremely important to be good at it.
[Trump] I must admit, I got to admit it, I wasn't thinking too much about relative merits, you know?
[Trump] Sometimes I do get a bit impatient.
[Trump] I really do.
[Trump] But I swear to God, we got to get faster.
[Trump] We need to get faster and better at this.
[Trump] It's no joke.
[Trump] Did you hear that Neel Nanda guy found some linear representations in Othello GPT in a single weekend?
[Trump] Why does everything else in real life take like a billion years?
[Trump] We got to get faster.
[Biden] Why not show them how it's done, Donald?
[Trump] I'm planning on it right after this video, if we ever finish this up.
[Trump] I'm going to figure out that algebraic value editing thing that Team Shard talked about a few weeks ago.
[Trump] God, finally, Team Shard has an exciting idea, and then they take fucking forever to release even the smallest piece of information about it.
[Obama] Guys, back on topic, please.
[Trump] Yeah, whatever, either B or A tier is fine with me.
[Biden] It belongs in A tier.
[Obama] I'm putting it in A tier.
[Obama] I think this is a promising direction, but the rubber isn't hitting the road in quite the right way.
[Obama] I applaud the existing talent and effort being poured into the field, and I also think that there's a lot of room for improvement and growth.
[Obama] Let's talk about the Machine Intelligence Research Institute's Agent Foundations agenda.
[Obama] This agenda wants to better understand realistic reasoning systems which don't have infinite time to think about how the world works.
[Trump] Instant F tier content, I'll tell you right now.
[Biden] I've about had it with you and your shit-tier takes, Donald.
[Biden] Agent Foundations tackles key confusions about what agents are and what alignment even means.
[Trump] That's what you like to tell yourself, huh?
[Biden] While it's good to understand what modern systems are doing, man, I sure get it, how it can feel good to make a smart language model output helpful things.
[Biden] I think that's not tackling the brunt of the problem.
[Biden] To do that, we have to really know what the hell we're doing.
[Trump] Yeah, maybe we do need to really know what we're doing, but let's be clear.
[Trump] Let's be absolutely clear here.
[Trump] I don't think Agent Foundations, the actual agenda we're discussing, Sleepy Joe, this agenda doesn't know what it's talking about.
[Trump] You know, I agree.
[Trump] I'm one of the people who most thinks we should know what we're doing here.
[Trump] We should slow down and take our time.
[Trump] We should do that in ideal alternate society, utopia, fantasy land.
[Trump] But back in reality, this agenda has made zero progress, zero that helps Trump Tower not get consumed by AI.
[Obama] Donald, you can be a little hyperbolic, and it detracts from the points you're trying to make.
[Obama] Functional decision theory was a real insight.
[Obama] I've definitely appreciated having those concepts in my mental tool belt, and logical induction was, you know, pretty cool.
[Trump] Cool?
[Trump] Sure, it's cool.
[Trump] But does it matter?
[Trump] We've got to stop fucking around.
[Trump] We really do.
[Trump] This is just absolutely nuts.
[Obama] I'm willing to entertain your critiques, Donald, but you should be more courteous.
[Obama] There are people, smart people, trying to help make a better future for us here.
[Trump] Oh, I'm one of the most grateful people.
[Trump] You wouldn't even believe it.
[Trump] I mean, these researchers could be lazing around and playing video games all day long.
[Trump] I'm grateful.
[Trump] I really am.
[Trump] I'm grateful to find people pitching in.
[Trump] People ask me all the time these days, how do you feel about alignment researchers?
[Trump] And I tell them, they're heroes.
[Trump] They really are.
[Trump] At worst, they're doing something about the problem.
[Trump] At best, they're saving us from total destruction like you wouldn't believe.
[Biden] It's good of you to say that, Donald, and I agree.
[Biden] I thank each and every person working honestly, diligently to make our future better and safer.
[Biden] And that includes alignment researchers, especially given the stakes of the challenge.
[Biden] And I thank them from the bottom of my heart.
[Trump] And—Agent Foundations is worthless.
[Biden] And you're delusional.
[Biden] First, a lot of their work has been secret so they don't boost AI capabilities.
[Trump] And so that they don't boost AI alignment either, apparently.
[Biden] Second, rather than throwing a bunch of hacky, a bunch of questionable ideas and schemes at the wall and hoping the problem goes away, Agent Foundations looks for understanding, for meaning.
[Biden] When you find yourself stuck in a deeply confusing problem, the first step is not to flail around and run experiments which use ideas you already understand.
[Biden] It's to look sideways and see if you can't come up with some new ideas.
[Trump] This is so vague.
[Trump] This is unbelievably vague.
[Biden] And Agent Foundations actually tackles these issues.
[Trump] And how have they done at that, Joseph?
[Obama] I'm worried you're going to start talking past each other.
[Obama] I think we should split this out into two propositions.
[Obama] First, Agent Foundations is on the right track to eventually contribute to an alignment solution.
[Obama] Second, and this is what matters for the tier ranking, Agent Foundations is good research to do right now, given the real-world situation we face.
[Biden] Thanks, Barack.
[Biden] I think that this agenda is on the right track.
[Biden] And I think it's a decent, but not perfect, not at all perfect, way to allocate additional research.
[Biden] I think Agent Foundations is on the right track, is on a solid road.
[Biden] It's not enough to just do interpretability and understand what gradient descent is doing, what its inductive biases are.
[Biden] You've got to know what kind of mind you want to get out the other end.
[Biden] You've got to understand how you want your superintelligence to reason, at least at a high level.
[Trump] Oh, yeah, let's just build an omnipotent god.
[Trump] Let's build a singleton.
[Trump] I can't imagine regretting that.
[Biden] Jesus, not this argument again.
[Biden] Anyways, Agent Foundations is an answer to the question, what kind of mind do we want to build anyways?
[Biden] And they've made some progress.
[Biden] Agent Foundations gave us logical induction.
[Trump] I mean, I think they're lost in fantasy land with strange philosophical assumptions.
[Trump] I mean, I tried arguing with one of the researchers, very smart guy, brilliant guy.
[Trump] I tried arguing with him to explain the mistakes, but they just can't see it.
[Obama] Here's what I think.
[Obama] I agree with Donald that publicly available research in Agent Foundations has been practically useless.
[Obama] I think I'm more open to investing in a wide range of alignment worldviews, even if I find them unconvincing.
[Obama] That said, due to how little time I think we have before AGI, I'm leaning towards D-tier at best.
[Trump] Put it in F-tier.
[Trump] You know it belongs, Barack.
[Biden] On my own, I'd give it B or A-tier, but I guess I'll settle for C-tier.
[Trump] I can't believe this guy.
[Trump] I really can't.
[Trump] Can you believe he's supposed to be president?
[Trump] We're staring down the barrel of a giant gun, a huge gun, pointed right at our beautiful nation, and Joe wants to encourage people.
[Trump] He really does.
[Trump] He wants to encourage them to work on an agenda that is yielded like the tiniest piece of progress imaginable, if that.
[Obama] You know what?
[Obama] I still think you're being obnoxious and hyperbolic, Donald, but I agree.
[Obama] This one goes in F-tier.
[Obama] Let me be clear.
[Obama] I applaud the researchers themselves and don't mean to mock or belittle anyone for pursuing an agenda they find promising, but part of being a leader is to speak frankly, even when it's hard.
[Trump] You know, maybe I was wrong about you, Barack.
[Trump] Once I demolish Joe in 2024, I'll need a vice president, and...
[Biden] You're a child, Donald.
[Obama] Let's consider Conjecture's research agenda.
[Biden] What is it?
[Trump] I don't know.
[Obama] So where are we going to put it?
[Trump] S-tier.
[Trump] The S is for socio-hazard.
[Obama + Biden] That's stupid.
[Trump] Wasn't the Conjecture agenda the one trying to, you know, derive some alternative to deep learning?
[Obama] I think so.
[Trump] That's stupid.
[Trump] That's so stupid you wouldn't believe it.
[Trump] These so-called LessWrong folk love talking about the planning fallacy, and then they're like, oh, yeah, let's just derive a new form of safe, transparent optimization, which is AGI competitive within four years.
[Trump] Delusional.
[Trump] Man up and figure out how to make models do what you want.
[Trump] Instead of pretending to believe in some fairy dust plan, that requires skills we won't have in time, which we really won't have in time.
[Obama] As an aside, I think Conjecture considers itself mostly distinct from LessWrong culture.
[Biden] Another red flag, if you're on a date and they don't have a LessWrong account, you know, you know they're trouble.
[Obama] Joe, when would any of us possibly benefit from that advice?
[Biden] Uh, what's the next agenda?
[Obama] We still haven't ranked Conjecture?
[Trump] I don't know.
[Trump] D-tier?
[Trump] This agenda sounds too vague to me.
[Trump] I mean, I know what it takes to make a deal go through, to make a plan go through, and their plan sounds insane to me.
[Trump] Believe me, I'm great at evaluating plans, and this plan doesn't make sense.
[Trump] At least what I've heard of it.
[Trump] Maybe it'd make more sense if Conjecture didn't consider the smallest possible English sentences to be info hazards.
[Trump] I mean, what a joke, guys.
[Trump] I mean, look, I respect them for trying to stop something bad from happening, but this "infosecurity" thing is going too far.
[Trump] I have news.
[Trump] There are lots of capabilities advances every day.
[Trump] There are thousands, absolutely thousands of people.
[Trump] You wouldn't believe how many people are trying to build a giant bomb that will kill them and their own family.
[Trump] These capabilities researchers are maniacs.
[Trump] The pedal is to the metal.
[Trump] And all these alignment researchers are so afraid of people getting mad at them, of people getting mad at them for so-called capabilities advances, they're becoming paralyzed, absolutely paralyzed by fear, by indecision.
[Trump] You hate to see it.
[Trump] Just share your goddamn work.
[Trump] We have real work to do.
[Trump] We have some jobs to create, some alignment jobs, and I want to create them.
[Trump] And that can't happen if everyone's scared to death of sharing the smallest pieces of information about their work, which, like, barely anyone is paying attention to anyways, to be totally honest.
[Obama] Yeah, D-tier it is.
[Obama] Have either of you managed to understand Vanessa Kosoy's infrabayesianism proposal?
[Trump] Obviously not.
[Biden] No.
[Trump] You know, I don't think anyone understands that.
[Trump] I really don't.
[Trump] At Mar-a-Lago, some of my wealthiest friends, smartest friends, you wouldn't believe how sharp these people are.
[Trump] They would ask me for help.
[Trump] And out of everyone, besides the main researchers maybe, I know the most about this agenda, and I just don't understand it.
[Trump] I didn't know what to tell them.
[Trump] I really didn't.
[Trump] Vanessa, smart person, wonderful person, no idea what this is.
[Trump] Let's make a new tier.
[Trump] The WTF tier.
[Obama] Agreed.
[Obama] This is going in WTF tier.
[Trump] I want to talk about that stupid shard theory bullshit.
[Obama] I can't wait to be graced by your perspective.
[Trump] I mean, at first I was really glad to see it.
[Trump] I really was.
[Trump] They were calling out some of the lowest quality reasoning.
[Trump] They finally noticed how bad it was.
[Trump] They were absolutely demolishing that outer alignment stuff.
[Trump] I was happy, maybe for the first time in years.
[Trump] But then they started making a bunch of their own shit up.
[Trump] I couldn't believe it.
[Trump] I thought it was a joke.
[Trump] And I was right.
[Trump] Shard theory is a joke.
[Biden] Donald, I got to admit it.
[Biden] The shard theorists at least force their theory to make predictions.
[Biden] They seem to live in reality, more so than other theorists, I'll admit.
[Trump] That's a pretty low bar.
[Trump] This research field is cringe.
[Obama] Can we talk about the substance of the theory?
[Obama] I think there are two primary claims.
[Obama] First, that AI policies are better understood as modular shards of influence and desire, desires which vary by context, rather than a really smart agent with a fixed single goal.
[Obama] Second, that we should anchor to humans as the one existing example of general intelligence we've ever seen.
[Trump] Oh, AI will be like humans.
[Trump] Great.
[Trump] I'm sure that'll be extremely accurate.
[Biden] I think lots of folks really already agree that fixed single goals are unrealistic.
[Biden] This aspect feels less new, although I see the appeal.
[Trump] Let the hindsight bias begin.
[Trump] I mean, you can't even tell what you're thinking right now, Joe.
[Trump] So I don't know why you'd remember what you used to think.
[Obama] I think the maze interpretability work provided good evidence for the shard frame.
[Obama] You know, that maze-solving network sure seemed to have multiple contextual goals.
[Obama] The AI goes to the cheese when near the cheese.
[Obama] It goes to the corner when the cheese is hard to get to.
[Biden] I liked how they reached into that network and puppeteered it, dragged the AI wherever they wanted in the maze, like a dog on a leash.
[Trump] What the hell, Joe?
[Trump] Anyways, I'm worried that they're fiddling around with tiny networks which won't teach us anything important about these giant, these large, very large, unbelievably large language models.
[Obama] Didn't you read their recent work on controlling GPT-2?
[Trump] Cool trick.
[Trump] Don't get me wrong.
[Trump] Maybe promising.
[Trump] Some folks I know are very excited about that.
[Trump] But what the hell does GPT-2 have to do with shard theory?
[Obama] Apparently the technique was discovered very quickly using shard theory reasoning.
[Trump] Fine.
[Trump] I mean, I got to give the so-called shard theorists props.
[Trump] I was getting real fed up with them, but I'm proud now.
[Trump] I mean, the shard theory itself.
[Trump] Oh, my God, don't get me started on the theory.
[Trump] More armchair philosophical bullshit.
[Trump] Jesus, is it so hard to just pick up a damn model and see what it's actually doing?
[Trump] But then they did it.
[Trump] Maybe for the first time ever in this godforsaken research field, someone made predictions and then ran an experiment.
[Trump] The shard theorists did it.
[Trump] They really did.
[Trump] Incredible.
[Trump] Wow.
[Obama] The sad thing about you is that I can't even tell if you're joking.
[Biden] I think they're ignoring the hardest issues like reflection and just focusing on the easy experiments which we can measure today.
[Trump] I think that's called living in reality, Joseph.
[Trump] Instead of making up embarrassing arguments, some of the worst and most tenuous arguments, I mean, and I'm just trying to be frank, how many times do you have to shoot yourself in the face before you learn to run experiments?
[Biden] I agree.
[Biden] I mean, Anthropic.
[Biden] They're racing ahead so quickly, it's very irresponsible.
[Trump] LMAO.
[Trump] There you have it, folks.
[Trump] And this is Sleepy Joe at his best, his absolute best.
[Obama] Are you OK, Joe?
[Obama] We aren't talking about Anthropic.
[Obama] We're discussing shard theory.
[Biden] Right.
[Biden] Thanks.
[Biden] I find it appealing.
[Biden] The theory is nice.
[Trump] Theory is useless, but they got some results.
[Trump] I was talking with some folks, some of the richest businessmen and smartest intellectuals, and that GPT-2 work was just the talk of the town.
[Trump] God, I couldn't even order my food because the server asked me about that blog post and he wouldn't stop.
[Obama] Donald, stop telling me about the people you're talking to.
[Obama] How do you even have time to talk to that many people?
[Biden] I think this should go in A tier.
[Trump] Nice try.
[Trump] This is B tier at best.
[Trump] Let's see how their experimental results pan out.
[Obama] I think this goes in low A tier.
[Obama] There's more variance here than usual, mostly because it's unclear how many of their claims and intuitions will pan out.
[Obama] I find shard theory perspectives refreshing, really a different frame on the whole problem.
[Obama] But I'm concerned by lack of formality in the theoretical claims, and it's not clear how important the GPT-2 work will end up being.
[Biden] Let's talk about the eliciting latent knowledge agenda.
[Biden] After years of flailing around at specifying complex objectives, Paul Christiano decided to pose a simple problem.
[Biden] Get a smart AI to honestly report whether it thinks a diamond is still present in a room.
[Trump] Yawn.
[Biden] Donald, this problem might be hard because the AI might be thinking in alien ways, which are hard to supervise or understand, and maybe it's easier for the model to learn to just say whether a person would think the diamond's still there.
[Trump] Maybe it's easier, huh?
[Trump] Maybe, maybe, maybe.
[Trump] We're spending all this time and attention on more armchair theory, but look, guys, how do we know that this is even an issue?
[Obama] Seems good, you know, if we can get models to be truthful.
[Trump] I'm not critiquing that.
[Trump] I mean, that Collin Burns stuff was really good.
[Trump] I'm talking about these so-called theorists.
[Biden] You have anything more substantial to critique here, Donald?
[Trump] Getting them truthful, fine, but how are these researchers, you know, how are they actually approaching the problem?
[Trump] Worst-case analysis.
[Trump] More like worst methodology.
[Trump] What's the assumption again?
[Trump] More of that "minimal loss algorithms get picked" bullshit?
[Trump] This is bigly not true.
[Trump] I mean, we aren't even training these models to convergence.
[Trump] Just read a paper on scaling.
[Trump] You can look at knowledge distillation as well.
[Trump] I mean, folks come up to me all the time.
[Trump] Everyone knows this.
[Trump] Everyone's talking about how wrong that shit is.
[Trump] Great assumption.
[Trump] And look, we've got this enormous document.
[Trump] You wouldn't believe how long it is.
[Trump] Going into this absolutely insane theoretical depth and detail.
[Trump] Want to guess how many experiments there are?
[Biden] They're doing worst-case analysis, Donald.
[Obama] I think they were also, you know, stating a problem for the community to work on.
[Trump] You're so right.
[Trump] This must be the most concise LessWrong content.
[Trump] Stating a problem in a thousand goddamn pages.
[Obama] Let me be clear.
[Obama] This problem seems quite real.
[Obama] Larger models tend to be more sycophantic.
[Trump] Barack, I'm an expert, an absolute expert on sycophants.
[Trump] Believe me.
[Trump] It's real.
[Trump] It's very real.
[Trump] But I just don't buy that.
[Trump] You know, we know what the fuck we're doing.
[Trump] That we're actually good enough to theorize like this.
[Trump] My God, guys, we can't be wasting time going off on long chains of abstract reasoning, ruling out all kinds of solutions.
[Trump] I mean, they're really ruling out methods left and right in that document without much grounding at all.
[Biden] I think you're missing the point.
[Biden] We can't run experiments on the really important stuff now.
[Trump] This is totally D-tier content.
[Obama] But the only other thing we have in D-tier is Conjecture's agenda.
[Obama] Do you really think those are comparable?
[Trump] Well, I asked Claude to check for me.
[Trump] These guys didn't run any experiments, but Conjecture runs experiments.
[Trump] So you're right, Barack.
[Trump] This belongs in high F-tier.
[Biden] No fucking way.
[Obama] Yeah, for real, Donald.
[Obama] What the fuck are you talking about?
[Trump] Once you look past the prestige associated with this agenda, you might notice how pointless it is.
[Obama] Why do I get the feeling you aren't going to shut up about this?
[Trump] Because I'm not.
[Obama] I've got the mouse.
[Trump] I'm coming to get it.
[Obama] What the hell, man?
[Obama] Stay in your own room.
[Trump] Out of my goddamn way.
[Obama] I'm telling you, man, people are not going to like this.
[Biden] But I think it should go in—
[Trump] I think we're done here.