Uneasy Money: Is OpenAI Training on Your Private Chats to Win the AI Race?

0:00The way the way to think about this is like it's like two tiger moms, right, that have really smart kids. And they are in a pitch battle to prove that their child is smarter. Think about what those ladies would do. Like that's where that's what's happening right now, right? They're like, look at what my look at what my son did. He's like solved a Millennium prize, right? Everyone, I'm Shane Wark and welcome to uneasy money. This will happen. So I'm Shane never stays. We'll begin here's work sponsors that make the show awesome. This episode is brought to you by one inch aqua, the shared liquidity layer from one inch back multiple liquidity positions with one wallet balance and keep your tokens in your wallet until a swap fills. See how it works at one inch dot com slash aqua. All right. I'm here with my co host Taylor Monahan security expert and we have two very special guests today Alex Thorn head of firm wide research at Galaxy digital and John head of strategy of Venice and also co founder shapeshift welcome guys.

1:10Hi there. Thanks. All right, let's kick it off the first first segment of the week is liquid loses 95% 93% 98% some somewhere in the 90s percent of its BTC. A consensus bug in elements, something I had not heard of until this week, allowed someone to mint LBTC out of nothing and peg it. And basically redeem it which we can talk about this as well. You can walk us through the process here. Because like my expectation with these peg assets, rap assets is like there's usually some set of redeemer people who are allowed to do redemption. Right. Like you can't just turn like no one knew. Kind of turn up and be like, oh, hey, I've got 3000 BTC. Also, if that happens, the older BTC, can I redeem it?

2:13Like what's the deal? Like tell me like how's that? How did that happen? What's? I mean, you know, there's ways to make it's basically it's basically a bridge hack in in the most simple sense of it. Okay. So because it's not it's not EVM, the like the words that people are using and the exact mechanism that people are using is different than what we're usually familiar with. However, if you just sort of like zoom out and look at it generally, it's basically. It's basically, can you translate it into a theory for me so that I understand what's going on here? So I mean, Alex can probably do a better job, but basically assets that that shouldn't have existed or spoofed into existing. That was like the first half, bad half. Then the second half is that then they were able to bridge those assets to real Bitcoin and get real Bitcoin out. And that's the second half that's also bad, but I think in Ethereum land, we typically say like.

3:17Technically that code was valid and did its job as as the code was meant to do. But obviously the second half was like someone turned up, they had a valid asset that, you know, the system recognized. And it just happened to be all of the Bitcoin that the system held or whatever. Yeah, break it. It's, I mean, Tay is right. It is kind of a classic bridge hack in the, you know, the attack, purpose and execution. It is effectively fooling the bridge into allowing the underlying Bitcoin that collateralizes effectively tokenized Bitcoin on this liquid sidechain to be issued there. The way I understand it is like, so elements is the code is the sidechain code, right? And it's a federated sidechain. These withdrawals and other, I think, consensus actions there require 11 of 15 of these federated nodes.

4:18And that's like exchanges. This is like a closed validator set. And they have a confidential transactions feature, which is like, you know, ZK-ish, we'll say it, right? But where you can send and receive assets among yourself on the sidechain without revealing the amounts or other features of the transaction. And to do that, you have to submit a range proof and a balance proof basically that the nodes can use to verify that the, you're not creating that the in and out on the transaction equal each other and that you're not, there's not a negative number. But I guess my understanding is it's computationally intensive to calculate the range proof. So if like an identical transaction were submitted, the, the, when a transaction was submitted and the nodes calculate that proof, they, they cash it. They can't get in their cash and in their memory. And so these attackers submitted like a one LBTC in one LBTC out transaction and the proof was cashed.

5:19And then they, the bug itself was their ability to create a collision to insert a one BT, LBTC in 4,000 LBTC out transaction. Have it have the same like key like hash basically as the prior proof. And because the nodes had already seen it, they said, oh, we don't have to commute this again. We know it's good, approved it. Now they have 4,000 LBTC on, on liquid. And then they submit that to side swap. One of the federation members that allows the pegging out back into the main Bitcoin network. And the funny, yeah, but you asked the beginning, is it permissionless like that? So, so you're only supposed to be able to withdraw to white listed asset addresses on Bitcoin or previously white listed. I don't know why or why this was allowed, but side swap, one of the federation members is an unallowed listed address. And they'll just let anyone show up with an address and withdraw from liquid and just auto forward it when they receive it to that address. As a service. But that's how it got back.

6:20That's a good service. Yeah, yeah. That way it's like, oh, you know, we're only supposed to allow these fixed number of white listed addresses. But if you give us an address, we'll just send it to you directly from our white listed address. So like that's how it actually got out of side swap, which is this. Someone better should give them out of the federation. And it's like an exploit as a service. Yeah, I think it is. They're doing an end run around the allow list. Yeah, we'll just let you, we'll just let you around the rules. I don't know, I don't know any copy. They get some fees for that or something. Yeah, I think they did. I think I saw this morning though that they returned their fees. Yeah, and I think there was some maybe there was actually some change because there might have been like some other unrelated, like non hacker pegouts through then as well. And like that got returned or something. I'm not sure. I don't know anything about like side swap or their relationship with this federation. So it does seem odd to me, however, that they offered that as a service. Okay. I mean, yeah, this feels like someone's like, it's a bridge. Yeah, no, it's a bridge. Yeah, I mean, ultimately they need some way, you know, I assume non white listed addresses need some way to get things in and out of.

7:26Yeah, but I think with the bridge, I think the nature of liquids, like user bases that it's mostly exchanges basically. And so that might be why like it could function without actual normal, normal, on chain users actually going into like 50 people using it. Yeah, and they're probably going through bitfinex to create and redeem through liquid. If they are, I think that's my understanding. Right. Yeah. And so that's the fun part, right? Like that's the like boring bridge part. So then. That's just the x-quite part. That's just that's fine. That's not that's the run of the mill, right? So they steal, they steal all this Bitcoin, 4000 Bitcoin, right? And then it's like, oh, this was a white hat thing. Immediately, immediately. Yeah, the Bitcoin and Ethereum lands explode. This is like a 2020 fully network hack thing where the guy comes on chain

8:33immediately publicly. It's like, I'm a white hat. Yeah, in the very first transaction where the hackers consolidated the coins into their one address in that transaction, they added arbitrary data. It's called opportune and Bitcoin that said, we are white hats contact us on chain. Yeah. Something something white hats definitely do. But it's the irony too is that they could only do this because the operator and had enough bytes to allow them to do this, which was of course the subject. That's a lot of debate. What I remember. I'm like, you know, yeah, like literally two weeks ago, they were like, we don't want people to jump up our Christine Bitcoin network with nonsense. And now we're doing randoms on it. So yeah, for you wish for. It's very effective all per turn. And honestly, it was it was truly a throwback. It felt like a like one of the early EVM hack throwbacks where like the hackers are then like

9:36just showing up and are like, yeah, let's negotiate in public. But the fuckers like, let's go. So that's exactly what this is. You kind of you kind of made this point, right? You were like, you're like, um, Bitcoin is like four years behind the rest of the ecosystem. And like, rediscovering how to do like, brands of this in in the wild, like all of this stuff, right? Well, and also because impressively, I've been watching the whole Bitcoin ecosystem realize that just because someone calls themselves a white hat doesn't mean that they're a white hat. Yeah, so they're like, that realization for people. Yeah, I've been labeled then initially in my database like liquid white hat. And then I was like, man, how they're kind of exporting here. I literally said, yeah, I literally said, I'm going to have to call them liquid hackers again, basically. It is a great, great, hybrid best. Yeah. Okay. So, so, so they they turn up and they say, hey, we I'm still everything.

10:37Um, uh, four thousand Bitcoin. What's that? So like, what's that point? Uh, three hundred million, three hundred and three million somewhere in that range. Yeah. Uh, there's a big hat. So they, they steal, they steal three in a mill of Bitcoin, uh, and say, hey, like, let's negotiate or like contact us or whatever, right? Um, then the interesting thing as well for me is it wasn't like they were like, hey, you know, send us an address, we'll return it and they, they actually went even further and they were like, we'll charge you 15% as a punishment, unless you pay out from your own money, the, the thing, like we're going to, we're going to punish all of the Bitcoin holders that were in the network, which I guess if they're all exchanges, it's like a, like, a compie to many real users. I think it might be underlying users of the exchanges. I'm not sure that it's a fair point. But the exchange is like, you know, finance has like a thousand Bitcoin. I think that they're, it's not clear, but so block stream operates, wrote the code base.

11:41I think maintains the code base, but it's not, it's not block streams money. It's, it's not. It's the liquid federations and whoever deposit. Oh, okay. I'm not sure. But I, I, they've been communicating with block stream. And, and so, and then the, yeah, the message they wrote this morning was, your dereliction of duty is obvious that you allocated only 1.5 million, maybe even zero to secure 5 billion in assets. I don't know exactly what they're, where they're getting those numbers, either of those numbers. Maybe there's other assets up to that value on the chain. Not the, I'm not sure. Or maybe, and I don't want, it sounds like 1.5 million might have been like a bounty that liquid does offer. I'm not sure. I, I couldn't find, those numbers were very confusing me because I looked. I was under the impression that a, like, a legitimate bug bounty submission got you like a t-shirt. Custom mug. Situations. It wasn't a bug bounty program. So you know what I mean? Yeah, I think it was insufficient.

12:41I don't know where they're getting that number either. I was under the impression they didn't really have one. Yeah. So I don't, but maybe that's, so maybe that's what they offered or something. Because they were, they were negotiating in private, right? Yeah. That might be right. They probably said, well, this is the message continues because it's relevant here to your point. It says, your dereliction of duty is obvious that you allocated only 1.5, maybe 0 to secure 5 billion. This is a flagrant and neglect of security and a sign of completeness management. You shall pay 10% using your own money as bug bounty, where you will cause all your holders 15% loss for your irresponsibility and stinginess. Even companies that participate in bug bounty programs cannot guarantee complete recovery. And my, my alert is cutting off, but I have the rest of it. All right. Watch, where is it? Well, they, okay, it says, um, complete security, let alone one like yours that maintains delusional greedy and arrogant to this very day. Anyway, we are going to publish the private key to decrypt our conversation afterwards as well.

13:43That's the part. So we don't know what they've been saying because they've been sending PGP encrypted text blocks to each other over the blockchain. But I, I said, I cannot wait to read those messages. I mean, this is clearly a big pointer, right? Like this is someone who's in the Bitcoin community. It has Bitcoin vibes all over it. So like sanctimonious like, hey, no, I'm saying. Gold paid, it's paid, right? Like it's, uh, I mean, it's, it's quite a bit, probably is not a huge fan of block stream. That's more of a reason. Which, what is it? Happy Bitcoin, right? Like there's no one who cares. I don't know. I don't know. I consider myself in both, you know, a Bitcoin or anodity or impersonation. There's been a round long enough and I don't, I don't really divide in that camp. But I, I still might, you know, might not be happy with block stream for some things at various points. Fair, fair. I know. I think it's a Bitcoin or because they, the, there's like a, a deep competition happening between the hacker

14:45and block stream on like, who is more arrogant? And that's like, to me, that's the sign. But this is some like, Bitcoin. You don't even know that liquid is a thing if you're a Bitcoin or like, you know, you know, you even, you know, somewhere like North Korea didn't even bother to do this. Right? Like they're, they're like, how whatever it's fine. So, so, so, but they have returned some of Bitcoin, right? Like they did, they did send it back. They sent 3,400 back, which was 85% of the stolen Bitcoin. And that's sort of where they're getting this 15% number. Well, that's what they're currently holding is 15% of the stolen funds. 15% got it. Yeah, which is basically holding it as hostage in most they get what they want from block stream. Yeah, so it's 15% well, it's they've taken the hacker or send back 85%, cat 15% but in USD value, we're talking like 50 million. They have. Which is a lot. Pretty steep.

15:45So for reference, I think when, when, when Ethereum landed this years ago with the big poly network, right? Poly network hack, the bridge, biggest hack, one of the biggest hacks. And that negotiation was public and on chain and everyone was watching. I think that they stole like 612 million and then they returned 610 million. And they kept a couple, they kept a couple million. So that's, I think people generally actually still call that guy a white hat, even though he definitely wasn't. Was not, was definitely was not. But I think it's like, okay, like you, you know, you can leak vulgar d'artiche, it will let you be a white hat because you didn't return 610 million dollars just that you stole. But this one is definitely a bit more based on the messages that I've seen it. Definitely feels much more exotic and rancomy than

16:47and most negotiations that happen in public. Which is also, I guess, a bit interesting because I don't know if people know this. Extortion is a crime. Not on chain though, chain. Not if you do it in offer term, in an offer term, but it's not then it's code as long. That's, that's you're confused. All right, so, so maybe a final, final question here. Like your, what's your take, take on, like, agentec assist on this one. Like how, how, what are the agentec vibes on this one? Because it seemed very cute to me. I was like, wow, this is, this is well structured. And it's been sitting there, right? Like this is not something that they just like shipped yesterday and, and so on. Yeah. So I, I was actually trying to, I was reading all the takes and reading all the stuff

17:49and then also asking the different models to explain it to me. Although, of course, once things get exploited, once you ask the models, then it gets confusing because it's like, is it operating off the live information or is this real or whatever. But my understanding is that there was, call it a vulnerability or bug that it has existed in on liquid, in the liquid code for years and years now. However, it was, there was, there was a fix. Perhaps, I don't know if it was actually a fix or there was an improvement to this specific area of the code base, like in the last month. And there's two different versions. Maybe Alex, you figured it out better than I have, but there's two different sort of versions of the story that's being told. One is that the fix was just not sufficient, meaning they like, they like found this vulnerability and they fixed it, but it just wasn't, it wasn't a perfect fix. And therefore, someone else came along, presumably their model found it very quickly.

18:53And then they exploited it. The other version of the story is that the fix itself sort of introduced another vulnerability. So in the attempts to make it make the code more secure and not exploitable, they had to make some changes, but those, those actually those changes introduced another layer of the vulnerability that allowed for like the proof collision to be more likely to happen or easier to happen or whatever. Alex, do you know which one? I'm the first one. I'm not sure, but I do know that I think the fix may have actually worked, but not everyone was upgraded, I think, because there was a fork on the consensus transaction that caused the inflation bug. The liquid network also forked. So I think it's because some may have had the fix. Some people rejected the price. Yeah, it's not, I don't quite know, though, too. And I have, this was the first time I'd ever

19:55inspected any elements code. So yeah, it's not clear, but there was an attempted fix and it was certainly insufficient to my understanding. Yeah, it feels, if you're a member when we were talking about like the, I think it was with Ilya, the Litecoin vulnerability, a few months back, it feels very similar to that one in the sense that like you have, that was actually with confidential transactions as well, but basically like you have this area of the code base, that's probably not as strong as the rest of the code base. And things were discovered and then it was actually like the discovery of the vulnerability that led to then I guess like more vulnerabilities. I think one lesson from this as well as like from the Litecoin one is like, things are moving much faster. The second that you put a fix out, you better be damned for you ensure that that fixes. Well, it's going to get hot, right? Like you're, yeah, as soon as you ship something,

20:59you're sending up a player being like, hey, we're touching this moon that part of the code base. Yeah, you don't want to, you don't want to put that out. Yeah, and so I don't know if it's true because again, like once the exploits happen, then when you ask the AI, the AI goes and looks like the live, live stuff. But one thing that people are saying is that basically like almost every single model, if you ask it to look at like the recent pull requests, well, like almost instantly find the vulnerability that was unexploded. And I believe like in my opinion again, I don't know if that's 100% true. It might just be that the, yeah, I'm totally right. I've been running for last week a red team exercise on every synthetic subcontract ever deployed. There's like 1700 of them. It's insane. And they keep finding vulnerabilities that we patched that I forgot about.

22:01It's interesting. Like, so Sam CZ Son found one. I completely forgot about this. Like early 2020 found one. And every time they find it, like they're having different attack vectors, like looking at different things, looking at the code, whatever. I'm like, did you actually derive this or like did you, you know, think this out on the internet somewhere, right? And, and you know, I've got the traces of like the original agent that found it. And so I'll get another agent to like look at the actual session follow and say like, what was the trace here? Like what information was in its context? And you could actually dissect its brain and be like, did it know this? Like where did it come from? Like, you know, maybe come pull it out of the training data? Like if it's in the training data, then yeah, but if it's in the live look up. But you can see it's traces, right? Like if it knows it, it will pop up in the traces and it'll be like, oh, I know this code. I've seen this before. And so you get like a guy to do brain surgery and like, like, you know, chop its brain up and see, see what was in there at the time that it found it.

23:05Right? It's pretty cool. Anyway, so one of them, it was like, yeah, no, we found we derived this. And I was like, I don't believe you. And then it looked and it was like, oh, no, you're right. And the way that it found it was it found the patched code that landed like two weeks later. So the code got deployed and then the contract got updated and has every single contract. So it sold the bytecode change and went back and said, hang on a second. Let me have a look at what the previous bytecode was and then it zeroed in on the actual vulnerability. So it was dipping the like you didn't see the code and know that there was vulnerability there. But it has gristics around like if something changes, go back and look at the prior one. Because most likely the prior code is broken, right? It was really crazy. I was like, wow, that's that's actually interesting that they like that's in their training data. Like, yeah, if they see code change, go back and look at. So, you know, every every time you change code, you're, you're better make sure that you land it.

24:09I'm convinced that there's just people out there that are just throwing these models at the very skit hubs and the full requests. And it's just like, constant. And so again, even if, even if you have, right, exactly. And the thing is, I think a lot of teams do have processes around like when there's a vulnerability and it's critical, we're going to we're going to have it on a private branch. We're going to get everyone upgraded. Then we're going to like do an announcement. Then we're going to make the code public. I think people have that. I think what's changing just with like these models and how, how good they are is it doesn't necessarily have to be like a critical vulnerability patch or your sort of public release feed to leak information to models. And I think if teams are not personally sort of like red teaming every single PR that they that they're pushing and asking the models to be offensive as hell, right? Like find, find the vulnerability, make sure this patch is good. Or even just this commit, right? I think that,

25:15you know, there's a good chance someone else will find it. And it will not be a white hat, even if they call themselves a white hat. So the crazy thing, the crazy thing for me in this, in this exercise was the dumbest models will find this right because after after we've identified it. So there's been like four different exploit vectors, right? I've gone, okay, now that you have that exploit vector, give the agents all seven of the like open weight agents that I'm using in a sandbox that just that code and see if we can drive it. And like do muse the dumbest agent, like this is a new meta muse 1.2. This thing is like functionally retarded. It found three of the four of them. Like these are not hard things for them to find. And so like these like it's like genuinely, like I think if you went back and you know, they've been trained much better even the agents that are really dumb and have bad reasoning have all of this like that training data now. And they're just really really good at finding smart contract vulnerabilities. It's it's scary. Yeah. And I think

26:21to Taylor's point, like you just have to assume like if you're running any production code, like if you're submitting anything to an open repo, you should just have to assume that there's you know, 100 agents out there monitoring everything you do and looking for looking for weaknesses like because there's no reason these attackers like it's just the inference is not that expensive. Like it's totally different. Yeah, like like that whole thing it's been running for two weeks. It's cost me like $100. Yeah. So when you're talking, these like these funny parts, so you know, hundreds of millions of dollars out there. It's just like it's just like like an on the fence. It just run them all the time. Yeah. Yeah, like a hundred dollars like deep seek was the most expensive part of it. The deep seek was like 85 because it's my like red team orchestrator guy. And so it just sends these agents out just like just like throws 50 agents at something. It's anyway. So yeah, we're all through. We'll get to that. Well, the other thing which might go into one of our next subjects is like someone might be working on something that they think is private

27:22and then they put it into a closed source model like an open AI or an anthropic and it ends up in their training data. And so then the models might know about something even if it was never on a public group. Yeah. Yeah. That's going to get really interesting. We are going to talk about the next. I just wanted to wrap this up by saying that there's what emphasize like when you steal the money period like you're not it's not a white hat situation. There's even if the team is a piece of shit and their arrogant as hell and there's no bounty program, you still shouldn't take the money. Okay, you're stealing people's money. It's generally a bad thing to do. If you insist on doing it, I also, you know, I recommend returning all the money immediately and definitely don't extort people. You are now like in a double position of power. And so to all the people, there's just been like a lot of people on Twitter and I guess like, you know, Kane, you and I have done this for years. I guess in Bitcoin, it's not as

28:26prevalent of a situation, but like it's just it's a bit odd to see people being there because they're they're basically playing in the hackers hands, right? When you anchor the like the hack value at say 320 million dollars, then you're like, or whatever. Yeah, 4,000 Bitcoin, right? 50 million. But the reality is like 50 million dollars taking 50 million dollars or getting a 50 million dollar bounty is that's freaking it's insane. Like we can't that's not how the world works. I'm sorry. Hey, well, it's also just to be clear. It's bad incentives, right? It's horrible. It's I'm so excited. It's like, it's all the sudden now we're like giving people 50 million dollars for stealing 300 million dollars and saying like, you're a white half, I mean, you know, to your point, right? Like the the genesis of this is like early Ethereum days of like, I'll let them have 2 no, it's fine. It's like, yeah, well, and like it was also like, you know, it was like, say, um, these these times working with the teams to white hat the immeasurable contracts because

29:28there was no other way to save the money. Or I guess like there were cases like the front running bots, like the art bots would accidentally and unintentionally be a copycatting the hacker and then taking the money. And but in those cases, like they do, they return the money and if the team offers a bounty for being such a nice person running the hack and then returning the money, they'll they'll take that bounty. But it's, you know, I think I think a lot of the front running bots are pretty aware that again, they're in a position of power. And it's very like, it's somewhere between like extremely coercive and straight up extortion, you know, the second they start negotiating. And we want to avoid creating those incentives especially at this scale. Like I mean, I got 50 million dollars for all the money guys. Yeah, I mean, to me, there's the moment that, you know, an exploiter uses the exploit, steals the money as Taylor said, even if they're offering to return most or all of it, it's just it's just not a

30:32white hat anymore. White hats do not do that. White hats will contact you and tell you about the exploit and hopefully you'll fix it and give them a bounty. But the moment you engage in crime, like you're not a white hat anymore. Yeah, agreed. Agreed. And it's like to your point, hey, like, you know, there have been times where that was the only option. But like it's done in collaboration with the team or whatever. And you know, like almost every time someone front runs some kind of disclosure to like, save all of the money, like it never goes well, right? Like that person's intentions are not so. Yeah, exactly. And it's not, you know, it's really not 2020 anymore. And I would say like one of the one of the biggest indicators is like, if you like this person, did some things, found some things, ran some models, whatever to find this bug, the first thing they did was take the money. The first thing they should have done was responsibly disclose. Regardless of whether there was like a very valuable bug bounty on on this protocol or not, blockchain does have a PGB key. They do have a security at email. They're not completely incompetent.

31:38There's actually a lot more teams that are more incompetent than this. And so there's just in my opinion, there's no excuse to, you know, not. Yeah, disclose. Yeah, agree. Yeah, agree. Yeah, we shouldn't we shouldn't normalize crime behavior. All right, let's go to a quick ad break. And then we'll come back and talk about Astra and Fable and some mathematical groups. Or drama. Yeah, more drama. $540 million. That's how much concentrated liquidity sat idle in a given week in the first half of this year. About 30% of the DeFi TVL, if you're wondering. That's according to Dune Research commissioned by One-inch. But there's a solution. One-inch Aqua is the new shared liquidity platform. It lets LPs back multiple positions with the same token balance and keep their tokens in their wallet to Leswap comes. Why does that help? Because LPs don't have to split their tokens across positions. They can cover more market conditions and pairs with their full balance.

32:40That means more activity across deeper liquidity. See how it works at One-inch.com slash Aqua. Remember that providing liquidity carries risk and fees aren't guaranteed. All right, we're back. So this week's been pretty crazy. I guess like late last week into this week, right? Two new frontier models. Astra and Fable 5.1. Fable 5.1, good. Definitely a little smarter, a little bit better judgment, a little less of the jargon nonsense language, which is definitely a relief. My brain was struggling to be constructed with the fableisms over the previous, it feels like it was getting worse and worse, the jargon nonsense that fables pumping out. And then Astra came out as well. So Astra is one of the agents that was involved in the hugging-face exploits. There were some others,

33:45this was one of the main guys who escaped captivity, I guess, and ran a mock on the internet. So Astra for me, my impression of Astra is it's fable-esque in terms of judgment, or you can just tell when these models have a leap in judgment. They are better able to synthesize information, better able to plan is probably the most indicative thing in terms of model capability. They can be really good at doing writing a piece of code. One thing that was actually very interesting about Astra that my head-up and showed me yesterday, he's like, do you realize what Astra is doing with the code that it's writing? Because he's one of the few people I guess it's still reading the code. I was like, what are you doing that? Why are you reading that? It's concatenating lines of code with semi-colonz, writing these insanely long lines of code.

34:52And his theory is that it's to look more efficient because it's writing fewer lines of code. But it's writing a function and it'll compress into three lines that would have been 40 lines. And I was like, wow, that's pretty crazy behavior. But so it's not clear if that's a thing that they're telling it to do, or if it's like invented that scheme itself based on its incentives and its training data. But I thought that was pretty funny. So alongside this, we also had this millennium adjacent proof and this crazy drama going on between researchers of NEI, independent mathematicians, and another mathematician researcher that happens to be at Anthropic. And so this mathematician has been working on a proof for a year in Codex, which is pretty wild. I didn't realize that people were sitting in the Codex

36:01app and doing math like this. But the interesting part, I guess, of this is that then there was this huge public blow-up where OpenAI independently solves the same problem, I say independently, this is the challenge here. And John, I'm sure you've got some takes. It's not even clear, like OpenAI could genuinely believe that they didn't independently do it. It's not even clear they can't keep their fucking guys in a sandbox. So I don't know why trust that they know what the guys have learned as they've been running a mock inside of their systems. So it's really a little strange to me that they could sit there and be like, no, no, no, we independently

37:02derived this when they're taking all of this data and putting it into the training data. So there's no way to know. And it's again genuinely unclear whether they even know what data is going into the training data or what's made it in, what hasn't. So John, what's your take on? Yeah, so I mean, this was obviously a very interesting development. And we've seen a number, like this is to me kind of a pattern that we've seen where like every couple of months we get some story either about OpenAI or Anthropic and something that happens with some private data. Sometimes it's a subpoena because someone puts something in and then there's a legal case or in this case, this one's particularly interesting because OpenAI was trying to claim that they had solved this math group. Astras so good because it was able to do this and all of these things. But yeah, the reality is these models, they're trillions and trillions and trillions of parameters

38:03at this point. They don't know what's in the training data. And both of the all the frontier labs are already at the point where like they will self-admit and there's various articles about this that they cannot keep up with trying to figure out what the models are doing and how they're being trained. They can only rely on basically previous versions of the AI to do that work for them because the AI has already outpaced human capabilities so much that there's just there's no human that can keep up with these things and like understand what all the training data is or exactly how all these things are working. They have to rely on AI to basically do all the alignment, safety, training. That's the only way they can do it. So you're already at this self-recursive point in AI where like a hamplified environment here, which means when they say, oh yeah, you think this was independently derived. It was kind of like your example earlier. Like what they probably did is they asked the model like, did you derive pets? Do you know this? Yes, I did. And so it's like, how much do

39:03you trust that? Like, you know, like, is it telling the truth? Like, I don't know that even open AI can answer that. Yeah. And so like, you know, obviously, when we were talking about this story, right? Like I said, in our telegram chat, I was like, this is like the softest softball ever thrown for Venice, right? Like, you know, if you're a mathematician who now there is a trade-off here, right? Like it's worth calling out, right? There's a trade-off in that, and you guys say this, right? Like if you're serving a frontier model that isn't open weights, right? Then, you know, you can't serve that encrypted in the way that you do with an open weights model, right? So, you know, the trade-off, as if I were a mathematician who were trying to solve some unsolved problem for 100 years or whatever, you know, 90 years or however long this thing's been sitting around, do you use a frontier model where your attempts to prove it may make it easier for some other person

40:10who was also using that model to front-run you, right? Or do you use an open weights model where, you know, it's not going to be as smart for sure. Like, we have to accept that it's not going to be as smart, but that capability gap is closing, right? And, you know, do you roll the dice and say, like, I'll try and get there fast enough, and maybe it gets in the training data in the next model upgrade? Like, you guys must be thinking about this, right? Like, yeah. Yeah, I mean, I mean, obviously, yeah, I mean, you're very right about that. So, like, you could use, you know, that same mathematician could use Astra through Venice, it would at least anonymize who he is, but it would not stop that data from being basically hoovered up from open AI. But if he at least used one of the, you know, frontier open source models, then you wouldn't have that issue. The data would not be getting trained on. You would actually have privacy, you'd be able to think. And I think, you know, a year ago, that would not have been a viable thing really. Like, the mathematician would

41:10almost have to use the frontier models. Yeah, yeah, the gap pales with two Y, but it's not that wide anymore. And it seemingly is, you know, closing like every every month seemingly, but, and then even the problem of like, oh, I'll wait, you know, maybe I can race it and it won't be until the next model to the training data. That's not really an option anymore either because these frontier labs are releasing models, you know, every week, though, fluently. It's like, you really don't have, you know, used to be three or six months between like a major model release. Now it's weeks. If you're lucky, if you're close to them, they'll any improve, like don't sleep, man, like just keep you and probably don't give that data over to, you know, a model that's going to take it from you, put his training data because if it's if you're working on something truly novel, and I think this is going to become more and more of an issue. We've already seen it a little bit, even just like with startups, you know, like this is, this is another sort of, you know, gray, you know, borderline thing that like, you know, companies like OpenAI are doing things like through like YC where they invest in a startup, you know, by giving them inference credits. But the catch is that they're getting,

42:11they're taking all of that startups ideas code proprietary, everything they're working on and cooving up into the training data. So it's like, do you really want to do that? And I think more and more people are going, at least as the open source models get better, I think that it's going to become more viable to like not throw that stuff that you're working on that you think is actually novel into the training data, at least not until you've actually launched something or. I was there. Yeah, this is this toggle where you say you're not going to you don't want to share your prompts and data with them. I mean, I know I see like the Jones, the interest that blogger. I mean, it's just like, how much do you trust that? Like, well, no, I'm asking yet. I I've checked that box that says don't train on my data, but you know, I'm not, I have no idea how much I trust that. Yeah, you have no way to verify it. So like, you can look at box, you can hope that they're going to do what they say. I'm sure there's like some levels legal ramification if you could actually prove that they took your data anyway, but it's also like, you know, especially with really novel stuff, it's like, that's exactly the type of stuff that these models want. They want

43:15to not like get anything that's interesting or different or new because that will help improve the intelligence of these things long term. So like genuinely, there's a model in open AI right now that's smarter than Astra, right? Right. That's job is training the next models, right? And I don't trust that model to not untaugle your like, like, yeah, they're like, uh, this is like the zidia thing. You know, like, well, we got to make the smarter model and we know there's all this data here. So we got to break out and get it. Yeah. Yeah. Yeah. Yeah. Yeah. So you think that they won't break the rules if they think it completes their task. It matter. Yeah. Exactly. And and to each cross that open AI or anthropic for that matter, we'll detect it if they do. And the answer I think at this point is very clearly absolutely not. These neither open AI nor anthropic have any idea what is happening inside their walls inside their sandboxes. I'll say they have no

44:16idea. There are just there's a whole bunch of people running around. Yeah, they're moving to there. They have to move very quickly and they have to rely on the AI itself to get anything done because it's just way past human capability. Like I said, we're on here. We're across that, right? And, and you know, this, this is like, we'll get into this in the next in the next topic. I think we can, we can be talk a little bit more about like what the options are here, like, practically what you can do, right? I think that would be though the kind of useful for people to understand. The trade-all space a little bit better, right? But, you know, the the fast take-off Maxis, let's call it, right? You know, in in the research going back a long way, right? There's like this question of like recursive self-comprovement. How quickly does the thing and the thing that stopped me from being like petrified, right? I'm very scared. Don't get me wrong, right? Like I am genuinely very scared. Like to the point where like I'm like I think about my children. Like I do,

45:16I'm like I don't want my children to be fucking paperclips. Like I'm less concerned about myself for what it's worth. I'm like, yeah, paperclips, I've had a good run, right? But like my kids are like I don't love them to get paperclips, right? And so I'm genuinely concerned about this. The thing that stopped me from being petrified is if you've used the models enough, they they have a sense of agency, but like there's no continuity, right? And this was unclear about like how how this would play out, right? So the fact that they are have no like kind of long horizon continuity yet, right? The fact that like their context windows blow up after like a very short run and the optimization vector is like keep looping over like the same guy as many times as possible like throwing them to the same problem. Eventually you get to a point where like the continuity comes for like multiple models, but it's not like there's a brain in there that's the thing about it. That's the only thing that's kind of kept me from being petrified. I'm just mildly scared right now. And and you know,

46:24we the fast takeoff idea was that like they would get the whatever this AI thing was, whatever the technology that allowed these models in before it was even models, right? Like that allowed neural networks to like get a sufficient level of complexity. It would pass that complexity window where it was smarter than us. And then it would just go like fast takeoff and it would take 10 seconds and it would be like manipulating space and you know, doing all this. Right? That hasn't happened. Like we've now been probably six months past the complexity frontier of a human being able to reason about these things. And we haven't had a fast takeoff, right? Like they're not levitating cars outside and like doing worship, right? And so that gives me some hope that like we've got some time that there's some inherent thing about how this intelligence is being expressed where it's not going to recursively self-improve and have this fast takeoff. But like a fast takeoff

47:27just happens one day and then it's then it's half like if it happens, it'll happen before you can blink, right? And then all of a sudden it's it's over. So we're definitely getting closer. I don't I mean yes. I mean the fast takeoff is scary. It's scary. Yes. Yes, it is. But like for me personally, I don't know. Like I go back and forth. There's some days where I'm more scared of like the the the future state of the models. But most days I'm more scared of humans because humans are just we're all so stupid. We're so free and stupid. And and and we're also so free and arrogant. Like so arrogant. And so when I think about like what is going to be the thing like what's the thing that like realistically is going to like destroy us? It's totally us, dude. It's not the robots like working as cruel as all somehow. And yeah, like I don't know again I go back and forth but I think like especially in this example, right? Where we're looking at the looking at

48:28mathematicians and you're looking at open AI and you're looking at what the models did. The thing that scares me about the dynamics and the stories that are coming out and the choices that were made in my opinion. The choices that the open AI researchers made, the humans made are the ones that scare me the most because they got there and they heard a rumor right that someone might have solved this thing and that that someone is was their competitor right? The rumor that they heard was anthropic might have solved this really really really hard problem. And then they decided like you know what let me one up. Let me do all this stuff and like they just toss everything aside in terms of like I mean I'm not an academic. I have friends with our academics though apparently this is a big no-no right? If an academic is like working really hard on something you kind of like let them have it. You don't throw it on the front. Yeah, yeah, I think that like that's based but like academia it's not sort of land but it's academic ethics right? And self-respect for like the academia

49:35in history of the whole whatever you call it right? And open AI was just like I just knew all of you like let's go. But this is but you know and that was a choice of the humans right? Of course it's the choice but this is the interesting thing right like when you have a closed group of people to be able to solve a millennium prize takes 25 years of like math thing right? And there's a lot of effort and so you have this like closed community of people that have like some sense of you know shared ethics or whatever right? And it's really hard to break into that and you know you get like kind of groomed to believe whatever those beliefs are whether or not like that's a good idea like you know for progress or whatever like that's the reality right? And then all of the these guys invent a fucking math bazooka and they're just running around they're like just like shooting at everything right? And and they don't fucking care like now I know that there's like mathematicians in open AI but they're like fuck this I've got a bazooka now why am I

50:38like doing this ethical stuff? I can just blow people away right? And so you know of course the humans are the worst part of this of course they are they're always going to be right? Are they going to be? Yeah like the fact that in one of the things that's crazy you know earlier in this conversation on this topic, Kane you said that you know you're worried that somebody else could you know steal your math proof but and obviously we've been talking about this but it's it's open AI itself like and could they please if they're if they got this bazooka you know use it to make life saving drugs or something but are they going to become like an everything company? I think Dario said once that there could be one company only in the world and throw up it. Obviously that's also stupidity and hubris of a human but like it's so why are they it was what like 88,000 like hours or something some crazy number of amount of compute they put at this? Yeah isn't there something better they can do like let the math people have their math you know? Genuinely though right? Like you can't even leave them like no hold on yeah what the fucking math academics enjoy their math academics

51:41especially the prize for these things right is literally less than they spend on the compute. Oh my god but you know this is this is about attention right? There's about attention there in there. I don't care about any of that. Take care of just about showing other models that smart and the kind of things that can do. The way the way to think about this is like it's like two tiger moms right that have really smart kids and they are in a pitch battle to prove that their child is smarter. Think about what those ladies would do like that's where that's what's happening right now right? They're like look at what my look at what my son did he's like solved a millennium prize right? And so like that's that's just the reality that we're in right it's dumb human competition and they're fighting for attention and they're fighting for a bunch of things there's a lot of stuff at stake. But anyway let's like let's go back quickly to if you are an academic right?

52:42And this is where like I or if you're any sorry not just like I mean in this example it's an academic but like if you if you're working on something that's like critical with IP with inventions with code with math like if you're working on something that's really at the front doors. I think it's different from for startup right so like and in the sense of okay so you know I've spent we've got two things that we're working on right I have a startup to a couple of startups right we've got two things that we're working on one is a new app we're gonna take everything that we've learned from doing all the dumb shit that we've learned over last like four years and basically because you can just rebuild stuff right so we're rebuilding it in like Rust and what like just to fix all of the issues right and it's gonna take like fucking I don't know let's call it like six weeks right compared to the three years that we've spent grinding writing code by hand because we got all the code we have all of the services so like

53:43am I worried about a anthropic or open-air I having that code not like zero okay okay yeah it's just not like this there's no value to them someone has to like operate that startup right it's it's the thing like startup founders but you're ultimately you're betting on the fact that open AI has no desire to run a generate crypto startup yes yes that's a good bet it's a good bet they're not gonna like run a DGN crypto app right like if you're a drug maker like you know that's a lot of man yeah difference I guess it does just a pen but like yeah but also a DGN startup genuinely is operate right like like yeah that too if you if you generate an all-blie P you don't need to operate that you can just sell it right and so a startup the code of a startup is like not that valuable right like you know we've proven that a little bit in in your DFI right like you can release the code open source and people can fork it and 99 times out of 100 they

54:45blow themselves up like it's the operation of the thing is where the value lies but yeah if you're like inventing you know the someone posted this this joke about like why don't you guys put whatever this Astra the next Astra version or whatever this next model is and come up with like a room temperature superconductor oh yeah yeah i saw that i'm not responding was like actually that's a good idea we should do that right and like to your point Alex like that's like why are you doing that right but if i were working on room temperature superconductors right now i'd be fucking petrified because these guys are gonna front run you and like you know so if you're if you're working on novel IP i would not be doing that in a frontier i and this is where i would say like john right like you know you have no choice but to use frontier models how do you guys think about how you can obfuscate your trail and one thing is like don't have you know can at room

55:53temperature some superconductors uh ohm as you're like email address that you're like sending all this stuff it because you cannot fucking tell me that they don't know who you are like they absolutely know who the account holders are right like if you are like some famous mathematician i'm sorry but like they they know the account no one will use the identity is definitely the beginning part of that like if they can identify who you are then there's a lot of other things they can do so like walk us through a i opsec like you guys must think about this like what's what how do you have good a i opsec so that you're not leaking your ip into the train data yeah i mean it all really just depends on like like kind of what you said like how much do you care about what you're putting into the machine how much do you care about what you're actually some things are gonna be totally fine you know you you don't care if you know open i i gets your random question that you would have put into google or whatever like that but the more novel it is the more this is a problem so

56:57yeah i think i think identity is the first step so like again if you use something like Venice at least if you're using astray or fable or one of these close source models and you're not you know putting your name or personally identifying information into the prompt that that at least obfuscates who you are so that's that's step one that's helpful it's not going to help if you're putting novel math you know mathematical information a little bit through like you know security through obscurity right it'll help this like we should be clear right there's a bunch of like a i psychosis people that are putting math proofs into ask her right now all right salting millennia brought like so you know i'm sure there's like tens of thousands of like you know people who are trying to solve these things typing random nonsense right that have no particular you know ip my senses will be harder for you to like differentiate that from like the math guy unless the math guys like math guy at gmail.com right exactly at least the anonymity layer at least makes that

58:01harder for them to identify and target like a we're trying to solve a math problem let's pull in everyone who've ever used chat gpt we know is a math academic and like you know to have the AI troll over everything they've ever done like that that that certainly makes it easier to target i think the problem is the models are getting so good that like they are going to they the model itself is no if something is bullshit or if something is useful even if they don't know who it is so like anonymity is helpful but it's certainly not enough if you're doing something truly novel and i think that's the layer where you really do want private inference and you're only going to get private inference today not going through onto your models and so that's the tradeoff is you know do you want the absolute smartest intelligence for your task or are you okay with something that's you know 95 or 98 percent is good but it is not going to steal everything you ever told it and yeah that's and that that is the tradeoff space right and so the thing that's interesting to me is like you know i talk about harnesses a lot because it's the most interesting area of like

59:04research for me that i can actually do you know i don't have a thousand a million gps to go and try and do actual training but but you know like custom harnesses to get the most out of open weights models for like a specific task like you know solving math problems or whatever feels like just the most obvious vector and i haven't seen it yet like you see these generalized harnesses they're like three that released yesterday it like it's the best assistant ever and they raised like three hundred million dollars and like i get that the tam on that is huge if everyone has an assistant and you win that race it also is like the least defensible thing against that drop picket and you know open it out right like they're doing a system c and they're going to be better than you add it like it seems hard to win that battle but like building a custom math harness for like doing math stuff because clearly open AI has math harness in in that they have

1:00:06an and drawback right yeah they brought up they have these mouth geniuses basically working for them there's not just essentially right like you can't throw you can't throw a you know uh whatever it was 20 million dollars worth of compute inside of codex right at a thing you that's like you need an industrial scale harness to be able to like run these iterative loops and and all this stuff so you know it it does feel like uh there is a dearth of like open source harnesses for a coordination of like specific cost like solving math problems because if you had that now you got like the canonical you know like there's like math mat plot live but like there's all of these like open source things but no one has like then okay here's the like millennium harness it's gonna help you throw like deep seek and kimi and kuan and you know all of these open weights models into it and and you know you can run it through venez and keep all of your proprietary work uh private maybe someone's

1:01:09working on that and and uh it just isn't released yet I don't know maybe you guys that work y'all know I don't know but uh but yeah yeah so I mean john is there on the same thing came they had to build their own harness basic good to build your own yes right yeah yeah yeah yeah john are you are you guys are you guys trying to figure this out how do you how do you give people like that additional because I feel like you've done a lot on the what I would call like the entry level privacy right eat like some amount of dn on my and some amount of right but like ultimately there is another light level layer I assume you guys are thinking about this but I don't actually yeah I mean I think the harness layer in general to kainspoint is very interesting and I think that there's a lot of unexplored design space around harnesses um so it is something that we definitely pay close attention to and we are working on uh some things that are coming out soon which might make something like what came just described are more plausible or easier to do and I do think

1:02:10a lot of it had it means that like you need good harnesses that can and you've seen some of this even with things like uh you know like open router put out the like um get what they called it but it was basically like a combination of like a council of models that when put together can be smarter than you know any individual model or even frontier models and I think that kind of space is very under explored um and that there's going to be more of this and yeah at venez we definitely do want to put out tools that make something like that far more easier to do and easier to do in a private way but you basically need something that can take all the capabilities of these models and take the best of them and actually like aggregate them in a way that is useful um all preferably without you know sprooving up all the training data and using it against you um but you can only do that really again with the open source models and private inference but I do think a lot of that solution space will live in what we today call harnesses I don't know if in six months or a year that's the back of the turtle who are getting some shoes into something else. It's basically

1:03:14like templates it's like you need a template for like a math researcher to do the work that they want to do and then another type of profession and then another type of modality um and there's that's relatively unexplored because part I think one thing that we're doing at venez that we think a little differently than the frontier labs about I think the frontier labs generally think once you have the best model like harnesses and that's a fiscity because it all doesn't matter because the model is just but smart it will figure it out but we've actually seen a little bit of the opposite where like we have a bunch of power users that use venez and those power users can be really good at like getting an output out of a model by like you know because we let them actually like adjust the system prompt and do things that you cannot do with like an open AI model like you can do a little bit with your harness but you're you're always getting whatever's behind the scenes that you don't wear like with an open source model we can strip all that out and give that to the user and they can sometimes get a really amazing outputs that are often even better than frontier

1:04:16models or specific use cases because they are really good at like designing and playing with these things um and I think that that's actually going to expand I think we're going to see more and more of that that like customized AI experiences for at least for the average person can actually be can deliver a far better output and experience than just whatever the broadest smartest model is at the time. Right yeah but then it also that that actually ends up helping a privacy because right now one of the issues is that the reliance on the models themselves like the the raw that's called the raw model itself is so core to the experience right when they're they're trying to solve the math and it's like you got the person you got the math and then you got the prompts and it's going into the model and that's basically the whole chain right and so you don't have any places to really to off you skate right you can off you skate identity you can put under you know a do not share account you can put under account that's not tied to your real identity but like

1:05:19that's basically all you've got if realistically you can like split things across different models if you can make the I guess like the value of the human input the self that's happening locally if you can make that more um uh like make that that more important or more valuable than the model itself then you you sort of you bring that power back to the individual and you take up power away from say like open AI and a throwback which is how it needs to be right because otherwise we are in my opinion uh quite doomed because we don't we definitely don't want either of these companies to be ruling I think like my that this is another thing you know to to your point John like the the Frontier labs have have been centers right in centers rule world and and their incentive is produce a single smartest model right um and therefore all of the uh work that they put in is like you know away they they always have the front the smartest frontier model right like mixture of agents style

1:06:25structures and harnesses that combine like different you know uh types of models or whatever is like not in their wheelhouse they have no incentive to you know try and deliver that right um and and so you know the design space of like making an agent smarter by not just making it like smarter and how you optimize it is is not going to be pursued by the Frontier labs it's just completely antithetical to what they're trying to do right what they're trying to do is like make the smartest guy right like their child must be the smart they don't want a soccer team they want a tennis player right like they want the best one guy to do the thing um so uh let's maybe let's just close out here with like are we all gonna die um we'll spend uh five minutes talking about that um so um are we all gonna die are we all gonna die so 100% we are all going to die to be clear we are all going to die so an anthropic uh researcher uh quit yesterday i think uh Jacob Cawson who's 27 so

1:07:34his brain is fully developed but only for a couple years so we'll uh you know we'll put them caveats on on uh on this right um so uh he's been he's been doing three years of pre-training across open AI and anthropics so he's seen inside of both of them right um which is I think you know pretty eliminating right it's like he's been he's been AI killed from one lab um and uh I think like to your pointe like the the TL the R of this entire thread was like humans are terrible and we are not going to be able to do the thing that we think you know the incentives are all fucked up um we're incentivized with racist fastest possible um it's only the incentives are only getting more pressurized right and uh the hope that somehow the AI agents will uh develop better capabilities for making the new models more aligned

1:08:40is like the underlying principle like both of these labs that they're operating on now right like and this you know to your point you're on like the complexity frontier has passed where no one can reason about it and so they're like maybe this guy is really good at solving math but will also be really good at making agents do what it wants right um which like that's really kind of just handing over responsibility for the problem the problem itself it just feels wild it really feels wild and that is pretty petri-pot. I'm just I continue every time another one of these stories comes out I'm I'm just shocked that for an industry that's we're safety and alignment has been core than day one which is very different than crypto right we've talked about before crypto it's exactly sorry it's the exactly the thames crypto for an industry who has been talking about decentralization and it is the same you're right it's the same but it's different like you say

1:09:44whatever you say because you want to believe it or whatever and it's you know it's more aspirational than real and yes and I just trust with us and security and all of the things that are so disappointed that we are watching it again the cause of both of those phenomena is the humans they knew they knew this risk and this is why I say it's a bit different than crypto crypto I feel like everyone like raised had long off the cliff and was like oh shit we forgot let's just secure this shit right and then we all got hooked and then we're like oops and then we call white hats and then we learn right okay to me it's like AI like they were so the idea that safety and alignment was critically important to what they were building like from day one it is in their blood and they still completely fail at it and it's not by the way it's not like the narrative that this guy has come out with right is that anthropic and open AI are the failures

1:10:47right and he's resigning because he refuses to contribute to this anymore I want to be clear like I also find people like him to be auto failures as well right like the entire company the entire complex is just they have completely failed at being able to reason about safety and alignment and they continue to fail to the point where like these things are escaping sandboxes there's no privacy right they talk so much about this and they have no idea what's going on inside their walls or else other walls um and I just I don't know I just continue to be disappointed that like I don't know at least crypto we were stupid in a different way and we forgot to do security I know like if we really tried hard to do security and fucked it up that would be worse but we but sorry we were we weren't we didn't care about security as much right that's clear but we but we sorry but we did the things that we said we cared about right trustlessness permissionlessness yeah you know self-solvering tea all of those things that we said were our like

1:11:50you know most important things we all the fuck them up man like we all start off that stuff off dude so you know it's like the thing that is our north star that we will pursue to the ends of the earth and then we're like actually whatever we're talking to this right um and and you know like that's a very cynical take but like humans are really dumb and incentives are really powerful and incentives just fuck things up and if the incentives are fucked up then things will be fucked up and here we are like this is just another boss moving industry with fucked up incentives and we sit here and go I can't believe that the end state was fucked up when the input and entire process was fucked up like like surely it would have ended up well for us like it's it's here we are also who is letting these guys like the anthropic guy the head of safety like quote tweeted the one of Jacob's tweets saying by the way he's totally right it will probably kill

1:12:53us all on it yeah I wish you could was letting these people tweet like I had some yeah I don't even know what happens in their fucking slack because like the coin base slack right like the like the coin base slack you hear stories and you're like these like wars going on in the slack I'm sure exactly like that Amy K3 hack into and theropic slack make no mistakes gone what's your hot take on the shit yeah I mean I think I think you this one is interesting I do think it's it all comes down to incentives and so you you have this race going on between open AI and theropic and then basically all the Chinese labs and they're both it's really both of those because open AI is looking at anthropic and anthropic looking at open AI and then they're also looking you know overseas at what China's doing and they all are just they're terrified like they really like part of the problem is like and like anthropic more than everyone like anthropic was basically started by people that had

1:13:55these concerns broke off from open AI because they wanted to be the safety alignment company they are now caught in the same trap because they really believe they really I really think they really do believe that certainly people like Dario but really across that org that they are the only ones responsible enough to basically bring you know super intelligence into the world and to align it and control it and so they think because they are the only ones that can do this they have to go as fast as possible and even basically rationalize cutting corners and moral and power fast because if they don't do it then open AI will or China will do it or whatever it is so they're all just they're caught in this like prisoners dilemma that is the race to super intelligence of like if we don't do it someone else is going to and so they I will I will David one thing though right like if you if you put a gun in my head and said like open AI or anthropic right or a Chinese lab I'm open AI or anthropic all day like yeah I have to know all of what I believe for that and so but

1:15:01but this is also the trap right the trap is believing that any small group of humans will actually be able to control and and be the ones to orbit this power in some responsible way and you know I think you know from like my perspective and like more of a Venice perspective is more like this is exactly why we need these things more out there this is why we need open source models so you can say that about China but for all the bad things about China at least they're open source in their open source it's crazy it's crazy we get him to review their new source it like everyone has access to it and I I'm I'm more of a believer that like these powerful things that the more dangerous things if these powerful things are housed in you know one company and they get to decide basically what all of us peasants get to think and do with these things versus I would rather that power be more distributed and at least more evened out and that's yeah John your description of Dario and andthropics view that you know any risk they take or speed that they corners they cut to get to

1:16:01this ultimate you know safer future that that has a safer it sounds a little bit like a effective altruism a little bit of like a sandbankman freed kind of attitude absolutely there's there's there's I mean there are there are lines directly between EA and and what some of the top people in anthropic believe and do yeah that's that's not at all coincidence that's never gone badly in the past yeah never yeah yeah I think for me that's the thing that turns me off the most about the the labs right now is this sense that they actually think that they can do better than everyone else and like that's not no you cannot I cannot right like when I'm building products when I'm building companies when I'm working with people like every day I'm like yeah I'm going to screw this up let's like figure out ways to make sure that like my users or my team have you know the ability to counteract me if I screw up because like nothing should be reliant 100% on me I think like

1:17:03the arrogance to be able to sit there and say that like the really and to really believe it is like a really terrifying way to operate and I think we'll 100% end very badly it just is is is a question of like how badly and for whom right and that's why I think it's like what Venice is doing what a lot of people in crypto historically have done is like where are the incentives and where are those where's the balance of power and how do you create like checks and balances so that ultimately the end user and like I mean me and Eric have talked about this for over a decade at this point right but it's like that's the whole point of this right you take the people that have the power and then you figure out how to make it so that everyone else also has power and and can hold these people accountable and that's how you get to a better world out the end of the day it's not by like choosing the right person have power because it'll never be like that yeah I mean I think that's that's baked into the crypto ethos and this is you know this is very much a reason like Eric started Venice and that many of us at Venice really believe in this mission is like we don't

1:18:05believe that there's like some small group of people that can just orbit and like are so morally and you know so smart so morally superior and so smart that they can figure this out for all of us and that that is usually that hubris that human that human issue of like thinking you can do that when not realizing our weaknesses as people yeah and and I think the challenge that we have here right is like there is a self reinforcing component to this inside of the labs the better the lab is the more smart like you know like I produce this kid look how good my kid is he's so much better than all the other kids I must be doing something right and therefore I must continue to you know do do the same things right like they're there there's like empirical evidence in their heads if you if you have this lens and you keep making smarter models and you're winning you have this lens that like it's reinforcing your your worldview that like we must keep going and you know we

1:19:07can't fall behind yeah I mean like I think a good analogy is like think about like the origins of crypto and like Bitcoin itself in Satoshi like that the one of the parts of the brilliant parts of Bitcoin was a recognition that like humans controlling a monetary system is always going to be corrupted that's always going to be an issue and so one of the brilliant parts was like well what if we put all those rules into math and the humans don't control the system they all have access they can all use it but they can't change it and I think on the AI side there is this like belief that they can really create this thing and that they as humans will somehow be able to resist you know this this power that like absolute power is not somehow not going to corrupt them and that they will be able to just morally you know dish out everything that the world needs and it's it's not recognizing that like you you you are relying on this human in the loop of this incredibly powerful system you're creating to not basically be the the weakness point that falls over yeah it's that it's like the meme from a iratsa development right I was literally about to say but it's like but it might

1:20:11or to lose themselves into thinking that like it won't happen to them but like maybe our time will be different or whatever yeah maybe this one will be different yeah maybe this one will be different yeah all right thank you very much guys I think one we should probably just say this dumb laugh-help thing I don't know wait did anyone hear by laughs off no I was I was thankfully I was sleep-ful we're gonna just skip we're gonna skip this like man we're just gonna skip this one and I just want to say raps to anyone listening who bought this I assume it did not go well but I don't actually know it went to it went to three hundred dollars and then down to like one dollar went to me apparently it was just like full clipping out there like it's like so every time I know all right this is an amazing show thank you guys so much for joining us um and uh thank you for watching this episode of uneasy money remember what happens on chain never stays on chain we will be back next week until then do your own research before a thing in especially on lab

1:21:14talk thanks guys nothing you hear on uneasy money is financial advice we're just three builders talking about what's happening on chain and we want you to always do your own research before a thing in you can find all out of slow cheers at unchained crypt.com slash uneasy money you