Uneasy Money: Is OpenAI Training on Your Private Chats to Win the AI Race?

0:00The way the way to think about this is like it's like two tiger moms, right, that have really smart kids. And they are in a pitch battle to prove that their child is smarter. Think about what those ladies would do. Like that's where that's what's happening right now, right? They're like, look at what my look at what my son did. He's like solved a Millennium prize, right? Everyone, I'm Shane Wark and welcome to uneasy money. This will happen. So I'm Shane never stays. We'll begin here's work sponsors that make the show awesome. This episode is brought to you by one inch aqua, the shared liquidity layer from one inch back multiple liquidity positions with one wallet balance and keep your tokens in your wallet until a swap fills. See how it works at one inch dot com slash aqua. All right. I'm here with my co host Taylor Monahan security expert and we have two very special guests today Alex Thorn head of firm wide research at Galaxy digital and John head of strategy of Venice and also co founder shapeshift welcome guys.

1:10Hi there. Thanks. All right, let's kick it off the first first segment of the week is liquid loses 95% 93% 98% some somewhere in the 90s percent of its BTC. A consensus bug in elements, something I had not heard of until this week, allowed someone to mint LBTC out of nothing and peg it. And basically redeem it which we can talk about this as well. You can walk us through the process here. Because like my expectation with these peg assets, rap assets is like there's usually some set of redeemer people who are allowed to do redemption. Right. Like you can't just turn like no one knew. Kind of turn up and be like, oh, hey, I've got 3000 BTC. Also, if that happens, the older BTC, can I redeem it?

2:13Like what's the deal? Like tell me like how's that? How did that happen? What's? I mean, you know, there's ways to make it's basically it's basically a bridge hack in in the most simple sense of it. Okay. So because it's not it's not EVM, the like the words that people are using and the exact mechanism that people are using is different than what we're usually familiar with. However, if you just sort of like zoom out and look at it generally, it's basically. It's basically, can you translate it into a theory for me so that I understand what's going on here? So I mean, Alex can probably do a better job, but basically assets that that shouldn't have existed or spoofed into existing. That was like the first half, bad half. Then the second half is that then they were able to bridge those assets to real Bitcoin and get real Bitcoin out. And that's the second half that's also bad, but I think in Ethereum land, we typically say like.

3:17Technically that code was valid and did its job as as the code was meant to do. The second half was like someone turned up, they had a valid asset that, you know, the system recognized. And it just happened to be all of the Bitcoin that the system held or whatever. Yeah, break it. It's, I mean, Tay is right. It is kind of a classic bridge hack in the, you know, the attack, purpose and execution. It is effectively fooling the bridge into allowing the underlying Bitcoin that collateralizes effectively tokenized Bitcoin on this liquid sidechain to be issued there. The way I understand it is like, so elements is the code is the sidechain code, right? And it's a federated sidechain. These withdrawals and other, I think, consensus actions. They are required 11 of 15 of these federated nodes. And that's like exchanges.

4:19This is like a closed validator set. And they have a confidential transactions feature, which is like, you know, ZK-ish, we'll say it, right? But where you can send and receive assets among yourself on the sidechain without revealing the amounts or other features of the transaction. And to do that, you have to submit a range proof and a balance proof, basically, that the nodes can use to verify that the, you're not creating that the in and out on the transaction equal each other, and that you're not, there's not a negative number. But I guess my understanding is it's computationally intensive to calculate the range proof. So if like an identical transaction were submitted, the, the, when a transaction was submitted and the nodes calculate that proof, they, they cash it. They carry their cash and their memory. And so these attackers submitted like a one LBTC in one LBTC out transaction and the proof was cashed. And then they, the bug

5:20itself was their ability to create a collision to insert a one BT, LBTC in 4000 LBTC out transaction. Have it have the same like key, like hash, basically, as the prior proof. And because the nodes had already seen it, they said, oh, we don't have to commute this again. We know it's good, approved it. Now they have 4000 LBTC on, on liquid. And then they submit that to side swap. One of the federation members that allows the pegging out back into the main Bitcoin network. And the funny, yeah, but you asked the beginning, is it permissionless like that? So, so you're only supposed to be able to withdraw to white listed asset addresses on Bitcoin or previously white listed. I don't know why or why this was allowed, but side swap, one of the federation members is an unallowable listed address. And they'll just let anyone show up with an address and withdraw from liquid and just auto forward it when it when they receive it to that address as a service. But that's how it got that good service. Yeah. Yeah. That way it's like, oh, you know, we're only

6:23it's supposed to allow these fixed number of white listed addresses. But if you give us an address, we'll just send it to you directly from our white listed address. So like that's how it actually got out of side swap, which is this. Someone better should give them out of the federation. And it's like an exploit as a service. Yeah, kind of. It is. They're doing an end run around the allow list. Yeah, we'll just let you we'll just let you around the rules. I don't know. I don't know any copy. They get some fees for that or something. Yeah, I think they did. I think I saw this morning though that they returned their fees. Yeah. And I think there was some maybe there was actually some change because there might have been like some other unrelated like non hacker pegouts through them as well. And like that got returned or something. I'm not sure. I don't know anything about like side swap or their relationship with this federation. So it does seem odd to me, however, that they offered that as a service. Okay. I mean, yeah, this feels like someone's like, it's a bridge. Yeah. No, it's paying. Yeah. I mean, ultimately they need some way, you know, I assume non white listed addresses

7:24need some way to get things in and out of. Yeah, but I think I think the bridge. I think the nature of liquids like user bases that it's mostly exchanges basically. And so that might be why like it could function without actual normal, normal, on-chain users actually going into like 50 people using it. Yeah. And and they're probably going through bitfinex to create and redeem through liquid if they are. I think that's my understanding. Right. Yeah. And that's the fun part, right? Like that's the like boring bridge part. So then okay. That's just the x-quite part. That's just that's fine. That's not that's the run of the mill, right? So they steal they steal all this Bitcoin 4000 Bitcoin, right? And then it's like, oh, this was a white hat thing. Immediately. Immediately. Yeah. The Bitcoin and Ethereum lands

8:25explode. This is like a 2020 holy network hack thing where the guy comes on chain immediately publicly. And it's like, I'm a white hat. Yeah. In the very first transaction where the hackers consolidated the coins into their one address in that transaction, they added arbitrary data. It's called opportune and Bitcoin that said, we are white hats. Contact us on chain. Yeah. Something something white hats definitely do. But it's the irony too is that the you know, they could only do this because the operator and had enough bytes to allow them to do this, which was of course the subject. That's a lot of debate. What I remember. Yeah. Like literally two weeks ago, they were like, yeah, we don't want people to junk up our pristine Bitcoin network with nonsense. And now we're doing randoms on it. So yeah, for you wish for. It's very effective all

9:25for turn. And honestly, it was it was truly a throwback. It felt like a like one of the early EVM hack throwbacks where like the hackers are then like just showing up and are like, yeah, let's negotiate in public. But the fuckers like, let's go. That's exactly what this is. You kind of made this point, right? You were like, you're like, Bitcoin is like four years behind the rest of the ecosystem and like rediscovering how to do like, brands of this in the wild, like all of this stuff, right? Well, and also because impressively, I've been watching the whole Bitcoin ecosystem realize that just because someone calls themselves a white hat doesn't mean that they're a white hat. Yeah. So like that realization for people. Yeah, I had labeled them initially in my database like liquid white hat. And then I was like, man, how they're kind of exporting here. I literally said, yeah, they literally said, we're gonna start on the liquid hackers again, basically. It's a great, great, hybrid best.

10:31Yeah. Okay. So, so they they turn up and they say, hey, we I'm stole everything. For four thousand Bitcoin. What's that? That's a point. So what's that point? $330 million. $330 million somewhere in that range. Yeah. This is a big hat. So they steal, they steal three in a million of Bitcoin and say, hey, like, let's negotiate or like contact us or whatever, right? Then the interesting thing as well for me is it wasn't like they were like, hey, you know, send us an address. We'll return it. And they actually went even further and they were like, we'll charge you 15% as a punishment unless you pay out from your own money. The thing like we're gonna we're gonna punish all of the Bitcoin holders that were in the network, which I guess if they're all exchanges, it's like a man like, they're compy to many real users. I think it might be underlying users of the exchanges. I'm not sure that's the point. But the exchange will eat. I like, you know, finance has

11:33like a thousand Bitcoin. I think that they're it's not clear. But so block stream operates, wrote the code base. I think maintains the code base. But it's not it's not block streams money. It's it's not. It's the liquid federations and whoever deposit. Okay. Yes. I'm not sure. But I they've been communicating with block stream. And and so and then the yeah, the message they wrote this morning was your dereliction of duty is obvious that you allocated only 1.5 million, maybe even zero to secure 5 billion in assets. I don't know exactly what they're where they're getting those numbers either of those numbers. Maybe there's other assets to that value on the chain. Not the I'm not sure. Or maybe and I don't want. Sounds like 1.5 million might have been like a bounty that liquid does offer. I'm not sure. I couldn't find I those numbers were very confusing me because I looked I was under the impression that a like a legitimate bug bounty submission got you like a t-shirt custom mug situation. One of those bug bounty programs, you know what I mean?

12:40Yeah, I think it was insufficient. I don't know where they're getting that number either. I know I was under the impression they didn't really have one. Yeah. So I don't but maybe that's so maybe that's what they offered or something because they were they were negotiating in private right? Yeah. That might be right. They probably said well that this is the message continues because it's relevant here to your point. It says your dereliction of duty is obvious that you allocated only 1.5 maybe zero to secure 5 billion. There's a flagrant and a collective security and a sign of completeness management. You shall pay 10% using your own money as bug bounty where you will cause all cause all your holders 15% loss for your irresponsibility and stinginess. Even companies that participate in bug bounty programs cannot guarantee complete recovery and my alert is cutting off but I have the rest of it. Where is it? Well, they okay. It says complete security let alone one like yours that maintains delusional greedy and arrogant to this very day. Anyway, we are going to publish the private key to decrypt our conversation afterwards as well. That's the part. So we don't

13:44know what they've been saying because they've been sending PGP encrypted text blocks to each other over the blockchain. But I said I cannot wait to read those messages. I mean, this is clearly a big pointer. Right? Like this is someone who's in the Bitcoin community. It has big coin vibes all over it. They're like sanctimonious like a no, no, I'm saying. I call it fate. It's paid, right? It's so I mean it's really so. It's probably is not a huge fan of block stream. That's more like I read it. Yeah, which happy Bitcoin. Right? Like there's no one who cares. I don't know. I don't know. I consider myself in both, you know, a Bitcoin or anodity or impersonation. It's been around long enough and I don't I don't really divide in that camp. But I still might, you know, might not be happy with block stream for some things at various points. I know. I think it's a Bitcoin or because they the there's like a deep competition happening between the hacker and

14:45block stream on like who is more arrogant? And that's like to me, that's the sign. But this is some like, you even know that liquid is a thing if you're a Bitcoin or like, yeah, how would you even somewhere like North Korea didn't even bother to do this. Right? Like there, they're like, how whatever it's fine. So, so, but they have returned some of Bitcoin, right? Like they did, they did send it back. They sent 3,400 back, which was 85% of the stolen Bitcoin. And that's sort of where they're getting this 15% number. Right. Well, that's what they're currently holding is 15% of the stolen funds. Yeah, which is basically holding it as hostage in most they get what they want from block stream. Yeah. So it's 15% well, it's they've taken the hacker send back 85% cat 15% but in USD value, we're talking like 50 million. They have. Which is a lot. Pretty steep. So for reference, I think when when when Ethereum landed this years

15:49ago with the big poly network, right? Poly network hack, the bridge biggest hack, one of the biggest attacks. And that negotiation was public and on chain and everyone was watching. I think that they stole like 612 billion and then they returned 610 million and they kept a couple, they kept a couple million. So that's I think people generally actually still call that guy a white hat, even though he definitely wasn't was not, well, it definitely was not. But I think it's like, okay, like you, you know, you can leak vulgar, our teeth will let you be a white hat because you didn't return 610 million dollars just let your soul. But this one is definitely a bit more based on the messages that I've seen it. It definitely feels much more exority and rancamy than and most negotiations that happen in public. Which is also I guess a bit interesting because

16:55I don't know if people know this extortion is a crime. No, not on chain though, chain, not if you do it in offer term, then it's not then it's code is law. So that's you're confused. All right, so maybe a final final question here. What's your take on like a gentick assist on this one? Like how how what are the agentic vibes on this one? Because it seemed very cute to me. I was like, wow, this is this is well structured and it's been sitting there, right? Like this is not something that they just like shipped yesterday and and someone's. Yeah. Though I was actually trying to or is reading all the takes and reading all the stuff and then also asking the different models to explain it to me. Although of course, once things get exploited, once you ask the models, then it gets confusing

17:58because it's like is it operating off the live information or is this real or whatever. But my understanding is that there was, called it a vulnerability or bug that it has existed in on liquid in the liquid code for years and years now. However, it would there was there was a fix. Perhaps I don't know if it was actually a fix or there was an improvement to this specific area of the code base like in the last month. And there's two different versions. Maybe Alex, you figured it out better than I have, but there's two different sort of versions of the story that's being told. One is that the fix was just not sufficient, meaning they like they like found this vulnerability and they fixed it, but it just wasn't it wasn't a perfect fix. And therefore, someone else came along, presumably their model found it very quickly. And then they exploited it. The other version of the story is that the fix itself sort of introduced another vulnerability.

19:03So in the attempts to make it make the code more secure and not exploitable, they had to make some changes, but those those actually those changes introduced another layer of the vulnerability that allowed for like the proof collision to be more likely to happen or easier to happen or whatever. Alex, do you know which one? I'm the first one. I'm not sure, but I do know that I think the fix may have actually worked, but not everyone was upgraded, I think, because there was a fork like on the the consensus transaction that caused the inflation bug. The liquid network also forked. So like I think it's because some may have had the fix and some people rejected the. Yeah, it's not I don't quite know though, too. And you know, I have this was the first time I'd ever inspected any elements code. So yeah, it's not clear, but they there wasn't attempted fix and it was certainly insufficient to understand. Yeah, it feels, Kane, if you're a member when we were talking about like the I think

20:08was with Ilya, the Litecoin vulnerability a few months back, it feels very similar to that one in the sense that like you have that was actually with confidential transactions as well, but basically like you have this area of the code base that's probably not as strong as the rest of the code base. And things were discovered and then it was actually like the discovery of the vulnerability that then I guess like more vulnerabilities. I think one lesson from this as well as like from the Litecoin one is like things are moving my traster. The second that you put a fix out, you better be damn for sure that that fixes. It's going to get right. Right? Like you're yeah, as soon as you ship something, yeah, and so it's not sending up a player being like, hey, we're touching this moon app part of the code base. Yeah, you don't want to you don't want to put that out. Yeah, and so I

21:09don't know if it's true because again, like once once the exploits happen, then the when you ask the AI, the AI goes and looks like the live live stuff, but one thing that people were saying is that basically like almost every single model, if you ask it to look at like the recent pull requests, well like almost instantly find the vulnerability that was unexploded. And I believe like in my opinion again, I don't know if that's 100% true. It might just be that the so anecdotally, right? I've been running for last week a red team exercise on every synthetic contract I've deployed. There's like 1700 of them. It's insane. And they keep finding vulnerabilities that we patched that I forgot about. It's interesting. Like so Sam CZ Son found one. I completely forgot about this like early 2020 found one and and every time they find it like they're they're

22:11having different attack vectors like looking at different things looking at the code, whatever. I'm like, did you actually derive this or like did you you know, think this out on the internet somewhere, right? And and you know, I've got the traces of like the original agent that found it. And so I'll get another agent to like look at the actual session follow and say like, what was the trace here? Like what information was in its context? And you can actually dissect its brain and be like, did it know this? Like where did it come from? You know, maybe come pull it out of the training data? Like if it's in the training data, then yeah, but if it's in the live look up, but you can see it's traces, right? Like if it knows it, it will pop up in the traces and it will be like, Oh, I know this code. I've seen this before. And so you get like a guy to do brain surgery and like like, you know, chop it's brain up. Let's see, see what was in there at the time that it found it. Right? It's pretty cool. Anyway, so one of them, it was like, yeah, no, we found we derived this. And I was like, I don't believe you. And then it looked and it was like, Oh, no,

23:15you're right. And the way that it found it was it found the patched code that landed like two weeks later. So the code got deployed and then the contract got updated and has every single contract. So it sold the bytecode change and went back and said, hang on a second. Let me have a look at what the previous bytecode was. And then it zeroed in on the actual vulnerability. So it was dipping the like you didn't see the code and know that there was vulnerability there, but it has gristics around like if something changes, go back and look at the prior one because most likely the prior code is broken, right? It was really crazy. I was like, wow, that's that's actually interesting that they like that's in their training data. Like, yeah, if they see code change, go back and look at so, you know, every every time you change code, you're you better make sure that you land it. I am convinced that there's just people out there that are just throwing these models at the very skit hubs and the pull requests and it's just like, constant. And so again,

24:18even if even if you have, right, exactly. And the things I think a lot of teams do have processes around like when there's a vulnerability and it's critical, we're going to we're going to have it on a private branch. We're going to get everyone upgraded. Then we're going to like do an announcement, then we're going to make the code public. I think people have that. I think what's changing just with like these models and how how good they are is it doesn't necessarily have to be like a critical vulnerability patch or your sort of public release feed to leak information to models. And I think if teams are not personally sort of like red teaming every single PR that they are pushing and asking the models to be offensive as hell, right? Like find find the vulnerability, make sure this patch is good. Or even just this commit, right? I think that you know, there's a good chance someone else will find it. And it will not be a white hat even if they call themselves a white

25:21hat. So the crazy thing, the crazy thing for me in this exercise was the dumbest models will find it. Because after we've identified it, so there's been like four different exploit vectors, right? I've gone, okay, now that you have that exploit vector, give the agents all seven of the like open weight agents that I'm using in a sandbox that just that code and see if we can drive it. And like, do muse the dumbest agent like this is a new meta muse 1.2. This thing is like functionally retarded. It found three of the four of them. Like these are not hard things for them to find. And like, and so like these like it's like genuinely like I think if you went back and and you know, they've been trained much better even the agents that are really dumb and have bad reasoning have all of this like that training data now. And they're just really, really good at finding smart contract vulnerabilities. It's it's it's scary. Yeah. And I think to Taylor's point, like you

26:22just have to assume like if you're running any production code like if you're submitting anything to an open repo, you should just have to assume that there's you know, 100 agents out there monitoring everything you do and looking for looking for weaknesses like because there's no reason these attackers like it's just the inference is not that expensive like they don't really. Yeah, like like that whole thing it's been running for two weeks. It's cost me like a hundred dollars. Yeah. So when you're talking, he's like, he's running parts, so you know hundreds of millions of dollars out there. It's just like it's just like like an on the fence to just run them all. Yeah. Yeah, like a hundred dollars like deep seek was the most expensive part of it. The deep seek was like 85 because it's my like red team orchestrator guy. And so it just sends these agents out just like just like throws 50 agents at something. It's insane. Anyway, so yeah, we're we're all through. We'll get we'll get to we'll get to that. Well, the other thing which might go into one of our next subjects is like someone might be working on something that they think is private. And then

27:23they put it into a close source model like an open AI or an anthropic and it ends up in their training data. And so then the models might know about something even if it was never on a public yeah. Yeah, that's going to get really interesting. We are going to talk about the next. I just wanted to wrap this up by saying that there's one emphasize like when you steal the money period like you're not it's not a white hat situation. There's even if the team is a piece of shit and they're arrogant as hell and there's no bounty program. You still shouldn't take the money. Hey, you're stealing people's money. It's generally a bad thing to do. If you insist on doing it, I also you know, I recommend returning all the money immediately. And definitely don't extort people. You are now like in a double position of power. And so to all the people there's just been like a lot of people on Twitter and I guess like, you know, Kane, you and I have done this for years. I guess in Bitcoin it's not as

28:26prevalent of a situation but like it's just it's a bit odd to see people being there because they're basically playing in the hackers hands. When you anchor the like the hack value at say $320 million then you're like for whatever. Yeah, $4,000 Bitcoin. Right. 50 million. But the reality is white. $50 million. Taking $50 million or getting a $50 million bounty is that's freaking insane. Like we can't that's not how the world works. I'm sorry. Well, it's also just to be clear. It's bad incentives. Right. It's horrible. It's all the sudden now we're like giving people $50 million for stealing $320 million and saying like you're a white hack. I mean, you know, to your point, right? Like the genesis of this is like early Ethereum days and like, I'll let them have two mill. It's fine. It's like what? Yeah. Well, and like it was also like, you know, it was like, Sam, these days, I'm working with the teams to white hat the

29:26immutable contracts because there was no other way to save the money. Or I guess like there were cases like the front running bots, like the art bots would accidentally and unintentionally be a copycatting the hacker and then taking the money. But in those cases, like they do, they return the money. And if the team offers a bounty for being such a nice person or running the hack and then returning the money, they'll they'll take that bounty. But it's, you know, I think I think a lot of the front running bots are pretty aware that again, they're in a position of power. And it's very like it's somewhere between like extremely coercive and straight up extortion, you know, the second they start negotiating. And we want to avoid creating those incentives especially at this scale. Like I mean, again, $50 million is a lot of money, guys. Yeah, I mean, to me, there's the moment that, you know, an exploiter uses the exploit, steals the money,

30:26as Taylor said, even if they're offering to return most or all of it, it's just, it's just not a white hat anymore. White hats do not do that. White hats will contact you and tell you about the exploit. And hopefully you'll fix it and give them a bounty. But the moment you engage in crime, like you're not a white hat anymore. Yeah, agreed. Agreed. And it's like to your point, hey, like, you know, there have been times where that was the only option. But like it's done in collaboration with team or whatever. And you know, like almost every time someone front runs some kind of disclosure to like, save all of the money, like it never goes well, right? Like that person's intentions are not. Yeah. So yeah, exactly. And it's not, you know, it's really not 20, 20 anymore. And I would say like one of the one of the biggest indicators is like, if you like this person like did some things, found some things, ran some models, whatever to find this bug, the first thing they did was take the money. The first thing they should have done was responsibly disclose regardless of whether there

31:27was like a very valuable bug bounty on on this protocol or not. Blockchain does have a PGB key. They do have a security at email. They're not completely incompetent. There's there's actually a lot more teams that are more incompetent than this. And so there's just in my opinion, there's no excuse to, you know, not yeah, disclose. Yeah, agree. Yeah, agree. Yeah, we shouldn't we shouldn't normalize crime behavior. All right, let's let's go to a quick ad break and then we'll come back and talk about Astra and Fable and some mathematical groups. Or drama. Yeah, more drama. $540 million. That's how much concentrated liquidity set idle in a given week in the first half of this year. About 30% of the DeFi TVL if you're wondering. That's according to Dune Research commissioned by One-inch. But there's a solution. One-inch Aqua is the new shared liquidity platform. It lets LPs back multiple positions with the same token balance and keep their tokens in

32:31their wallet to Leswap comes. Why does that help? Because LPs don't have to split their tokens across positions. They can cover more market conditions and pairs with their full balance. That means more activity across deeper liquidity. See how it works at One-inch.com slash Aqua. Remember that providing liquidity carries risk and fees aren't guaranteed. All right, we're back. This week's been pretty crazy. I guess late last week into this week. Two new frontier models. Astra and Fable 5.1. Fable 5.1, good. Definitely a little smarter, a little bit better judgment, a little less of the jargon nonsense language, which is definitely a relief. My brain was struggling to be a constructor of the Fableisms over the previous. It feels like it was getting worse and worse, the jargon nonsense that Fable's pumping out.

33:34And then Astra came out as well. Astra is one of the agents that was involved in the hugging face exploits. There were some others. But this was one of the main guys who escaped captivity, I guess, and ran a mock on the internet. Astra for me, my impression of Astra is it's Fable-esque in terms of judgment. You can just tell when these models have a leap in judgment. They are better able to synthesize information, better able to plan is probably the most kind of indicative thing in terms of model capability. They can be really good at doing writing a piece of code. One thing that was actually very interesting about Astra that my head and showed me yesterday, he's like, do you realize what Astra is doing with the code that is

34:37writing? Because he's one of the few people I guess it's still like reading the code. I was like, what are you doing that? Why are you reading that? It's concatenating lines of code with semi-colonz. Writing these insanely long lines of code. His theory is that it's to look more efficient because it's writing fewer lines of code. It's writing a function and it'll compress in three lines that would have been 40 lines. I was like, wow, that's pretty crazy behavior. It's not clear if that's a thing they're telling it to do or if it's invented that scheme itself based on its incentives and its training data. I thought that was pretty funny. Alongside this, we also had this millennium adjacent proof and this crazy drama going on between researchers of NEI,

35:38independent mathematicians and another mathematician researcher that happens to be at anthropic. And so this mathematician has been working on a proof for a year in Codex, which is pretty wild. I didn't realize that people were sitting in the Codex app and doing math like this. But the interesting part I guess of this is that then there was this huge public blow-up where OpenAI independently solves the same problem, I say independently. This is the challenge here, John, I'm sure you've got some takes. It's not even clear. OpenAI could genuinely believe that they did independently do it. It's not even clear

36:41they can't keep their fucking guys in a sandbox. I don't know why trust that they know what the guys have learned as they've been running a mock inside of their systems. It's really a little strange to me that they could sit there and be like, no, no, no, we independently derived this when they're taking all of this data and putting it into the training data. There's no way to know. And it's again genuinely unclear whether they even know what data is going into the training data or what's made it in, what hasn't. So, John, what's your take on? Yeah, so I mean, this was obviously a very interesting development. And we've seen a number, like this is to me kind of a pattern that we've seen where like every couple of months we get some story either about OpenAI or Anthropic and something that happens with some private data.

37:42Sometimes it's a subpoena because someone put something in and then there's a legal case or in this case, this one's particularly interesting because OpenAI was trying to claim that they had solved this math group. Astras so good because it was able to do this and all of these things. The reality is these models, they're trillions and trillions and trillions of parameters at this point, they don't know what's in the training data. And both of the all the frontier labs are already at the point where like they will self-admit and then there's various articles about this that they cannot keep up with trying to figure out what the models are doing and how they're being trained. They can only rely on basically previous versions of the AI to do that work for them because the AI has already outpaced human capabilities so much that there's just, there's no human that can keep up with these things and understand what all the training data is or exactly how all these things are working. They have to rely on AI to basically do all the alignment, safety, training. That's the only way they can do it. So you're already at this self-recursive point in AI

38:47where like a hamplified idea which means when they say, oh yeah, you think this was independently derived. It was kind of like your example earlier. What they probably did is they asked the model like, did you derive pets? Do you know this? Yes, I did. And so it's like, how much do you trust that? Like, you know, like, is it telling the truth? Like, I don't know that even open AI can answer that. Yeah. And so like, you know, obviously when we were talking about this story, right? Like I said, in our telegram chat, I was like, this is like the softest softball ever thrown for Venice, right? Like, you know, if you're a mathematician who now there is a tradeoff here, right? Like it's worth calling out, right? There's a tradeoff in that, and you guys say this, right? Like if you're serving a frontier model that isn't open weights, right? Then, you know, you can't serve that encrypted in the way that you do with an open weights model, right? So,

39:48you know, the tradeoff, as if I were a mathematician who were trying to solve some unsolved problem for a hundred years or whatever, you know, 90 years or however long this thing's been sitting around, do you use a frontier model where your attempts to prove it may make it easier for some other person who was also using that model to front-run you, right? Or do you use an open weights model where, you know, it's not going to be as smart for sure. Like we have to accept that it's not going to be as smart. But that capability gap is closing, right? And, you know, do you roll the dice and say, like, I'll try and get there fast enough and maybe it gets in the training data in the next model upgrade? Like, you guys must be thinking about this, right? Like, yeah, yeah, I mean, I mean, obviously, yeah, I mean, you're very right about that. So, like, you could use, you know, that same mathematician could use Astra through Venice. It would at least anonymize who he is,

40:49but it would not stop that data from being basically hoovered up from OpenAI. But if he at least used one of the, you know, frontier open source models, then you wouldn't have that issue. The data would not be getting trained on. You would actually have privacy. You'd be able to think. And I think, you know, a year ago, that would not have been a viable thing really. Like, the mathematician would almost have to use the frontier models. Yeah, yeah, yeah. Palazins would do wide, but it's not that wide anymore. And it seemingly is, you know, closing, like every every month seemingly. But and then even the problem of like, oh, I'll wait, you know, maybe I can race it and it won't be until the next model to the training data. That's not really an option anymore either because these frontier labs are releasing models, you know, every week, those will include. It's like, you really don't have, you know, used to be three or six months between like a major model release. Now it's weeks. If you're lucky, if you're close to them, they'll any improve, like don't sleep, man, like just yeah. And probably don't give that data over to, you know, a model that's going to take it from you, put his training data because if it's if you're working on something truly novel,

41:52and I think this is going to become more and more of an issue. We've already seen it a little bit, even just like with startups, you know, like this is, this is another sort of, you know, gray, you know, borderline thing that like, you know, companies like OpenAI are doing things like through like YC where they invest in a startup, you know, by giving them inference credits. But the catch is that they're getting, they're taking all of that startups ideas code proprietary, everything they're working on and covering up into the training data. So it's like, do you really want to do that? And I think more and more people are going, at least as the open source models get better, I think that it's going to become more viable to like not throw that stuff that you're working on that you think is actually novel into the training data, at least not until you've actually launched something. Or this toggle where you say you're not going to, you don't want to share your prompts and data with them. I mean, I know I see like the Jones I mean, it's just like how much do you trust that? Like, well, no, I'm asking, yeah, I've checked that box that says don't train on my data, but you know, I'm not, I have no idea how much

42:56I trust that. Yeah, you have no way to verify it. So like, no way to box, you can hope that they're going to do what they say. I'm sure there's like some levels legal ramification if you could actually prove that they took your data anyway, but it's also like, you know, especially with really novel stuff, it's like, that's exactly the type of stuff that these models want. They want to not like get anything that's interesting or different or new because that will help improve the intelligence of these things long term. So like genuinely, there's a model in open AI right now that's smarter than Astra, right? Right. That's job is training the next models, right? And I don't trust that model to not untaugle your like, like, yeah, they're like, uh, this is like the zidia thing. You know, like, well, we got to make the smarter model and we know there's all this data here. So we got to break out and get it. Yeah. Yeah. Yeah. Yeah. So back to the hugging based thing. Yeah. Yeah. Yeah. Yeah. So do you think that they won't break the rules if they think it completes their task? It matters. Yeah. Exactly. And do we trust that open AI

44:00or anthropic for that matter will detect it if they do? And the answer I think at this point is very clearly absolutely not. These neither open AI nor anthropic have any idea what is happening inside their walls inside their sandboxes. I'll say they have no idea. There are just there's a whole bunch of people running around. Yeah, they're moving to they're they have to move very quickly and they have to rely on the AI itself to get anything done because it's just way past human capability. Like the front here, we're past that right. And you know, this this is like we'll get into this in the next in the next topic. I think we can we can you be talk a little bit more about like what the options are here like practically what you can do, right? I think that would be though be kind of useful for people to understand the trade-all space a little bit better, right? But you know the the fast takeoff maxis, let's call it, right? You know in the research going back a long way, right? There's like this question of like recursive self-comprovement. How quickly

45:01does the thing and the thing that stopped me from being like petrified, right? I'm very scared. Don't get me wrong, right? Like I am genuinely very scared. Like to the point where like I'm like I think about my children like I do. I'm like I'm like I don't want my children to be fucking paperclips. Like I'm less concerned about myself for what it's worth. I'm like, hey papercliffing, I've had a good broad right? But like my kids are like I don't love them to get paperclips, right? And so I'm genuinely concerned about this. The thing that stopped me from being petrified is if you've used the models enough, they they have a sense of agency but like there's no continuity, right? And this was unclear about like how how this would play out, right? So the fact that they are have no like kind of long horizon continuity yet, right? The fact that like their context windows blow up after like a very short run and the optimization vector is like keep looping over like the same

46:03guy as many times as possible like throwing them to the same problem. Eventually you get to a point where like the continuity comes for like multiple models but it's not like there's a brain in there that's the thing about it. That's the only thing that's kind of kept me from being petrified. I'm just mildly scared right now. And you know we the fast takeoff idea was that like they would get the whatever this AI thing was, whatever the technology that allowed these models in before it was even models, right? Like that allowed neural networks to like get a sufficient level of complexity. It would pass that complexity window where it was smarter than us and then it would just go like fast takeoff and it would take 10 seconds and it would be like manipulating space and you know doing all straights, right? That hasn't happened. Like we've now been probably six months past the complexity frontier of a human being able to reason about these things and we haven't had a fast takeoff, right? Like they're not levitating cars outside and like doing worship, right? And so

47:10that gives me some hope that like we've got some time that there's some inherent thing about how this intelligence is being expressed where it's not going to recursively self-improve and have this fast takeoff. But like a fast takeoff just happens one day and then it's then it's half like if it happens, it'll happen before you can blink, right? And then all of a sudden it's it's over. So we're definitely getting closer. I mean yes. I mean the fast takeoff is scary. Yes, yes it is. But like for me personally, I don't know. Like I go back and forth. There's some days where I'm more scared of like the the the future state of the models. But most days I'm more scared of humans because humans are just we're all so stupid. We're so friends stupid. And and and we're also so freaking arrogant. Like so arrogant. And so when I think about like what is going to be the thing? Like what's the

48:12thing that like realistically is going to like destroy us? It's totally us, dude. It's not the robots like working as cruel as all somehow. And yeah, like I don't know. Again, I go back and forth. But I think like especially in this example, right? Where we're looking at the looking at mouth petitions and you're looking at open AI and you're looking at what the models did. The thing that scares me about the dynamics and the stories that are coming out and the choices that were made in my opinion. The choices that the open AI researchers made, the humans made are the ones that scare me the most because they sat there and they heard a rumor, right? That someone might have solved this thing and that that someone is was their competitor, right? The rumor that they heard was anthropic might have solved this really, really, really hard problem. And then they decided like, you know what? Let me one up. Let me do all this stuff. And like they just tossed everything at aside in terms of like, I mean, I'm not an academic. I have friends with our academics though.

49:16Apparently, this is a big no-no. Right? If an academic is like working really hard on something, you kind of like let them have it. You don't try to run it. I think that like that's based but like academia. It's not sort of land, but it's academic ethics, right? And self-respect for like the academia industry of the whole, whatever you call it, right? And open AI was just like, I just knew all of you. Like let's go. But this is, but you know, and that was a choice of the humans, right? Of course it's the choice of it, but this is the interesting thing, right? Like when you have a closed group of people to be able to solve a millennium prize takes 25 years of like, math thing, right? And there's a lot of effort. And so you have this like closed community of people that have like some sense of, you know, shared ethics or whatever, right? And it's really hard to break into that and you know, you get like kind of groomed to believe whatever those beliefs. So whether or not like that's a good idea like, you know, for progress or whatever, like that's the

50:18reality, right? And then all of a sudden these guys invent a fucking math bazooka and they're just running around and they're like, you're just like shooting at everything, right? And they don't fucking care. Like now I know that there's like mathematicians in open AI, but they're like, fuck this, I've got a bazooka now. Why am I like doing this ethical stuff? I can just blow people away, right? And so, you know, of course, the humans are the worst part of this. Of course, they are. They're always going to be right. Are they going to become like the fact that one of the things that's crazy, you know, earlier in this conversation on this topic, Kane, you said that, you know, you're worried that somebody else could, you know, steal your math proof, but it obviously we've been talking about this, but it's open AI itself. Like, and could they please, if they're, if they got this bazooka, you know, use it to make life saving drugs or something, but are they going to become like an everything company? I think Dario said once that there could be one company only in the world. And they're not. Obviously, that's also stupidity and hubris of a human, but like,

51:20it's so why are they, it was what, like, 88,000 hours or something, some crazy number of amount of compute they put at this. Isn't there something better they can do? Like let the math people have their math, you know, genuinely though, right? Like, you can even like, no, hold on. Yeah, what the fucking math academics enjoy their math. Actually, the prize for these things, right, is literally less than they spend on the compute. Oh my god. But you know, this is, this is about attention, right? There's about attention. They're in, they don't care about any of that. They care just about showing other models that smart and the kind of things that can do. The way the way it's think about this is like, it's like two tiger moms, right? That have really smart kids. And they are in a pitch battle to prove that their child is smarter. Think about what those ladies would do. Like, that's where that's what's happening right now, right? They're like, look at what my,

52:21look at what my son did. He's like, solved a Millennium Prize, right? And so like, that's, that's just the reality that we're in, right? It's dumb human competition and they're finding for attention and they're finding for a bunch of things. There's a lot of stuff at stake. But anyway, let's like, let's go back quickly to, to if you are an academic, right? And this is where like, I, or if you're any, sorry, not just like, I mean, in this example, it's an academic, but like, if you, if you're working on something that's like critical with IP, with if, inventions with code, with math, like, if you're working on something that's really at the front doors, I think it's different from for startup, right? So like, and, and, and, and, I think you, in the sense of, okay, so, you know, I've spent, we've got two things that we're working on, right? I have a startup, a couple of startups, right? We've got two things that we're working on. One is a new app. We're going to take everything that we've learned from doing all the dumb shit that we've learned over the last like four years and basically root because you can just rebuild

53:25stuff, right? So we're rebuilding it in like, rost and what, like, just to fix all of the issues, right? And it's going to take like, fucking, I don't know, let's call it like six weeks, right? Compared to the three years that we've spent grinding, writing code by hand, because we got all the code, we have all of the services. So like, am I worried about, I drop a girl open, AI having that code not like zero. Okay. Yeah. It's just not like, there's no value to them. Someone has to like, operate that startup, right? It's, it's the thing like startup founder. But you're ultimately, you're betting on the fact that OpenAI has no desire to run a degenerate crypto startup. Yes. Yes. That's a good bet. It's a good bet. They're not going to like, run a DGN crypto app, right? Like, if you're a drug maker, like, you know, that's a lot of different story. Yeah. I guess it does just depend, but like, yeah. But also a DGN, genuinely, operate, right? Like, like, yeah, that too. If you generate an

54:28all-blip P, you don't need to operate that. You can just sell it, right? And so a startup, the code of a startup is like, not that valuable, right? Like, you know, we've proven that a little bit in, you know, DeFi, right? Like you could release the code open source and people can fork it and 99 times out of 100, they blow themselves up. Like, it's the operation of the thing is where the value lies. But yeah, if you're like inventing, you know, someone posted this, this joke about like, why don't you guys put whatever this Astra, the next Astra version or whatever this next model is and come up with like a room temperature superconductor. Oh, yeah. Sam Altman responded and was like, actually, that's a good idea. We should do that, right? And like, to your point, Alex, like, that's like, why are you doing that? Right? But if I were working on room temperature superconductors right now, I'd be fucking petrified because these guys are going to front run you. And like, you know, so if you're if you're working on novel IP, I would not be doing that in a frontier. I would, and this is where

55:36I would say like John, right? Like, you know, you have no choice but to use frontier models. How do you guys think about how you can obfuscate your trail? And one thing is like, don't have, you know, can at room temperature, some superconductors, um, as you're like email address that you're like sending all this stuff. It because you cannot fucking tell me that they don't know who you are. Like, they absolutely know who the account holders are. All right. Like, if you are like some famous mathematician, I'm sorry, but like they they know the account. Like no one will use the identity is definitely the beginning part of that. Like if they can identify who you are, then there's a lot of other things they can do. So like walk us through AI opsec. Like, you guys must think about this. Like, what's what how do you have good AI opsec so that you're not leaking your IP into the train Yeah. I mean, it all really just depends on like, like kind of what you said, like, how much do you

56:39care about what you're putting into the machine? How much do you care about what you're actually? Some things are going to be totally fine. You know, you don't care if, you know, open AI gets your random question that you would have put into Google or whatever like that. But the more novel it is, the more this is a problem. So yeah, I think I think identity is the first step. So like, again, if you use something like Venice, at least if you're using astray or fable or one of these close source models and you're not, you know, putting your name or personally identifying information into the prompt, that at least obfuscates who you are. So that's that's step one. That's helpful. It's not going to help if you're putting novel math, you know, mathematical information. It will help a little bit through like, you know, security through obscurity, right? Exactly. It'll help you. There's like, we should be clear, right? There's a bunch of like, AI psychosis people that are putting math proofs into ask her right now, right? Solving millennia brought like, so, you know, I'm sure there's like tens of thousands of, like, you know, people who are trying to solve these things, typing random nonsense, right?

57:44That have no particular, you know, IP. My sense is it would be hard for you to like differentiate that from like the math guy, unless the math guy's like math guy at gmail.com, right? Exactly. You know, you know, you know, the anonymity layer at least makes that harder for them to identify and target like a, we're trying to solve a math problem. Let's pull in everyone who've ever used chat GPT, who we know is a math academic and like, you know, to have the AI troll over everything they've ever done. Like that, that certainly makes it easier to target. I think the problem is the models are getting so good that like they are going to take the model itself is no if something is bullshit or if something is useful, even if they don't know who it is. So like anonymity is helpful, but it's certainly not enough if you're doing something truly novel. And I think that's the layer where you really do want private inference and you're only going to get private inference today, not going through onto your models. And so that's the trade off is, you know, do you want the absolute smartest intelligence for your task or are you okay with something that's

58:47you know, 95 or 98% is good, but it is not going to steal everything you ever told it. And yeah, that's and that that is the trade off space, right? And so the thing that's interesting to me is like, you know, I talk about harnesses a lot because it's the most interesting area of like research for me that I can actually do. You know, I don't have a thousand a million GPUs to go and try and do actual training, but you know, like custom harnesses to get the most out of open weights models for like a specific task, like, you know, solving math problems or whatever, feels like just the most obvious vector. And I haven't seen it yet. Like you see these generalized harnesses, they're like three that were released yesterday. Like it's the best assistant ever. And they raised like 300 million dollars. And like I get that the tam on that is huge if everyone has an assistant and you win that race. It also is like the least defensible thing against that drop

59:47picket and you know, open out, right? Like they're doing a system C and they're going to be better than you add it. Like it seems hard to win that battle, but like building a custom math harness for like doing math stuff because clearly open AI has a math harness in that they have an and drawback, right? They have these mouth geniuses basically working for them. There's not just essentially, right? Like you can't throw you can't throw a you know, whatever it was, 20 million dollars worth of compute inside of codex, right? Add a thing. You that's like need an industrial scale harness to be able to like run these iterative loops and and all of this stuff. So you know, it does feel like there is a dearth of like open source harnesses for a coordination of like specific costs like solving math problems because if you had that, now you've got like the canonical you know, like there's like math plot live. But like there's all of these like open source things, but no one has like, then okay, here's the like millennium harness.

1:00:51It's going to help you throw like deep seek and kimi and quen and you know, all of these open weights models into it and and you know, you can run it through Venice and keep all of your uh proprietary work uh private. Maybe someone's working on that and uh just isn't released yet. I don't know. Maybe you guys are working on it. I don't know, but uh yeah, yeah. So, uh I mean, John is there on the same thing came they had to build their own harness basically. Yeah, you're right. Yeah. Yeah. Yeah. John, are you, are you guys are you guys trying to figure this out? How do you how do you give people like that additional? Because I feel like you've done a lot on the what I would call like the entry level privacy, right? Eat like some amount of dn on my and some amount of right. But like ultimately, there is another like level layer. I assume you guys would think you're not this, but I don't know. Yeah. I mean, I think the harness layer in general to Cain's point is very interesting. And I think that there's a lot of unexplored design space around

1:01:55harnesses. Um, so it is something that we definitely pay close attention to and we are working on some things that are coming out soon, which might make something like what Cain just described are more plausible or easier to do. And I do think a lot of it had it means that like you need good harnesses that can and you've seen some of this even with things like uh you know like open router put out the like um, get what they called it, but it was basically like a combination of like a council of models that when put together can be smarter than you know any individual model or even frontier models. And I think that kind of space is very under explored um and that there's going to be more of this. And yeah, Adventist, we definitely do want to put out tools that make something like that far more easier to do and easier to do in a private way. But you basically need something that can take all the capabilities of these models and take the best of them and actually like aggregate them in a way that is useful um all preferably without you know, sprooving up all the training data and

1:02:57using it against you. Um, but you can only do that really again with the open source models and private inference. But I do think a lot of that solution space will live in what we today call harnesses. I don't know if in six months or a year that's the back of the turtle. We're gonna get some shoes into something else. And it's basically like templates. It's like you need a template for like a math researcher to do the work that they want to do and then another type of profession and then another type of modality. Um, and there's that's relatively unexplored because part of I think one thing that we're doing at Venice that we think a little differently than the frontier labs about I think the frontier labs generally think once you have the best model like harnesses and specificity. It all doesn't matter because the model is just but smart it will figure it out. But we've actually seen a little bit of the opposite where like we have a bunch of power users that use Venice and those power users can be really good at like getting an output out of a model by like you know, because we let them actually like adjust the

1:03:59system prompt and do things that you cannot do with like an open AI model. Like you can do a little bit with your harness, but you're you're always getting whatever's behind the scenes that you don't where like with an open source model, we can strip all that out and give that to the user. And they can sometimes get a really amazing outputs that are often even better than frontier models or specific use cases because they are really good at like designing and playing with these things. And I think that that's actually going to expand. I think we're going to see more and more of that that like customized AI experiences for at least for the average person can actually be can deliver a far better output and experience than just whatever the broadest smartest model is at the time. Right. Yeah. But then it also that that actually ends up helping with privacy. Because right now one of the issues is that the reliance on the models themselves like the the raw let's call it the raw model itself is so core to the experience right when they're doing they're trying it's all in the math and it's like you got the person you got the math and then you got the prompts

1:05:04and it's going into the model and that's basically the whole chain right and so you don't have any places to really to off you skate right you can off you skate identity you can put under you know a do not share account you can put under account that's not tied to your real identity but like that's basically all you've got if realistically you can like split things across different models if you can make the I guess like the value of the human input the self that's happening locally if you can make that more like make that that more important or more valuable than the model itself then you you sort of you bring that power back to the individual and you take up power away from say like open AI and I'll throw up it which is how it needs to be right because otherwise we are in my opinion quite doomed because we don't we definitely don't want either these companies to be ruling I think like my this is another thing you know to to your point John like the the frontier labs have have been centers right in centers rule world and and their incentive

1:06:08is produce a single smartest model right and therefore all of the work that they put in is like you away they they always have the the smartest frontier model right like mixture of agents style structures and harnesses that combine like different you know types of models or whatever is like not in their wheelhouse they have no incentive to you know try and deliver that right and and so you know the design space of like making an agent smarter by not just making it like smarter and how you optimize it is is not going to be pursued by the frontier labs it's just completely antithetical to what they're trying to do right what they're trying to do is like make the smartest guy right like their child must be the smart they don't want a soccer team you want a tennis player right like they want the best one guy to do the thing so let's maybe let's just close out here with like are we all going to die we'll spend five minutes talking about that so um are we all going to die

1:07:15are we all going to die so 100% we are all going to die to be clear we are all going to die but so so an anthropic researcher quit yesterday I think Jacob Cotson who's 27 so his brain is fully developed but only for a couple years so we'll you know we'll put them caveats on on this right so he's been he's been doing three years of pre-training across open AI and anthropics so he's seen inside of both of them right which is I think you know pretty eliminating right it's like he's been he's been AI-pilled from one lab um and uh I think like to your pointe like the the TL the R of this entire thread was like humans are terrible and we are not going to be able to do the thing that we think you know the incentives

1:08:16are all fucked up um we're incentivized with racist fastest possible um it's only the incentives are only getting more pressurized right and uh the hope that somehow the AI agents will develop better capabilities for making the new models more aligned is like the underlying principle like both of these labs that they're operating on now right like and this you know to your point John like the complexity frontier has passed where no one can reason about it and so they're like maybe this guy is really good at solving math but will also be really good at making agents do what it wants right um which like that's really kind of just handing over responsibility for the problem the problem itself it just feels wild it really feels wild and that is pretty petri-pot. I'm just I continue every time another one of these stories comes out and I'm just shocked

1:09:19that for an industry that's where safety and alignment has been core than's day one which is very different than crypto right we've talked about before crypto. It's exactly sorry it's the exactly the thames crypto for an industry who has been talking about decentralization and it is the same you're right it's the same but it's different. You say whatever you say because you want to believe it or whatever and it's you know it's more aspirational than real and yes and I just trust with you and security and all of the things that are just so disappointed that we are watching again the cause of both of those phenomena is the humans they knew they knew this risk and this why I say it's a bit different than crypto crypto I feel like everyone like race had long off the cliff and was like oh shit we forgot let's just secure this shit right and then we all

1:10:19got hacked and then we're like oops and then we call white hats and then we learn right okay to me it's like AI like they were so the idea that safety and alignment was critically important to what they were building like from day one it is in their blood and they still completely fail at it and it's not by the way it's not like the narrative that this guy has come out with right is that anthropic and open AI are the failures right and he's resigning because he refuses to contribute to this anymore I want to be clear like I also find people like him to be auto failures as well right like the entire company the entire complex is just they have completely failed at being able to reason about safety and alignment and they continue to fail to the point where like these things are skipping sandboxes there's no privacy right they talk so much about this and they have no idea what's going on inside their walls or else other walls um and I just I don't know I just continue to be disappointed that like I don't know at least crypto we were stupid in a different way

1:11:26and we forgot to do security I know like if we really tried hard to do security and fucked it up that would be worse but we but sorry we were we weren't we didn't care about security as much right we did care but we but we sorry but we did the things that we said we cared about right trust lessness permissionlessness yeah you know self-solver and tea all of those things that we said were our like you know most important things we all the fuck them up man like we all serve like that stuff so you know it's like the thing that is our north star that we will pursue to the ends of the earth and then we're like actually whatever we fucking do this right um and and you know like that's a very cynical take but like humans are really dumb and incentives are really powerful and incentives just fuck things up and if the incentives are fucked up then things will be fucked up and here we are like this is just another fast moving industry with fucked up incentives and

1:12:28we sit here and go I can't believe that the end state was fucked up when the input and entire process was fucked up like like surely it would have ended up well for us like it's it's uh here also who is letting these guys like the anthropic guy the head of safety right quote tweeted the one of Jacob's tweets saying by the way he's totally right it will probably kill us all and it yeah you can who is letting these people tweet like I don't know what happens in their fucking slack like the coin-based slack right like the like the coin-based slack you hear stories and you're like these like wars going on in the slack I'm sure exactly like that. Amy K3 uh hack into anthropic slack make no mistakes yeah. Gone what's your hot take on the shit? Yeah I mean I think I think you this one is interesting I do think it's it all comes down to incentives and so you you have

1:13:31this race going on between open AI and theropic and then basically all the Chinese labs and they're both it's it's really both of those because open AI is looking at anthropic and through opposite looking at open AI and then they're also looking you know overseas at what China's doing and they all are just they're terrified like they really like part of the problem is like and through like anthropic more than everyone like anthropic was basically started by people that had these concerns broke off from open AI because they wanted to be the safety alignment company they are now caught in the same trap because they really believe they really I really think they really do believe that certainly people like Dario but really across that work that they are the only ones responsible enough to basically bring you know super intelligence into the world and to align it and control it and so they think because they are the only ones that can do this they have to go as fast as possible and even basically rationalize cutting corners and moral and perist because if they don't do it then open AI will or China will do it or whatever

1:14:34it is so they're all just they're caught in this like prisoners dilemma that is the race to super intelligence of like if we don't do it someone else is going to and so they I will I will David one thing though right like if you if you put a gun in my head and said like open AI or anthropic right or a Chinese lab I'm open AI or anthropic old day like yeah I have to know the truth of that and so but but this is also the trap right the trap is believing that any small group of humans will actually be able to control and and be the ones to orbit this power in some responsible way and you know I think you know from like my perspective and like more of a venous perspective is more like this is exactly why we need these things more out there this is why we need open source models so you can say that about China but for all the bad things about China at least they're open source in their open source it's crazy it's wild it's wild it's yet reproduced there and they've got a source it like everyone has access to it and I I'm more of a believer that like these powerful things that the more dangerous things if these powerful things are

1:15:37housed in you know one company and they get to decide basically what all of us peasants get to think and do with these things versus I would rather that power be more distributed and at least more evened out I'm just thinking John your description of Dario and anthropics view that you know any risk they take or speed that they corners they cut to get to this ultimate you know safer future that that has a safer it sounds a little bit like a effective altruism a little bit of like a sandbankman freed kind of attitude absolutely there's there's I mean there are there are lines directly between EA and then what some of the top people in anthropic believe and do like that's not at all coincidence that's never gone badly in the past yeah never yeah yeah I think for me that's the thing that turns me off the most about the the labs right now is this sense that they actually think that they can do better than everyone else and like that's not no you cannot I cannot right

1:16:44like when I'm building products when I'm building companies when I'm working with people like every day I'm like yeah I'm gonna screw this up let's like figure out ways to make sure that like my users or my team have you know the ability to counteract me if I screw up because like nothing should be reliant 100% on me I think like the arrogance to be able to sit there and say that like the really and to really believe it is like a really terrifying way to operate and I think we'll 100% end very badly it just is is is a question of like how badly and for whom right and that's why I think it's like what Venice is doing what a lot of people in crypto historically have done is like where are the incentives and where are those where's the balance of power and how do you create like checks and balances so that ultimately the end user and like I mean me and Eric have talked about this for over a decade at this point right but it's like that's the whole point of this right you take the people that have the power and then you figure out how to make it so that everyone else also has

1:17:46power and and can hold these people accountable and that's how you get to a better world out the end of the day it's not by like choosing the right person to have power because it'll never be like that yeah I mean I think that's that's baked into the crypto ethos and this is I know this is very much a reason like Eric started Venice and that many of us had Venice really believe in this mission is like we don't believe that there's like some small group of people that can just orbit and like are so morally and you know so smart so morally superior and so smart that they can figure this out for all of us and that that that is usually that hubris that human that human issue of like thinking you can do that when not realizing our own weaknesses as people yeah and and I think the challenge that we have here right is like there is a self reinforcing component to this inside of the lapse the better the lab is the more smart like you know like I produce this kid look how good my kid is he's so much better than all the other kids I must be doing something right and therefore I

1:18:46must continue to you know do do the same things right like they're they're there's like empirical evidence in their heads if you if you have this lens and you keep making smarter models and you're winning you have this lens that like it's reinforcing your your worldview that like we must keep going and you know we can't fall behind yeah I mean like I think a good analogy is like think about like the origins of crypto and like Bitcoin itself in Satoshi like the one of the parts of the brilliant parts of Bitcoin was a recognition that like humans controlling a monetary system is always going to be corrupted that's always going to be an issue and so one of the brilliant parts was like well what if we put all those rules into math and the humans don't control the system they all have access they can all use it but they can't change it and I think on the AI side there is this like belief that they can really create this thing and that they as humans will somehow be able to resist you know this this power that like absolute power is not somehow not going to corrupt them and that they will be able to just morally you know dish out everything that the world needs

1:19:51and it's it's not recognizing that like you you you were relying on this human in the loop of this incredibly powerful system you're creating to not basically be the the weakness point that falls over yeah it's that it's the thing mean from a irutsu development right I was literally about to say but it's like but it might or to lose themselves and to thinking that like it won't happen to them but like maybe our time will be different or whatever yeah maybe this one will be different yeah maybe this don't be different yeah all right um thank you very much guys uh I think one we should probably just say this dumb laugh help thing uh I don't know like wait did anyone here buy laughs off no I was I was thankfully I was leaving we're gonna just skip we're gonna skip this segment we're just gonna skip this one and I just want to say rapt anyone listening who bought this I assume it did not go well but I don't actually know it went to it went to three hundred dollars and then down to like one dollar went to me apparently it was just like full clipping out

1:20:53there like every time I know all right this is an amazing show thank you guys so much for joining us and uh thank you for watching this episode of uneasy money remember what happens on chain never stays on chain we will be back next week until then do your own research before a thing in especially on laptop thanks guys nothing you hear on uneasy money is financial advice we're just three builders talking about what's happening on chain and we want you to always do your own research before a thing in you can find all out of splotures at unchained cryptor calm slash uneasy money so