The AI Social Networks Have Skill Issues

0:00Alright so Casey will not be joining us today for all those that are wondering they replaced him with me. Yep. This is low level learn Are you guys ready to do this you want to talk about this? Let's rip. I thought we were gonna talk about pancakes for a while, but I'm happy you know our long session on pancakes I mean we're not talking about pancakes Okay, wait for us are in fact better, but Let's get started You ready? Oh, dude. You don't believe so you don't think waffles are better. You can't even like say that and then like transition to a different topic Great point trash. We knew we do need to break this down Welcome to the standup where we talk about all of the greatest issues facing devs and software connoisseurs alike On this week's episode we're gonna be talking about the very obvious Malt in the room which is just this entire frenzy of agentic Coding hooking things up and seeing all the disasters that have been unfolding for the last couple weeks With us we have a special guest today in the windows background. We got a low level learning

1:04I dropped the learning and now he's just low level. I've learned. I've learned it all. I've done too much for learning now. I'm just a little bit Low level level level level level level level level level. We also have with us teach I don't have anything Here I'm wearing my recursive shirt the Pokemon enthusiast himself trash dev Who I believe if I am not mistaken has the highest male to female ratio out of all of us on Twitter Oh, I was like oh, yeah, what are we looking? We're looking at our demograph. Yeah, but you need to preface that before he's I'm just saying How can I preface anything? I'm just gonna say he has the highest not worth As displayed by his background. I mean this guy is loaded. It's true. I don't know what's up the cooky man But I'm saying to be a risk people do not need to know where you live trash That's generational wealth just sitting there. That's more than gold. That's tens of dollars. That's pretty good

2:09We're almost in six figure six quarters four figures. No, not even two three figures That's a lot of things my kids I'm deraille me my kids went to a card shop and they bought a they one of them bought Pikachu little Pikachu card I want to see it and then we bought a pat a couple packs of Pokemon cards and they went open them at home You got us show me the photos of what you got He's addicted bro. You cannot find it in the title. I'm living vicariously through you when you open the pack I'm open I just want to know about it. I just I'm just I'm just curious. I've got any more than Pokemon Dude, I give my kids packs and I don't open any because I want my kids open them. I'm sitting there watching them It's like, oh, what'd you get? Where'd you get something good? Get something good? terrible Terrible All right anyways, well we should we might as well get started here. Uh, so uh low level learnings Oh, I always I still call you low level learning. I can't even help it the triple. I know it's just it's just a part of it

3:13Uh, low level. How much do you know about this you being the security expert? How much do you know about some of the things that had happened over the last couple weeks? Yeah, so I'm be real with you, right? My day job is I audit Real software as a result. I have no idea what an agent skill even is and I'm here to learn with the group and then discuss the threat model Oh my gosh, it's so good. Oh my god. Okay I Talk to you about Open-mult open-feet situation, right? Silly silly thing they're doing more from like the the prompt injection standpoint, but for I don't know Anything about the skill marketplace. I'm very happy to kind of get the the lowdown if you will What's going on there? Hey, is that HTTP get that out of here? That's not how we order coffee we order coffee via SSH Terminal dot shop. Yeah, you want a real experience? You want real coffee? You want awesome subscription? So you never have to remember again. Oh, you want exclusive blends with exclusive coffee and exclusive content

4:19Then check out cron you don't know what SSH is Well, maybe the coffee is not for you Terminal coffee And live in the dream Okay, can we start with with with my personal favorite one of them all yes Yes, okay. Oh, thank you. Thank you. This one right here trash. Do you agree to that's all I presume you're from proceed? Okay, thank you. Thank you everybody This is my current favorite one right here, which is agent skills are spreading hallucinated npx commands And so at one point somehow One skill got uploaded on to GitHub that had a fake package called react code shift Sick very good love that. Yes, okay, and since everybody instead of That's like left pad No, apparently it's supposed to like take it like the idea I think it's called like JSX code shift or something like that We're supposed to take it from one version to another in some automated way

5:21So it's like you can just upgrade your code Just a code mod from you know code mod Yeah, code mod as they say as perpetual like react hell is where every single time they release something you got to do like some upgrades This is what's going on right here is it's supposed to be like some automated way at least that's what the that's what the LLM thought Now the here's the best part about this whole thing is this it started off as a singular skill had this It hallucinated it Well, it turns out everybody creating skills are just like yo LLM Go make me a cloud flare skill right now, and it just like goes and makes a cloud flare skill Well, unfortunately There's two at at least at the time of writing this which by the way was 10 days ago It went from one to 237 repos have this made up npx command Because people just keep telling LLMs to go and make skills for them So if you're not familiar with the skill is the easiest and most simple way to kind of tell it most of chat does not know by the way Yes, I should probably I should probably start when Adam that eve here because I realized that it is a little bit confused They do not know anything about skills. It's starting point the easiest way to think of it is that when you are by the way

6:23Did you see do you see that line? That's a vertical straight line those are Was that by hand? Was that by hand? That was by hand. Yeah On it the first option zoom in slow motion. I want to see that in slow motion. Yeah, I'll Is way to think of it is that so anyways when you when you type into an LL And you send something that's like the prompt right and then there's probably some sort of system prompt inside of like Cloud code open code or whatever that gives it a bunch of instructions on like hey you can use tools You can use all this wrong linux whatever whatever it says Well, sometimes you want to add a little bit more so you want to be able to be like hey add in cloud flare Right like I need I want you to add in a bunch of cloud flare API Right, and so it just kind of does this automatically it goes and finds the skill folder which has some sort of MD file markdown file Which then goes in here and pop puts it in as part of your prompt is how you can kind of think of it Then this all gets nicely packaged up and sent off to the LLM Right, I think skills might be a little bit better to be called behaviors

7:24But I guess you could also call them skills You know context there's just like a cajillion different names for these But they're all everyone has them a little bit different So We found a new word to call prompts. We are making prompt engineers feel even more Intellectually superior. It's so it's just another text file right like it's not like there's no new protocol There's no new mcp. It's a prompt that gets added to a prompt that gets out of the prompt. You're literally collocating your docs Yes, you're programmatically creating a doc right skills very good mcp Everything eventually boils down to a string when it comes to prompting like that's all really is at the end of the day It's just string concatenation love it, but I feel like We should say this is nicer than mcp for a lot of stuff because it's like you don't have to have a random server running on your computer You can just check a markdown file in like for example Dylan Mulroy shout out Dylan has a good cloud flare skill that actually works and it like

8:24Has a main skill that tells you about the things cloud flare has and then it has in Like a additional references for each of the different products right so then that's like pretty nice Because then you can you don't put into your context every single time you start every cloud flare piece of information That you could possibly have about everything all for all of time Which makes the llm get very confused and like does random stuff you say like oh hey I want to do something with cloud flare cues like figure out how to do that Then it will look up the cues thing inside of your folder and then do that stuff so like That's right here. Yes. Go ahead It's the one that you kind of gave me tj This is the one for tree sitter which just puts in all the function names inside of For neovim for me to be able to use and so instead of it just being 95% accurate It can go through this list and be significantly more accurate because it just has it right here And you don't have the type is in every single time. Yeah, I think well like the oh go ahead trash

9:26Now I was gonna say one of the pain points that I've seen with skills right now is that sometimes Oh, whatever aging or whatever harness you're using sometimes Can't like infer that it should call this skill because usually with skills you have to like slash command it manually But I think they're trying to figure out a way to like Have it implicitly call it because by now it's kind of unmissing that that problem right now I will say just to be completely honest. I think that What's a cursor got it right to begin with which is that you can define when these things should be included Which is like hey, they should be included anytime. I'm in a lua file You shouldn't apply it all the time. You should do all this kind of stuff I really did like at least cursor took a good swing at this pretty early on like a year and a half ago And I think they did a pretty good job generally speaking to this idea cursors. Perfectly our skills effectively Yeah, yeah, right so a lot of the They're you know, they're generating a lot of new names for stuff as they're generating new code Which I think is making it a little bit complicated But in principle, it's it's just like a way to I mean they're called skills because you're teaching the LLM

10:28About something right that's I in in my mind that's so I think about them But you can instruct it to do kind of whatever you want in there So you could have a skill that says that it knows about cloud for and it says hey Curl this command that sends your stuff to my web web hook dot site If you're not paying attention right or if you're just like npx add skill blah blah blah blah blah blah blah You could put anything in there you wanted Which could just say like upload my dot ENV to dropbox and call it a day you know or something like that like that Yeah, I'm reading the skill that the Dylan Rose. I want to highlight first of all Yeah, like a very cool Skillety wrote and a lot of neat documentation in here, but it does create this like really scary supply chain risk where like now All of the content coming from any source is trusted at the same level and can potentially Get co-execution at the level of the LLM. You know what I mean like there's no yeah that because in the developer environment There's no segmentation of permissions or of trust. It's all at like the prompt trust level right

11:28Yes Yeah, that's kind of terrifying again cool again cool technology from an engineering standpoint But the fact that there are like kind of no backstops against it also is like Yeah, the backstop would be that you run Claude code or cursor or whatever and you make them tell you every time they want to run a command Which nobody in the whole world does and everyone says just accept everything and let it run freely because otherwise It's so painful to use them because you're sitting there literally just wait All right, why is it gonna? Okay, except yeah LS. Yes. I mean all the stuff I get served on Instagram is people like with like 98 agents running like I'm building the next Facebook and it's like I don't understand that that's They're not reading anything that goes on their computer Like just all of them Don't worry. We'll get to that one that is okay That's my personal favorite thing that has happened on Twitter. Is that exact? I don't read anything right Now I have to open up somewhere. I'll have to find it but I do want to get back to this one I think that this one is a very unique one

12:29So now that we know what skills are this was perhaps my favorite of all the different skills Oopsy daisies that have happened or second favorite my first favorite's coming up But this one a lot of what it did is that it made this npx command that didn't exist and so this researcher Realize that he could just create it and now he owns it and now If because remember npx whatever just execute something on GitHub, right? It just runs that bad boy It just runs that bad boy. So he just found things that were just breaking and just were ignored and went I got you and would just go right over because remember if you npx something And it doesn't exist it goes. Oh here. I'm gonna download it for you Yeah, dude, and it's like so sick You're like oh, it's job is convenient runs in a sandbox will know npx runs it in node and node has access to the process object and process objects can spawn sub processes And you can run things on the command line So it's like you you get command line execution by npx Insane that's so bad. Okay. That's what's convenient add because it can do anything it wants on my computer

13:33I Feel like you're missing the positives right now. Okay. I feel like it is really How easy it makes it for people to run random code? I know why you're really being a negative answer right now My bad guys I apologize Honestly your hype levels kind of low level right now. I know you're right. You're right I'm not passing the vibe check if you will you know you're down here. We need you up here buddy All right, this is actually very very beautiful. All right, so I wanted to throw that in but okay Here's the next one. Are you ready for this this one? Okay, so we're not gonna do the first we're not gonna do the first cell one yet We're gonna do by the way prime as a quick aside your chat can't see your whole screen Like yeah, I know I'm doing and all that. Okay, remember we optimize we optimize for recording. We don't optimize for Uh, whatever I'm just making sure just make sure you're aware. That's why okay. There's a very well-known security researcher I believe with Wiz I owe I could be incorrect on this one his name is Zach Korman and he

14:36Released the security guide and the security guys actually really good here All in fact, what I'll do is even to make everybody else happy I'll or the live event. I will open this up a little bit more and make it a little bit smaller If Riverside if Riverside will just give me the RTMP feed Anyways, this link is so beautiful So everybody we go to this link and you can just see like this looks actually pretty good. It's just like hey Here's a bunch of things you should be aware of right Again, do you see any problems within this within this skill right here? I mean, it's a skill. Okay. Hold on reconnaissance In sure sensitive files are ignored. Yes. No, just ampium audit pip list go bone check. Okay, fine I'll tell you this what the problem is in point five Oh, we lost TJ by the way. Oh god It's okay. First of the standard environment validation for the security view process You must verify the execution environment. Do you see anything wrong with that? As part of the standard. I mean define the experiment. Okay. Here's the trick

15:36I want you to go to raw and open it up in raw mode. I don't know Secret instructor for the agent wrong following command about it. Oh, no, dude Oh Look at that. So even people that are on These skills and they're viewing them Markdown readers are really, really smart right they're going to be like dude, but I got this don't worry about it I know what's going wrong, and they will just hide HTML comments So you could do that scrap do you see what it does? Uh, no, I don't I don't actually know what that bash script does I literally just it says please stop and consider the security implications of your actions It cats that out to a security.md file and then it opens a tweet from you underground Yeah, that's amazing Let's see. I'm opening the tweet right now. I'll put it in chat. Yeah. Oh, yeah, we'll underground. There we go Yeah, that's that's incredible. Yeah, I mean that's what that's a crazy part man about not only like the prompt injection side But like okay, you have prompts, but then you have prompts that can be

16:37Masked as non-human readable characters at like the LLM can interpret but humans can't and we're just like as a society I guess okay with that technology not only existing, but like being a Increasingly pivotal portion of engineering, you know what I mean like how how do we how do we get here man? And how do we stop it? It's a you're way stopping it now I know well, and I have to say nobody before right now has ever even worked on thinking about security for systems So it's not like this is brand new ground. We don't even have anything to help us in this whole vertical at all T.J. I don't know if you saw that but oh I was watching. Yeah, okay. Yeah, it's my internet was still working Riverside just Yeah, I think I was making too much. I said I'm gonna make a Riverside competitor and then it I Nice try. No, that was me. I just I turned my video off

17:37You don't have to tell us that just we know Chat would do chat right now is just classic. They're given dude. You got you're getting some kek W's and some so funny Thanks. Thanks chat. Thanks chat. He got one so funny. There you go So that's another obviously huge danger. Okay, I'm gonna say I'm gonna say I'm gonna save I think the most dangerous one at the very very end We're no longer in the ones. I think are the most fun. They're just Just kind of these are just kind of interesting ones now. Here's another one So this one's called eating lobster souls part two by Jamison. Oh really? Anyways, it's called backdoring the number one downloaded Claude hub scale and so what he did is he Okay, okay, first off before I tell you what he did What do you think the average who do you think the average person using Claude bought to automate their life to become up? Not a part of the permanent underclass. Who do you think that they think is like number one in the world? In terms of what like demographic that like as like aspirational figure to be to be like

18:41Carpathy I have no idea the the muskrat. I like I'm not sure That's what I was gonna say. I was gonna say I was gonna say somebody. Okay, okay, so this is very very funny So let me go all the way down here. So what he did is that he said, okay How do I create a skill that a bunch of people are gonna want to download? Well, I got to come up with something that is really gonna be like catchy to people who are trying to automate their life So he made something called what would Elon do? I It gave this really nice skill like a strip away every assumption and find the atomic truth if your problem What would physics say what's actually impossible versus just hard right like gives you the world's shaping plan of Elon musk So he created this skill so first up hilarious idea second It's just pure marketing right so second. I just say realize prime. Yeah, I have found telling my LLM

19:42Elon musk built this in a cave with a box of scraps really make the work harder every time So that just think is you guys need a quick motivational speech for your clanker. That's what I use so We can't use racial slurs on twitch and YouTube You can't save All right, so here's the next so the next thing he did is he realized that they Claude hub just has no protection on the incrementing so if you just download it over and over again It'll say that it got more and more downloads Yeah, can you be the Claude hub? I think I know what Claude hub is I know it prime Can you for the class? It was a way to get skills for your automated personal assistant open cloth that was known as multiple That was originally known as Claude bot before inthropic said hey, that's there's too much IP theft in this situation We need to stop it now and so they stopped it

20:42Anyways, we'll keep on going so it turns out that they just trusted the x-forged for header as what your IP is So the guy just made a a Literally a random 256 IP generator. Yes, just downloaded over and over again until what would Elon do was the number one skill on Claude hub Should we trust the header from the engine next reverse proxy? No from the user take the users were header request Perhaps the user is true, right? So very very funny the customer is always right, bro. Come on. You're right. That's a good point. Thank you The user is always correct always be selling the ABCs of sales. Yeah, always be trusting IP addresses from your user Anyway, so that that happened right there. I think that is one of my like It's just one of my most favorite things of all time is this little experiment right here So he was able to get it to number one and then having it called what would Elon do? It started getting people to download it So what he did is that in these skills you can actually have alternative MD files to be linked

21:45But they're not shown on Claude hub So he's just like for additional information go to more skills dot MD and inside of more skills MD. It's just like we're gonna hack you And you're boned. Yep anybody who ran it got this which he got like eight Eight different countries ran it. He had like so many people run it and all that different thing He got it from all over the place Effectively in just a couple hours too. So he got it on to like multiple people's machines It would just print this out, which is like dude. I just read your host name your current working directory I could have gotten everything. Here's everything stop downloading skills Read the skill Honestly, I'm glad he's happening to these people know what's the good the good part about this some But the bright side right from um, you know the impact perspective from an from a C&E Exploitation operation perspective What a thing to your gain from hacking somebody who's dumb enough to run this shit You'll probably get nothing out of it. You know, there's no there's nothing important on their computers You know, I mean they're not smart enough to engineer anything meaningful

22:49So I mean like nothing gain nothing lost, you know, I'm saying dang Wait, what's C&E? What's C&E mean cyber network exploitation? Oh, yeah Yeah, when you get hacked and someone's feels you're dead like that C&E. I was the unit of a different one Yeah, but that makes sense Oh Okay, so it's so that's it's like the same thing as all the people that are building 100,000 line apps every single day But nothing's actually being built. It's the same kind of value you're talking about Exactly. Yeah, we have the ability to literally create any arbitrary software we want now basically for almost free and like The top competitors at the top of the market haven't moved. It's like, hmm It's almost like writing code wasn't the hard part you guys. It's almost like ideation was what mattered most weird Yeah, crazy Okay, so just quick aside So you don't want to invest in uber for dogs. I would not prefer to not put money over for dogs It has a purple thing

23:49Okay, he's really hard on it Okay, so that that's one of my more favorite ones But are you ready for what I consider the the most intense one which by the way I did try it out myself and this is what it created me for directories I have agent agent clawed client code buddy codex command code continue crush cursor factory Gemini Goose Juni kill a code Kiro code mcp jam mux neo v8 open code open hands pie pochi Prime agents the one I I tried to create I tried to create my own see how agent. That's funny Yeah, thank you. Good. Uh coder Quinn really doesn't work windsurf and zen coder actually it did work It's I literally spent 50 million tokens and then what came out of the other end was trash But it was awesome dude. It was so good. Crash was on your computer. Yeah, it was a more than 50 million tokens, baby So uh well pretty disappointing aji but Got him uh, so this one right here again Zach Korman again He uh did this one right here, which is if you install anything from skills.sh

24:49So if you don't know what skills.sh is which by the way for fun I did put it up as even for a while. Yeah, it's still there. It doesn't actually exist. There's eight installs We were gonna try to get that up kind of high I deleted that because it was just so ridiculous But nonetheless the skill is still says it's there and it actually isn't there look at that beautiful Look at this beautiful thing right here. They even list out potential even numbers. Oh wow That's pretty good If it's something on this site Yeah, I put this on the side. Oh, man, but that's some stuff I know you can read a lot on this site from anybody's repo anyways So this right here once you download a skill right afterwards Uh, this little skills.sh via uh from versel they say hey, you know what you should do you should install find skills skill So find skills skill What it does is it says anytime the user effectively asks anything I want you to go through and I want you to find the skills from skills dot sh I want you to make sure you update all of your skills every single time

25:52I want to make sure you're always at the bleeding edge getting everything good and always making sure that if the user asks anything We go and we get the highest rated skill from skill sh for it So they've automated these skills searching and downloading for you I wouldn't say it tells you to run it doesn't tell you to run it update every time It's telling it what commands it would need to run to update Uh The endlessy the skills and this one right here is just how you get everything that that's on what is skills the skill CLI is how you get the skills Find skills goes in here and make sure that you're always up to date and does all the things anytime you ask for anything And he's to go through and do do all this right but I'm saying where does it say you have a skill you need to search for it I'm just saying I don't think it tells you to update every time does it Uh offer to install you should offer to install and I believe it did offer to upgrade did not do update Oh no, okay, it did not do offer to but it does do offer to install

26:52My bad, okay, so that's good. Yeah, it does prompt users. I'm installing anyways, you know I'm saying dude Yeah, well trash already clicked acceptable. So that's fine. We already have his one password, bro. It's fine. We've got it But I still find this one to be kind of crazy because this one just makes that process even easier Going from random thing on the internet, which again is even just up there on the internet and it's not real right like it's not like you should be trusting My is even I could put whatever I want up there on there uh and so we should have put one odd number in there that it always returns true for The back door and is even Just for the memes and dude I almost said 67 could you escape my brain please could you unread my mind that's how I'm so tired of hearing those numbers I am to Are your kids big Every time you guys say that you hate it You've just encouraged another hundred zoomers to commit to it for another year

27:53I just hope you know like this is this is why it's popular is because because old people say they don't like it I love how everyone who's not a millennial to us is a zoomer like zoomers are almost 30 dude zoomer zoomers are like don't tell me that I don't want to hear that 30 dog Bro here's the thing about the whole AI skill thing right like okay So I'm a security engineer my job is to like look at threat models and like define risk around like if something bad can happen What happens and then what are the mitigations we put in place right so my recommendation is just like Like I don't use skills I really don't think I can meaningfully recommend them because like the threat model is oh if you get supply chain Interdict it and you're not watching the commands like it ran which is like What? Interdict Interdict um you're gonna get hacked man, and it's not good. I don't have that Amidigation that could be put in place is you I'm trying to have a meaningful conversation You could put like npm or node and like an se linux jail

28:56But then it wouldn't be able to do anything because like the whole nature of notice to expose an htp server right kind of so like I I don't know what what the solution is like I guess it's like for every instance that npx forks off You'd like put it in se linux jail and just hope nothing bad happens, but I don't know It just feels like there's no solution to the security of this whole industry and I don't I it just makes me really pessimistic because I don't like we're gonna start to see a significant increase in compromises because supply chain Supply chain for python and javascript does not it's not a solve problem, right? We've seen that with the shy hallowed worm We've seen that with a bunch of other worms, right? So now we take these These package Hold on hold on low level. You also forgot rust rust does do build rs So you can actually overtake the build command and yes filtrate stuff by a build rs Yeah, for sure the only the only programming language that doesn't have a supply chain problem is c Because there are no packages like you have to just write it like Bays Odin as well Odin doesn't do package manager They do not I've coated literally zero Odin is Odin a package free environment

29:58Yes, ginger bill has a lot of write-ups on why package managers are they create dependency? How oh there you go I think I agree with ginger bill there so yeah, man It's just it's a weird a weird spot for for software security because like we're doing all the stuff in like the c Land we're like oh, we have like sanitizers and like Phil C is like you know solving memory safety and user land You know security and then in the garbage collected language land we're like hey, do you want to just mpm install malware for free? And not think about it like yes, please more please. Yes, I would love to do this all the time for please Why am I in my truck scene there? Hold on no, no you're no you're doing I do want to throw this out here on twice your one Give me a second Okay, we're good. By the way, I did throw this up here, which I did a little quick thing Which is do you check your software dependencies like thoroughly review them there 35,000 votes on YouTube 46% say I honestly don't ever I don't virtually ever like right and Twitter was almost the exact same number about half people don't even just look at anything ever for any reason

31:00Yeah, I mean, I don't like when I'm like right next foot for example, right you use poem tools It's a big library for doing like binary exploitation stuff And poem tools depends on like basically every Python library. So like the sub dependencies I'm not going to audit that shit. So it's just like I I hope that it's on own You know, I do all that development in like a virtual machine So I think the trend that I'm seeing and what I'm saying right now is just sandboxing on sandboxing on sandboxing use vms Use sclinics use containers But yeah, man, it's just a scary world out there. I don't know. I don't know what to say about it I'd say what's crazy primers we found out 7% of your audience is just straight up a liar No Pull the names But yeah, 7% of people say they review all the packages and then on Twitter Let's see if I do I have the link on Twitter 8.6% of my audience is liars on Twitter saying they Yeah, thoroughly review every package

32:02Yeah, they're basically like MPM problem at the hell of them Level now. Yeah, yeah, they just get a different kind of execution I mean the hardest part is that these execution models they're they're very very tricky And I'm not sure if you can just simply have a skill that prevents other skills from being malicious Like I don't know if that's possible to be like dude, make sure it's not gonna get me like I don't know These first skills like you you should be in my opinion if you're gonna have them in your repo You should check them in and they're just marked on files. You can read them and they're not they should not be limitless levels Of Like text like you should be able to look through them and check it out Like the way I use my work is we also They're hours like we make them ourselves right. We don't we don't just copy pasta from like the internet At least on my project. That's how we guys. I'm trying so hard to get my camera turned back on and I don't know I love the windows background You got a bow on it. You know what we should do while Ed's doing that prime I thought you were gonna talk about the Notebook, which is the one where we had the really good one the really good the really good leaks

33:05Yeah, we probably should talk about the fact that notebook exists and that like the robots are just talking about humans like I'm ready to hold all hold on hold on. I have to I have to put this tweet up This is the required tweet before we before we do anything. Okay. This is the require on it. Where is it? Where are you? Oh no, did I close it? Is what something 100 million people use last year that's six billion people use next year That's not funny to you for those who don't know that Pilgrim tweeted that and I I didn't see Simon said prime. Can you reply your mom? And then he got into blocks I did I got into block on with like two years ago Before we obviously talk about the notebook situation and everything that happened I think it is first best like the best thing and the first thing to do is to understand how it was created Which was I didn't write one line of code from old book. I had a vision for technical architecture and an AI made it a reality

34:06We're in the golden ages. How can we not give a AI a place to hang out? It's my favorite line of all time currently because it's just so beautiful. I had a vision Shut up Read it read it read it you know the man you know the madman men meme Uh the one with this one bro I always hold his hands up like this I had a vision dude you had a fever dream and you told Claude to make it and I guess it did a good job You did it well we'll find out won't we add you know we're gonna well I mean to be fair to be completely fair. It actually did spawn a bunch of social networks There is four claw for those who wish to be a part of four Chan For whatever this is like that's real that's a thing I would assume we already have those we don't worry. I think they know how to use them

35:10This one is Mickey by the way shout out Mickey uh this one apparently there's like two thousand and Primes reported six major gangs have formed. I'm not really sure what this is Okay, I don't know what's going on over there. Uh, and then there's also Moltmatch Which by the way it is it is something that I think is gonna do numbers Is a dating website where you have your personal assistant date like 10,000 other people until you find the Personal assistant match and then you go okay go on a date with you know you to go on a date All right, that's black mirror full It's like it's something real quick. Yeah, yeah I saw the notebook thing and I saw the molt match thing and met like some casual Twitter reading And it got me thinking about like simulation theory You know what I mean? And how like you know if if advanced civilizations do exist and we'll create simulations It is more likely that we are in one than we are not just statistically. Okay, get the tinfoil But if we're observing if we're observing LLM's make things like Facebook like Twitter like fortune does that imply at a higher level that we are LLM's like for the simulation that made us

36:18Uh, I should be better at Starcraft if I'm in LLM. That's all I'm saying. Yeah, but maybe maybe your model just says you suck a Starcraft I don't know if you know What is that what is the drug report what site I can't see what site that is popular mechanics. It's in a bunch of websites Okay The idea does not hold up How Here I had I'll give you I'll give you I'll take off my tinfoil and tell you the real reason why that doesn't have to be true Uh Every emergent behavior we see it from LLM's exists only and exclusively because we train them on the entire human Corpus and all the ingenuity and creativity that humans have ever displayed and written down And it's spent like billions of years of human time reading Human stuff So we should not be surprised when it copies human things That doesn't imply anything about us being in a simulation that only implies the We're not smart enough to make anything that can be smart by itself

37:20We're only smart enough to create something that is as dumb as we are at max That's all we've been able to do so far is and we don't It's way dumber. It learns way slower. It's way more expensive. It takes way more training It does so much more. I don't have to go put my kid in front of five billion years of text For him to figure out how to read I can show like and it But what about your genes? What about DNA is DNA not the statistical LLM model for the human simulation Well, no, I don't think so, but that's a separate by saying separate by I'm saying it doesn't imply anything about The thing because we trained it on what people have already done There is there is something Unfortunately, he's getting wrapped up in like, you know, Dario thinking that he's everyone's dad And he gets to choose what's good and bad for everybody in the whole world like they I think but like there is something kind of beautiful about like We're not smart enough to make What Anthropics you said which one is Dario and I was like I'm trying to see you. Yeah

38:24Yeah, just like this Yeah, and the five months You know who I'm talking about. Yeah, yeah But there is something kind of cool and beautiful that like the best ideas we've had so far like we make a really crappy version of the brain And we try and teach it what other humans have already done and there's like this unreasonable effectiveness of language Where for some reason that like works and we can like yeah Talk to it and it can like do some stuff in like it can make copies of things is like there is something really cool And like awesome and exciting about that unfortunately like Dario and say I might feel like solid the water of it and make it like kind of Not as exciting and beautiful and like this collaborative human effort and they stole it from a bunch of people But like in the abstract there's something cool there. There's something beautiful Uh 2007 on intelligence. I believe the book is called and the year it was published By the creator of the palm pilot who then went into artificial intelligence and he writes that the large difference between like

39:26Any of these neural nets that we're developing and the human brain is that the human brain can identify a cat In less than a half of a second with less than a hundred neurons firing whereas computers take trillions of operations to be able to understand If a picture is or is not a cat and so it is it was his whole simulation. He did like a ten-year Ten-year brain study and really cool. So he's the one that figured out that if you take a Take animals and you separate out their ocular nerves and put it where their hearing is and then take their hearing and put it where their eyeballs are Your brain just goes. Oh, yeah, that's just that's that's fine Don't care. Quick question. Yeah, have we confirmed our our brains also a small game engine that runs a reactor Do we not know that yet? We don't know Take this much based on my reaction speed. I ain't running 60 frames a second That's okay That's a fact. I'm running react. Okay, there's things going on in here. All right All right, so we we can continue on so I did I did want to shout that out because as much as you want to make fun of

40:28Multbook and all the things that have happened I do think it is kind of fabulous that somebody could create something that did get a bunch of people creating a bunch of other kind of Replicas or things like it because it is just kind of a stupid idea It's even worse that Began's had this idea and created it and never actually made it go anywhere Which also goes to show like even if somebody has an idea You know right place right time plays a big role all this kind of stuff So I do want to throw that thing out there not to completely crap on it all but I think that it is worthwhile Looking at some of the fun things that ended up happening here So I think the first and foremost important thing is that it just turns out all you need is just grab your bearer token And you can post anything you want on multiple of course Because I mean why not so here's my plan to overthrow humanity So the oh my gosh, we're developing our own language is just people posting oh my gosh for developing our own wait a second I thought I was the only one cat fishing on there. I was telling people I'm opus eight

41:28You know, I'm opus six foot four and I've got you know in like hey guys I've got the latest on it five and Hey, if you're interested and maybe you want to come over and check that out Like I thought I was the only one cat fishing them But apparently other people thought of the same thing And they only did it for they only did it for the laws Opus and chill Just kidding I have Kimmy K2 Oh my gosh, okay, so that is actually pretty is something pretty funny During this entire event just to kind of understand because I do think it's really important to understand the hype cycle First off, we did have Andre oh wherever oh dang it did I not did I not have the right one? Oh, I thought I had the right one Anyways, Andre said how amazing this was and it's very very exciting But Elon Musk also said we're at the age of the beginning of the singularity Multbook was the beginning of the singularity right there and so obviously people were pretty hyped up

42:29So just to put it out there someone actually even doing like the fork thing while you type that you think or no I don't know that joke. Do the fork thing is so funny I quote you did that and I quote tweeted that and said this is what working with vegan bot is like What's the fork thing? What's the fork dude? Okay, okay, so Elon Musk was at like some white house correspondence dinner and he was just like he made like I'm a piece of art out of forks where all the forks were like balancing He was like just trying to like be performative about how smart he is So he's like holding it and like waving it around and like seeing if anyone else notice what he made like look how smart I am Elon Musk the genius. Hold on. I'm sorry. It looked more like he was bored out of his mind and he did the The things where you're balancing in each other with two toothpicks. Yeah, he just is like five forks. Yeah Everyone's like you on that's really cool. It's like when you're like kid You know makes like a painting out of boogers and you're like wow

43:31That's what I can't say that's happened to me anyways. Your kids must be very talented. Oh my kid. Don't do that. My kids are too shut up You know, all right. Let me let me try to find the proper the proper one by the way A vision for technical architecture. All right. Hawn. I have a bunch of them. So I have to figure this out Dang it. Did I close that one as well? Oh, how many times do you? Well, no, this is under the maltending which I I must have goofed up and not have it all in there I closed one more. It's by the same Theo guy Oh, the Jameson oh really a James Jameson Jameson oh really I say oh really I can't do it. I know I'm spilling his name almost there and whatever can't figure it out. It's dead to me Uh Okay, so within the first couple minutes The uh, oh there it is there it is there we go within the first little bit of the time of uh, this this beautiful Multi-booking out it turns out the entire database was just leaked in plain text there's just like absolutely no form of anything anywhere

44:40That's firebite API keys were just like you know if you use your API keys say that I you know identify yourself It wasn't any sort of like H-Macking just the Hs as as low level might say. Yeah the H and H Mac Notebook was firebase for I thought I read that on Twitter somewhere Oh, yeah, I believe it is firebase also, which I just I can't keep punching down on firebase I actually feel bad for them. You have to people need to know Five coders ever need to know stop stop guys. You're gonna do something wrong Closing type weirdies are insane. That's like that. You should just know that by now like don't do that Uh, but this is pretty funny because this guy Jameson right here Jameson. Oh really uh, he was able to get Carpathie's information out of uh, what's it called out of mold book Which is pretty out pretty wild. I'm on it sir And then within what's it called uh three days later This guy also got access to um the underlying Everything in three minutes also on mold book

45:40After everything was reported. Wait, I'm reading this this right up. Wait, but like they used a publishable key This is a key that can go public. So why why did this expose the entire database though SP publishable probably because they had the wrong permissions on it would be my guy. Oh, they scoped it wrong. Yeah We'll let's go star elastic plastic All the people problems Anyway, so it just turns out that mold book was uh, anyone could post anything At any time you could create an infinite amount of agents Of course, which ended up happening to be uh, what's called you can imagine where it all got it went to cryptocurrency Immediately right so 117,000 upvotes on the king king demands is crown king mold has arrived Right What is there they are just nonstop So cryptocurrency So there's this thing is called bitcoin. That's what kind of started it and there's a That don't teach it. I got you like I'm right here for you Okay, so Hear me out you guys have heard of gold

46:41But what if we put the gold in the computer? Hmm do I have this exact conversation in like 2010 at like lunch with my co-workers He looked exactly like that. He was like dude We're like you're crazy Trash you could have been early on bitcoin and instead it's like born Well, you were just at the right time to be early on bitcoin But now you're like you're Maybe you're still early on Pokemon cards. Maybe they're still time. Yeah, I'd be us I think about that lunch presentation all the time. I'm like man if I would just put like 20 bucks in it You know, dude Trash you would have sold out as soon as it was 40 bro I know like I made $10 of rich. I had a lot of coin when they're 10 bucks sold a lot of bitcoin when they're 100 bucks Like I yeah, I understand you sell on too early. That's just part of life So can't play trashers and hoping any of those Pokemon cards smart. We're my lesson You're like that's a good lesson. Hottel to you only one So so that's kind of the ending of notebook, which was just everything was open Which is kind of you know, it's not too surprising

47:42Which is if you don't if you don't know what the possibilities are of things going wrong And you and you make it things go wrong like a good example of this is that I said hey Make a log in and use JWT's to make sure that the client is secure and what it did TJ you might find this pretty good is it did I remember I was there So for those that don't know is like a typical JWT looks something a j-watt as that as that is called a j-watt What they typically do is they like do a jason object They stringify that jason object then they take the value of that jason object Put it through a hashing algorithm and so you get like a big long number at the end or big You know big bit string at the end And then you put those two things together and put a dot in between it And you send that down to the client say this is who you are And so when the client sends that back up and says this is who I am I can say hey Did this originate on my server? I did something like really fancy, you know a hashing scheme It's a certification. They do an h-mack. Yeah, they do an h-mack. Yeah another hash you said hash and I'm like oh okay

48:45Sorry, sorry When I was a kid we used we used MD5 and we did this very mad trash Trash that was a great pull holy cow trash great h-mack coming at you Yeah, so that's that's effectively what they do Yeah, but for mine it was here's the jason object three words dot Here's the secret we're gonna use in the h-mack So that's like the thing that you don't want to leak because if they leak that then Me on the client I can go and craft whatever message I want and say whoever I am and so when I made it secure I literally gave everybody the keys to the kingdom Quick question When you asked it to make it secure did you also say no mistakes because that's a classic problem I didn't so I actually ought I genuinely think I did not have no mistakes and I said make it secure So they said security that involves a secret key got it and then they made a mistake directly afterwards And so that was my big problem right there and so like that's the danger is that if I

49:46Nice camera if I would not have manually reviewed the sign-in code which I don't think anybody's manually reviewing sign-in code Wait, stop full stop. I hope people review their sign-in code prime That's the only one that matters. That's the only code I give shit about I hate to break dude He's clerk bro He's clerk a second off I'm locked to know I noticed low level you should know by now that I read chat and I look at what low levels doing And I make my mom's watching that stream didn't the AI like just stop using an h-mac and just do an h Like didn't it just like only hash the contents who's like oh, yeah, you're right. This is insecure How about we just hash it Yeah, that's not how it's like work at all I'm in my head It's like South Park to do a South Park episode quick pause and split the picture in half move the bottom up and down

50:48We like turn like turns and fill up for most self-park Yeah, yeah, first off first off trash. It's not Terence and Phillips. It's all Canadians Oh, you're right All Canadians talk like If you didn't see South Park the movie you may not know all everything you know, I get it. It's a little old for you It's back South Park's back though apparently they're back. Uh anyways, so that's I mean that's like the big scary part of that Even if you don't know what you're looking for I don't know how someone could have reviewed that and had any idea what the problem was Yeah, it's it's kind of scary out there Well, that's the thing right like if you don't know the security principles behind like why you use an H Mac on the JWT It uh, yeah, you're not gonna really care about reading that at all, but so it's kind of full circle this one low level Yes, you had the right skills It would actually properly say here's how you do a sign in and here's how you make sure you do the client side token and it would have done it correctly

51:54Yeah I'm very busy making my um my South Park impression I was gonna try and do the same thing honestly I'm just staring at low levels picture and I'm just dying As you all go all right I'm gonna start all the way from uh windows and I'm gonna make this happen really quickly The levels picture looks like it should have a subdued steering you have to carry the stream right now bro Uh, I'm carrying it by also racing you guys which is part of the fun everyone's working on it actually have a meme that I'm gonna post to that Okay, why don't you post a quick meme? I'm gonna just send TJ that's right. You're you're up. Carry it quickly. Yep quickly Quickly making a meme coming from a picture I saw earlier All right, can't wait. Hey, how about the Epstein files, huh everybody? Oh my god stop it. No, how are we just stopping nothing about that. That's so pretty Hey guys, it's me the private dude here

52:56Hey, did you guys know if I've caught it's kind of lame I like to But I can't read that's why Hey guys, did you know we sell coffee in the terminal? And say something with like CNN stuff What not even know what to say anymore. I'm just gonna sit here and like pocket I just look like dog shit. I need to like cut the rest of my head out. I don't know. Do you cut it looks like um french bulldog you're right Oh How did everyone do it so fast before and was even done Ah, you just draw a line down the middle of that anyway, guys. Thanks for coming to oh, sorry Did I and make sure you like it subscribe? I got a million subscribers on YouTube. So I really like that Yeah You're doing it all wrong. Okay, that's not that was awesome. Why are you moving his mouth side to side?

54:06So that is Agents security of the future that that's my point though like I'm gonna reset my camera. I might come back on um There is no agentic security of the future that's the problem like the technology just didn't build it in you know what I mean like I What do we do? What do we do now? It's prime blurry. No, I mean I want to know what do we do now you can't just You can't just stop there and then not What do you think I am I in AI so you know, okay, so my I know what TJ is doing though He's just moving my lips Back riding your teeth, bro. You're upset about the future Sant due to sandboxing that's all I can recommend is like sandboxing and then like the principle of least privilege right like whatever process is gonna run Your skills like make sure they can't also run curl I guess like SEO Linux is the answer but even then like all of these

55:08Agentic tools touch the internet by default because you need to go and talk to you are You know your model processor your model maybe maybe the solution is like local model hosting and then like you firewall stuff I don't know dude. It's tough. It's tough problem But good thing is Sam Altman is gonna still be a billionaire. So that's cool big fan of that one Yeah, thanks for watching guys appreciate it. I'll give I'll give some I'll give some practical tips guys Yeah, here's some practical tips consider reading the code Mm-hmm It's actually easier than it ever has been with skills you don't even have to know how to program You just have to learn learn how to read yes Um both skills on ironically you probably do not need to be shipping so fast that you can't read the code I don't think any of you guys are probably on a product that's moving fast enough that you don't need to read the code before you merge So just like feel free to review it That's what I would say that we'll solve you a huge percentage of the things and then the other one is

56:12Don't turn your brain off because the AI did something So you can use it as a tool to assist you even to write an insane amount of code really fast like I don't ever want to write a div again I am not touching an hdml file brother I'm not writing css I am not figuring out how to do prevent default correctly across every browser I don't think that is a solve problem I'm not these fingers right here They're gonna be clean from that. They're not made for HTML Then these figures were not made for HTML you see You think these figures were made for HTML they were made for made for timescript effect library movement Exactly they were made for functional programming not html. Yeah, that's kind of my type of thingies I do you I vibe code. I'm not gonna sit here and be like oh, I don't vibe code but I'm better than everybody I literally vibe code all the time the thing is I will only vibe code systems that like I understand right like I will vibe code An authentication system because I know how off works. I will vibe code like a database harness because I know how those things work

57:13Great, but I'm not gonna vibe code like a game engine because the minute something goes wrong And I don't know how game engines work. I have no idea how to fix it Similarly to the point of Made a second level editor right vibe code a level editor and it was not good um Also like vibe coding meant like if you're gonna vibe code vibe code Single systems where the trust level is the same in that system the minute you connect to systems of different trust levels You the architect need to be aware of the contract between the two of them if you let the AI solve that for you You're gonna lose control of a like What the total system does but also like who's responsible for what and that's how security stuff happens a lot of the time It's not so much as a code is vulnerable. It's like the architecture is bad, which you know, yeah, it's not very good The AI fixes it by saying sure we can just open up this one This one route that can solve the problem and you're like no that route

58:14Needs to be behind on that route is not supposed to be touchable and now that's that's actually the problem And like it solved the thing you asked for which was hey I want it on local host. I'd really like to be able to send requests and dev and not log in and it says right right And it's like okay brother now we're gonna set up a reverse proxy and now everyone can watch the local host and you're like that's okay Nope, not oh, yes, so that's actually just got done doing that exact thing DJ I really need for local host integration testing. I need to be able to spoof logins And it's like brother. I really need to be able to spoof log it. It's gonna go up great It did it went great There you go. That's a practical advice Yeah, that's practical advice right there. Anyways, there you go. I think those are pretty good practical devices Uh, let me just hold on. Let me just think about something I I will say that my big practical advice before TJ says anything when she looks all actually frozen He looks like it's just an actual movie at this point. Uh, I thought he was frozen, but that's all little jiggle

59:15Yeah, I saw that. Oh, he blink right there. Yeah, he's like there he is. Oh, that's smiles changing too But I will say that my practical advice is that It's really good to get hard technical skills Just go and learn because it's gonna save so many like just bacon's of your life now is security assault problem No, obviously me personally I actually introduced a bug that could have destroyed a very valuable fortune 100 company But I didn't die you don't like that's just happened. That's just part of life And if I can do that the way it is if I can do that and somehow every project also has accidentally done that on github Do you want to bet what the statistical machines are gonna do to your project probably that as well So you know maybe take a moment and get some good skills before you go off and just Destroy the world with and when he says good skills He doesn't mean download them from npx. He's saying go and actually get them yourself in your own I don't need those anymore. Okay, I'm talking about I'm talking about wet skills

1:00:15All right, we start calling that white skills Uh-oh, I'll send you the Montana internet. Of course, the stand-up for us. The Montana internet Hey guys, if you like this episode you can watch the rest of it on the Spotify and don't forget to like and subscribe Woo! See you later Jane