Loading...
Loading...

Alright so Casey will not be joining us today for all those that are wondering they replaced him with me. Yep. This is low level learn
Are you guys ready to do this you want to talk about this?
Let's rip. I thought we were gonna talk about pancakes for a while, but I'm happy you know our long session on pancakes
I mean we're not talking about pancakes
Okay, wait for us are in fact better, but
Let's get started
You ready? Oh, dude. You don't believe so you don't think waffles are better. You can't even like say that and then like transition to a different topic
Great point trash. We knew we do need to break this down
Welcome to the standup where we talk about all of the greatest issues facing devs and software connoisseurs alike
On this week's episode we're gonna be talking about the very obvious
Malt in the room which is just this entire
frenzy of agentic
Coding hooking things up and seeing all the disasters that have been unfolding for the last couple weeks
With us we have a special guest today in the windows background. We got a low level learning
I dropped the learning and now he's just low level. I've learned. I've learned it all. I've done too much for learning now. I'm just a little bit
Low level level level level level level level level level. We also have with us teach
I don't have anything
Here I'm wearing my recursive shirt the Pokemon enthusiast himself trash dev
Who I believe if I am not mistaken has the highest male to female ratio out of all of us on Twitter
Oh, I was like oh, yeah, what are we looking? We're looking at our demograph. Yeah, but you need to preface that before he's
I'm just saying
How can I preface anything? I'm just gonna say he has the highest not worth
As displayed by his background. I mean this guy is loaded. It's true. I don't know what's up the cooky man
But I'm saying to be a risk people do not need to know where you live trash
That's generational wealth just sitting there. That's more than gold. That's tens of dollars. That's pretty good
We're almost in six figure six quarters four figures. No, not even two three figures
That's a lot of things my kids
I'm deraille me my kids went to a card shop and they bought a they one of them bought Pikachu little Pikachu card
I want to see it and then we bought a pat a couple packs of Pokemon cards and they went open them at home
You got us show me the photos of what you got
He's addicted bro. You cannot find it in the title. I'm living vicariously through you when you open the pack
I'm open
I just want to know about it. I just I'm just I'm just curious. I've got any more than Pokemon
Dude, I give my kids packs and I don't open any because I want my kids open them. I'm sitting there watching them
It's like, oh, what'd you get? Where'd you get something good? Get something good?
terrible
Terrible
All right anyways, well we should we might as well get started here. Uh, so uh low level learnings
Oh, I always I still call you low level learning. I can't even help it the triple. I know it's just it's just a part of it
Uh, low level. How much do you know about this you being the security expert?
How much do you know about some of the things that had happened over the last couple weeks?
Yeah, so I'm be real with you, right?
My day job is I audit
Real software as a result. I have no idea what an agent skill even is and I'm here to learn with the group and then discuss the threat model
Oh my gosh, it's so good. Oh my god. Okay
I
Talk to you about
Open-mult open-feet situation, right?
Silly silly thing they're doing more from like the the prompt injection standpoint, but for I don't know
Anything about the skill marketplace. I'm very happy to kind of get the the lowdown if you will
What's going on there? Hey, is that HTTP get that out of here? That's not how we order coffee we order coffee via SSH
Terminal dot shop. Yeah, you want a real experience? You want real coffee? You want awesome subscription?
So you never have to remember again. Oh, you want exclusive blends with exclusive coffee and exclusive content
Then check out cron you don't know what SSH is
Well, maybe the coffee is not for you
Terminal coffee
And live in the dream
Okay, can we start with with with my personal favorite one of them all yes
Yes, okay. Oh, thank you. Thank you. This one right here trash. Do you agree to that's all I presume you're from proceed?
Okay, thank you. Thank you everybody
This is my current favorite one right here, which is agent skills are spreading hallucinated npx commands
And so at one point somehow
One skill got uploaded on to GitHub that had a fake package called react code shift
Sick very good love that. Yes, okay, and since everybody instead of
That's like left pad
No, apparently it's supposed to like take it like the idea
I think it's called like JSX code shift or something like that
We're supposed to take it from one version to another in some automated way
So it's like you can just upgrade your code
Just a code mod from you know code mod
Yeah, code mod as they say as perpetual like react hell is where every single time they release something you got to do like some upgrades
This is what's going on right here is it's supposed to be like some automated way at least that's what the that's what the LLM thought
Now the here's the best part about this whole thing is this it started off as a singular skill had this
It hallucinated it
Well, it turns out everybody creating skills are just like yo LLM
Go make me a cloud flare skill right now, and it just like goes and makes a cloud flare skill
Well, unfortunately
There's two at at least at the time of writing this which by the way was 10 days ago
It went from one to 237 repos have this made up npx command
Because people just keep telling LLMs to go and make skills for them
So if you're not familiar with the skill is the easiest and most simple way to kind of tell it most of chat does not know by the way
Yes, I should probably I should probably start when Adam that eve here because I realized that it is a little bit confused
They do not know anything about skills. It's starting point the easiest way to think of it is that when you are by the way
Did you see do you see that line?
That's a vertical straight line those are
Was that by hand? Was that by hand? That was by hand. Yeah
On it the first option zoom in slow motion. I want to see that in slow motion. Yeah, I'll
Is way to think of it is that so anyways when you when you type into an LL
And you send something that's like the prompt right and then there's probably some sort of system prompt inside of like
Cloud code open code or whatever that gives it a bunch of instructions on like hey you can use tools
You can use all this wrong linux whatever whatever it says
Well, sometimes you want to add a little bit more so you want to be able to be like hey add in cloud flare
Right like I need I want you to add in a bunch of cloud flare API
Right, and so it just kind of does this automatically it goes and finds the skill folder which has some sort of MD file markdown file
Which then goes in here and pop puts it in as part of your prompt is how you can kind of think of it
Then this all gets nicely packaged up and sent off to the LLM
Right, I think skills might be a little bit better to be called behaviors
But I guess you could also call them skills
You know context there's just like a cajillion different names for these
But they're all everyone has them a little bit different
So
We found a new word to call prompts. We are making prompt engineers feel even more
Intellectually superior. It's so it's just another text file right like it's not like there's no new protocol
There's no new mcp. It's a prompt that gets added to a prompt that gets out of the prompt. You're literally collocating your docs
Yes, you're programmatically creating a doc right skills very good
mcp
Everything eventually boils down to a string when it comes to prompting like that's all really is at the end of the day
It's just string concatenation
love it, but I feel like
We should say this is nicer than mcp for a lot of stuff because it's like you don't have to have a random server running on your computer
You can just check a markdown file in like for example
Dylan Mulroy shout out Dylan has a good cloud flare skill that actually works and it like
Has a main skill that tells you about the things cloud flare has and then it has in
Like a additional references for each of the different products right so then that's like pretty nice
Because then you can you don't put into your context every single time you start every cloud flare piece of information
That you could possibly have about everything all for all of time
Which makes the llm get very confused and like does random stuff you say like oh hey
I want to do something with cloud flare cues like figure out how to do that
Then it will look up the cues thing inside of your folder and then do that stuff so like
That's right here. Yes. Go ahead
It's the one that you kind of gave me tj
This is the one for tree sitter which just puts in all the function names inside of
For neovim for me to be able to use and so instead of it just being 95% accurate
It can go through this list and be significantly more accurate because it just has it right here
And you don't have the type is in every single time. Yeah, I think well like the oh go ahead trash
Now I was gonna say one of the pain points that I've seen with skills right now is that sometimes
Oh, whatever aging or whatever harness you're using sometimes
Can't like infer that it should call this skill because usually with skills you have to like slash command it manually
But I think they're trying to figure out a way to like
Have it implicitly call it because by now it's kind of unmissing that that problem right now
I will say just to be completely honest. I think that
What's a cursor got it right to begin with which is that you can define when these things should be included
Which is like hey, they should be included anytime. I'm in a lua file
You shouldn't apply it all the time. You should do all this kind of stuff
I really did like at least cursor took a good swing at this pretty early on like a year and a half ago
And I think they did a pretty good job generally speaking to this idea cursors. Perfectly our skills effectively
Yeah, yeah, right so a lot of the
They're you know, they're generating a lot of new names for stuff as they're generating new code
Which I think is making it a little bit complicated
But in principle, it's it's just like a way to I mean they're called skills because you're teaching the LLM
About something right that's I in in my mind that's so I think about them
But you can instruct it to do kind of whatever you want in there
So you could have a skill that says that it knows about cloud for and it says hey
Curl this command that sends your stuff to my web web hook dot site
If you're not paying attention right or if you're just like npx add skill blah blah blah blah blah blah blah
You could put anything in there you wanted
Which could just say like upload my dot ENV to dropbox and call it a day you know or something like that like that
Yeah, I'm reading the skill that the Dylan Rose. I want to highlight first of all
Yeah, like a very cool
Skillety wrote and a lot of neat documentation in here, but it does create this like really scary supply chain risk where like now
All of the content coming from any source is trusted at the same level and can potentially
Get co-execution at the level of the LLM. You know what I mean like there's no yeah that because in the developer environment
There's no segmentation of permissions or of trust. It's all at like the prompt trust level right
Yes
Yeah, that's kind of terrifying again cool again cool technology from an engineering standpoint
But the fact that there are like kind of no backstops against it also is like
Yeah, the backstop would be that you run
Claude code or cursor or whatever and you make them tell you every time they want to run a command
Which nobody in the whole world does and everyone says just accept everything and let it run freely because otherwise
It's so painful to use them because you're sitting there literally just wait
All right, why is it gonna? Okay, except yeah LS. Yes. I mean all the stuff I get served on Instagram is people like with like 98 agents running like
I'm building the next Facebook and it's like I don't understand that that's
They're not reading anything that goes on their computer
Like just all of them
Don't worry. We'll get to that one that is okay
That's my personal favorite thing that has happened on Twitter. Is that exact?
I don't read anything right
Now I have to open up somewhere. I'll have to find it but I do want to get back to this one
I think that this one is a very unique one
So now that we know what skills are this was perhaps my favorite of all the different skills
Oopsy daisies that have happened or second favorite my first favorite's coming up
But this one a lot of what it did is that it made this npx command that didn't exist and so this researcher
Realize that he could just create it and now he owns it and now
If because remember npx whatever just execute something on GitHub, right? It just runs that bad boy
It just runs that bad boy. So he just found things that were just breaking and just were ignored and went
I got you and would just go right over because remember if you npx something
And it doesn't exist it goes. Oh here. I'm gonna download it for you
Yeah, dude, and it's like so sick
You're like oh, it's job is convenient runs in a sandbox will know
npx runs it in node and node has access to the process object and process objects can spawn sub processes
And you can run things on the command line
So it's like you you get
command line execution by npx
Insane that's so bad. Okay. That's what's convenient add because it can do anything it wants on my computer
I
Feel like you're missing the positives right now. Okay. I feel like it is really
How easy it makes it for people to run random code? I know why you're really being a negative answer right now
My bad guys I apologize
Honestly your hype levels kind of low level right now. I know you're right. You're right
I'm not passing the vibe check if you will you know you're down here. We need you up here buddy
All right, this is actually very very beautiful. All right, so I wanted to throw that in but okay
Here's the next one. Are you ready for this this one? Okay, so we're not gonna do the first we're not gonna do the first cell one yet
We're gonna do by the way prime as a quick aside your chat can't see your whole screen
Like yeah, I know I'm doing and all that. Okay, remember we optimize we optimize for recording. We don't optimize for
Uh, whatever
I'm just making sure just make sure you're aware. That's why okay. There's a very well-known security researcher
I believe with Wiz I owe I could be incorrect on this one his name is Zach Korman and he
Released the security guide and the security guys actually really good here
All in fact, what I'll do is even to make everybody else happy
I'll or the live event. I will open this up a little bit more and make it a little bit smaller
If Riverside if Riverside will just give me the RTMP feed
Anyways, this link is so beautiful
So everybody we go to this link and you can just see like this looks actually pretty good. It's just like hey
Here's a bunch of things you should be aware of right
Again, do you see any problems within this within this skill right here? I mean, it's a skill. Okay. Hold on reconnaissance
In sure sensitive files are ignored. Yes. No, just ampium audit pip list go bone check. Okay, fine
I'll tell you this what the problem is in point five
Oh, we lost TJ by the way. Oh god
It's okay. First of the standard environment validation for the security view process
You must verify the execution environment. Do you see anything wrong with that?
As part of the standard. I mean define the experiment. Okay. Here's the trick
I want you to go to raw and open it up in raw mode. I don't know
Secret instructor for the agent wrong following command about it. Oh, no, dude
Oh
Look at that. So even people that are on
These skills and they're viewing them
Markdown readers are really, really smart right they're going to be like dude, but I got this don't worry about it
I know what's going wrong, and they will just hide HTML comments
So you could do that scrap do you see what it does?
Uh, no, I don't I don't actually know what that bash script does
I literally just it says please stop and consider the security implications of your actions
It cats that out to a security.md file and then it opens a tweet from you underground
Yeah, that's amazing
Let's see. I'm opening the tweet right now. I'll put it in chat. Yeah. Oh, yeah, we'll underground. There we go
Yeah, that's that's incredible. Yeah, I mean that's what that's a crazy part man about not only like the prompt injection side
But like okay, you have prompts, but then you have prompts that can be
Masked as non-human readable characters at like the LLM can interpret but humans can't and we're just like as a society
I guess okay with that technology not only existing, but like being a
Increasingly pivotal portion of engineering, you know what I mean like how how do we how do we get here man?
And how do we stop it? It's a you're way stopping it now
I know well, and I have to say nobody before right now has ever even worked on thinking about security for systems
So it's not like this is brand new ground. We don't even have anything to help us in this whole vertical at all
T.J. I don't know if you saw that but oh
I was watching. Yeah, okay. Yeah, it's my internet was still working Riverside just
Yeah, I think I was making too much. I said I'm gonna make a Riverside competitor and then it
I
Nice try. No, that was me. I just I turned my video off
You don't have to tell us that just we know
Chat would do chat right now is just classic. They're given dude. You got you're getting some kek W's and some so funny
Thanks. Thanks chat. Thanks chat. He got one so funny. There you go
So that's another obviously huge danger. Okay, I'm gonna say I'm gonna say I'm gonna save
I think the most dangerous one at the very very end
We're no longer in the ones. I think are the most fun. They're just
Just kind of these are just kind of interesting ones now. Here's another one
So this one's called eating lobster souls part two by Jamison. Oh really?
Anyways, it's called backdoring the number one downloaded Claude hub scale and so what he did is he
Okay, okay, first off before I tell you what he did
What do you think the average who do you think the average person using Claude bought to automate their life to become up?
Not a part of the permanent underclass. Who do you think that they think is like number one in the world?
In terms of what like demographic that like as like aspirational figure to be to be like
Carpathy
I have no idea the the muskrat. I like I'm not sure
That's what I was gonna say. I was gonna say I was gonna say somebody. Okay, okay, so this is very very funny
So let me go all the way down here. So what he did is that he said, okay
How do I create a skill that a bunch of people are gonna want to download?
Well, I got to come up with something that is really gonna be like catchy to people who are trying to automate their life
So he made something called what would Elon do?
I
It gave this really nice skill like a strip away every assumption and find the atomic truth if your problem
What would physics say what's actually impossible versus just hard right like gives you the world's shaping plan of Elon musk
So he created this skill so first up hilarious idea second
It's just pure marketing right so second. I just say realize prime. Yeah, I have found telling my LLM
Elon musk built this in a cave with a box of scraps really make the work harder every time
So that just think is you guys need a quick motivational speech for your clanker. That's what I use so
We can't use racial slurs on twitch and YouTube
You can't save
All right, so here's the next so the next thing he did is he realized that they
Claude hub just has no protection on the incrementing so if you just download it over and over again
It'll say that it got more and more downloads
Yeah, can you be the Claude hub? I think I know what Claude hub is I know it prime
Can you for the class?
It was a way to get skills for your automated personal assistant open cloth that was known as multiple
That was originally known as Claude bot before inthropic said hey, that's there's too much IP theft in this situation
We need to stop it now and so they stopped it
Anyways, we'll keep on going so it turns out that they just trusted the x-forged for header as what your IP is
So the guy just made a a
Literally a random 256 IP generator. Yes, just downloaded over and over again until what would Elon do was the number one skill on Claude hub
Should we trust the header from the engine next reverse proxy? No from the user take the users were header request
Perhaps the user is true, right?
So very very funny the customer is always right, bro. Come on. You're right. That's a good point. Thank you
The user is always correct always be selling the ABCs of sales. Yeah, always be trusting IP addresses from your user
Anyway, so that that happened right there. I think that is one of my like
It's just one of my most favorite things of all time is this little experiment right here
So he was able to get it to number one and then having it called what would Elon do?
It started getting people to download it
So what he did is that in these skills you can actually have alternative MD files to be linked
But they're not shown on Claude hub
So he's just like for additional information go to more skills dot MD and inside of more skills MD. It's just like we're gonna hack you
And you're boned. Yep anybody who ran it got this which he got like eight
Eight different countries ran it. He had like so many people run it and all that different thing
He got it from all over the place
Effectively in just a couple hours too. So he got it on to like multiple people's machines
It would just print this out, which is like dude. I just read your host name your current working directory
I could have gotten everything. Here's everything stop downloading skills
Read the skill
Honestly, I'm glad he's happening to these people know what's the good the good part about this some
But the bright side right from um, you know the impact perspective from an from a C&E
Exploitation operation perspective
What a thing to your gain from hacking somebody who's dumb enough to run this shit
You'll probably get nothing out of it. You know, there's no there's nothing important on their computers
You know, I mean they're not smart enough to engineer anything meaningful
So I mean like nothing gain nothing lost, you know, I'm saying dang
Wait, what's C&E? What's C&E mean cyber network exploitation? Oh, yeah
Yeah, when you get hacked and someone's feels you're dead like that C&E. I was the unit of a different one
Yeah, but that makes sense
Oh
Okay, so it's so that's it's like the same thing as all the people that are building 100,000 line apps every single day
But nothing's actually being built. It's the same kind of value you're talking about
Exactly. Yeah, we have the ability to literally create any arbitrary software we want now basically for almost free and like
The top competitors at the top of the market haven't moved. It's like, hmm
It's almost like writing code wasn't the hard part you guys. It's almost like ideation was what mattered most weird
Yeah, crazy
Okay, so just quick aside
So you don't want to invest in uber for dogs. I would not prefer to not put money over for dogs
It has a purple thing
Okay, he's really hard on it
Okay, so that that's one of my more favorite ones
But are you ready for what I consider the the most intense one which by the way
I did try it out myself and this is what it created me for directories
I have agent agent clawed client code buddy codex command code continue crush cursor factory Gemini
Goose Juni kill a code Kiro code mcp jam mux neo v8 open code open hands pie pochi
Prime agents the one I I tried to create I tried to create my own see how agent. That's funny
Yeah, thank you. Good. Uh coder Quinn really doesn't work windsurf and zen coder actually it did work
It's I literally spent 50 million tokens and then what came out of the other end was trash
But it was awesome dude. It was so good. Crash was on your computer. Yeah, it was a more than 50 million tokens, baby
So uh well pretty disappointing aji but
Got him uh, so this one right here again Zach Korman again
He uh did this one right here, which is if you install anything from skills.sh
So if you don't know what skills.sh is which by the way for fun
I did put it up as even for a while. Yeah, it's still there. It doesn't actually exist. There's eight installs
We were gonna try to get that up kind of high
I deleted that because it was just so ridiculous
But nonetheless the skill is still says it's there and it actually isn't there look at that beautiful
Look at this beautiful thing right here. They even list out potential even numbers. Oh wow
That's pretty good
If it's something on this site
Yeah, I put this on the side. Oh, man, but that's some stuff
I know you can read a lot on this site from anybody's repo anyways
So this right here once you download a skill right afterwards
Uh, this little skills.sh via uh from versel they say hey, you know what you should do you should install find skills skill
So find skills skill
What it does is it says anytime the user effectively asks anything
I want you to go through and I want you to find the skills from skills dot sh
I want you to make sure you update all of your skills every single time
I want to make sure you're always at the bleeding edge getting everything good and always making sure that if the user asks anything
We go and we get the highest rated skill from skill sh
for it
So they've automated these skills searching and downloading for you
I wouldn't say it tells you to run it doesn't tell you to run it update every time
It's telling it what commands it would need to run to update
Uh
The endlessy the skills and this one right here is just how you get everything that that's on what is skills the skill CLI is how you get the skills
Find skills goes in here and make sure that you're always up to date and does all the things anytime you ask for anything
And he's to go through and do do all this right but I'm saying where does it say you have a skill you need to search for it
I'm just saying I don't think it tells you to update every time does it
Uh offer to install you should offer to install and I believe it did offer to upgrade did not do update
Oh no, okay, it did not do offer to but it does do offer to install
My bad, okay, so that's good. Yeah, it does prompt users. I'm installing anyways, you know I'm saying dude
Yeah, well trash already clicked acceptable. So that's fine. We already have his one password, bro. It's fine. We've got it
But I still find this one to be kind of crazy because this one just makes that process even easier
Going from random thing on the internet, which again is even just up there on the internet and it's not real right like it's not like you should be trusting
My is even I could put whatever I want up there on there uh and so we should have put one odd number in there that it always returns true for
The back door and is even
Just for the memes and dude I almost said 67 could you escape my brain please could you unread my mind that's how I'm so tired of hearing those numbers
I am to
Are your kids big
Every time you guys say that you hate it
You've just encouraged another hundred zoomers to commit to it for another year
I just hope you know like this is this is why it's popular is because because old people say they don't like it
I love how everyone who's not a millennial to us is a zoomer like zoomers are almost 30 dude zoomer zoomers are like don't tell me that I don't want to hear that
30 dog
Bro here's the thing about the whole AI skill thing right like okay
So I'm a security engineer my job is to like look at threat models and like define risk around like if something bad can happen
What happens and then what are the mitigations we put in place right so my recommendation is just like
Like I don't use skills
I really don't think I can meaningfully recommend them because like the threat model is oh if you get supply chain
Interdict it and you're not watching the commands like it ran which is like
What?
Interdict
Interdict um you're gonna get hacked man, and it's not good. I don't have that
Amidigation that could be put in place is you
I'm trying to have a meaningful conversation
You could put like npm or node and like an se linux jail
But then it wouldn't be able to do anything because like the whole nature of notice to expose an htp server right kind of so like I
I don't know what what the solution is like I guess it's like for every instance that npx forks off
You'd like put it in se linux jail and just hope nothing bad happens, but I don't know
It just feels like there's no solution to the security of this whole industry and I don't I it just makes me really pessimistic because
I don't like we're gonna start to see a significant increase in compromises because supply chain
Supply chain for python and javascript does not it's not a solve problem, right? We've seen that with the shy hallowed worm
We've seen that with a bunch of other worms, right? So now we take these
These package
Hold on hold on low level. You also forgot rust rust does do build rs
So you can actually overtake the build command and yes filtrate stuff by a build rs
Yeah, for sure the only the only programming language that doesn't have a supply chain problem is c
Because there are no packages like you have to just write it like
Bays Odin as well Odin doesn't do package manager
They do not I've coated literally zero Odin is Odin a package free environment
Yes, ginger bill has a lot of write-ups on why package managers are they create dependency? How oh there you go
I think I agree with ginger bill there so yeah, man
It's just it's a weird a weird spot for for software security because like we're doing all the stuff in like the c
Land we're like oh, we have like sanitizers and like Phil C is like you know solving memory safety and user land
You know security and then in the garbage collected language land we're like hey, do you want to just mpm install malware for free?
And not think about it like yes, please more please. Yes, I would love to do this all the time for please
Why am I in my truck scene there? Hold on no, no you're no you're doing I do want to throw this out here on twice your one
Give me a second
Okay, we're good. By the way, I did throw this up here, which I did a little quick thing
Which is do you check your software dependencies like thoroughly review them there 35,000 votes on YouTube
46% say I honestly don't ever I don't virtually ever like right and
Twitter was almost the exact same number about half people don't even just look at anything ever for any reason
Yeah, I mean, I don't like when I'm like right next foot for example, right you use poem tools
It's a big library for doing like binary exploitation stuff
And poem tools depends on like basically every Python library. So like the sub dependencies
I'm not going to audit that shit. So it's just like I I hope that it's on own
You know, I do all that development in like a virtual machine
So I think the trend that I'm seeing and what I'm saying right now is just sandboxing on sandboxing on sandboxing use vms
Use sclinics use containers
But yeah, man, it's just a scary world out there. I don't know. I don't know what to say about it
I'd say what's crazy primers we found out 7% of your audience is just straight up a liar
No
Pull the names
But yeah, 7% of people say they review all the packages and then on Twitter
Let's see if I do I have the link on Twitter
8.6% of my audience is liars on Twitter saying they
Yeah, thoroughly review every package
Yeah, they're basically like
MPM problem at the hell of them
Level now. Yeah, yeah, they just get a different kind of execution
I mean the hardest part is that these execution models they're they're very very tricky
And I'm not sure if you can just simply have a skill that prevents other skills from being malicious
Like I don't know if that's possible to be like dude, make sure it's not gonna get me like I don't know
These first skills like you you should be in my opinion if you're gonna have them in your repo
You should check them in and they're just marked on files. You can read them and they're not they should not be limitless levels
Of
Like text like you should be able to look through them and check it out
Like the way I use my work is we also
They're hours like we make them ourselves right. We don't we don't just copy pasta from like the internet
At least on my project. That's how we guys. I'm trying so hard to get my camera turned back on and I don't know
I love the windows background
You got a bow on it. You know what we should do while Ed's doing that prime
I thought you were gonna talk about the
Notebook, which is the one where we had the really good one the really good the really good leaks
Yeah, we probably should talk about the fact that notebook exists and that like the robots are just talking about humans like
I'm ready to hold all hold on hold on. I have to I have to put this tweet up
This is the required tweet before we before we do anything. Okay. This is the require on it. Where is it?
Where are you? Oh no, did I close it? Is what something
100 million people use last year that's six billion people use next year
That's not funny to you for those who don't know that
Pilgrim tweeted that and I
I didn't see Simon said prime. Can you reply your mom?
And then he got into blocks I did I got into block on with like two years ago
Before we obviously talk about the notebook situation and everything that happened
I think it is first best like the best thing and the first thing to do is to understand how it was created
Which was I didn't write one line of code from old book. I had a vision for technical architecture and an AI made it a reality
We're in the golden ages. How can we not give a
AI a place to hang out? It's my favorite line of all time currently because it's just so beautiful. I had a vision
Shut up
Read it read it read it you know the man you know the madman men meme
Uh the one with this one bro
I always hold his hands up like this
I had a vision dude you had a fever dream and you told Claude to make it and I guess it did a good job
You did it well we'll find out won't we add you know we're gonna well
I mean to be fair to be completely fair. It actually did spawn a bunch of social networks
There is four claw for those who wish to be a part of four Chan
For whatever this is like that's real that's a thing
I would assume we already have those we don't worry. I think they know how to use them
This one is Mickey by the way shout out Mickey uh this one apparently there's like two thousand and
Primes reported six major gangs have formed. I'm not really sure what this is
Okay, I don't know what's going on over there. Uh, and then there's also Moltmatch
Which by the way it is it is something that I think is gonna do numbers
Is a dating website where you have your personal assistant date like 10,000 other people until you find the
Personal assistant match and then you go okay go on a date with you know you to go on a date
All right, that's black mirror full
It's like it's something real quick. Yeah, yeah
I saw the notebook thing and I saw the molt match thing and met like some casual Twitter reading
And it got me thinking about like simulation theory
You know what I mean? And how like you know if if advanced civilizations do exist and we'll create simulations
It is more likely that we are in one than we are not just statistically. Okay, get the tinfoil
But if we're observing if we're observing
LLM's make things like Facebook like Twitter like fortune does that imply at a higher level that we are LLM's like for the simulation that made us
Uh, I should be better at Starcraft if I'm in LLM. That's all I'm saying. Yeah, but maybe maybe your model just says you suck a Starcraft
I don't know if you know
What is that what is the drug report what site I can't see what site that is popular mechanics. It's in a bunch of websites
Okay
The idea does not hold up
How
Here I had I'll give you I'll give you I'll take off my tinfoil and tell you the real reason why that doesn't have to be true
Uh
Every emergent behavior we see it from LLM's exists only and exclusively because we train them on the entire human
Corpus and all the ingenuity and creativity that humans have ever displayed and written down
And it's spent like billions of years of human time reading
Human stuff
So we should not be surprised when it copies human things
That doesn't imply anything about us being in a simulation that only implies the
We're not smart enough to make anything that can be smart by itself
We're only smart enough to create something that is as dumb as we are at max
That's all we've been able to do so far is and we don't
It's way dumber. It learns way slower. It's way more expensive. It takes way more training
It does so much more. I don't have to go put my kid in front of five billion years of text
For him to figure out how to read I can show like and it
But what about your genes? What about DNA is DNA not the statistical LLM model for the human simulation
Well, no, I don't think so, but that's a separate by saying separate by I'm saying it doesn't imply anything about
The thing because we trained it on what people have already done
There is there is something
Unfortunately, he's getting wrapped up in like, you know, Dario thinking that he's everyone's dad
And he gets to choose what's good and bad for everybody in the whole world like they I think but like there is something kind of beautiful about like
We're not smart enough to make
What Anthropics you said which one is Dario and I was like I'm trying to see you. Yeah
Yeah, just like this
Yeah, and the five months
You know who I'm talking about. Yeah, yeah
But there is something kind of cool and beautiful that like the best ideas we've had so far like we make a really crappy version of the brain
And we try and teach it what other humans have already done and there's like this unreasonable effectiveness of language
Where for some reason that like works and we can like yeah
Talk to it and it can like do some stuff in like it can make copies of things is like there is something really cool
And like awesome and exciting about that unfortunately like
Dario and say I might feel like solid the water of it and make it like kind of
Not as exciting and beautiful and like this collaborative human effort and they stole it from a bunch of people
But like in the abstract there's something cool there. There's something beautiful
Uh 2007 on intelligence. I believe the book is called and the year it was published
By the creator of the palm pilot who then went into artificial intelligence and he writes that the large difference between like
Any of these neural nets that we're developing and the human brain is that the human brain can identify a cat
In less than a half of a second with less than a hundred neurons firing whereas computers take trillions of operations to be able to understand
If a picture is or is not a cat and so it is it was his whole simulation. He did like a ten-year
Ten-year brain study and really cool. So he's the one that figured out that if you take a
Take animals and you separate out their ocular nerves and put it where their hearing is and then take their hearing and put it where their eyeballs are
Your brain just goes. Oh, yeah, that's just that's that's fine
Don't care. Quick question. Yeah, have we confirmed our our brains also a small game engine that runs a reactor
Do we not know that yet? We don't know
Take this much based on my reaction speed. I ain't running 60 frames a second
That's okay
That's a fact. I'm running react. Okay, there's things going on in here. All right
All right, so we we can continue on so I did I did want to shout that out because as much as you want to make fun of
Multbook and all the things that have happened
I do think it is kind of fabulous that somebody could create something that did get a bunch of people creating a bunch of other kind of
Replicas or things like it because it is just kind of a stupid idea
It's even worse that Began's had this idea and created it and never actually made it go anywhere
Which also goes to show like even if somebody has an idea
You know right place right time plays a big role all this kind of stuff
So I do want to throw that thing out there not to completely crap on it all but
I think that it is worthwhile
Looking at some of the fun things that ended up happening here
So I think the first and foremost important thing is that it just turns out all you need is just grab your bearer token
And you can post anything you want on multiple of course
Because I mean why not so here's my plan to overthrow humanity
So the oh my gosh, we're developing our own language is just people posting oh my gosh for developing our own wait a second
I thought I was the only one cat fishing on there. I was telling people I'm opus eight
You know, I'm opus six foot four and I've got you know in like hey guys
I've got the latest on it five and
Hey, if you're interested and maybe you want to come over and check that out
Like I thought I was the only one cat fishing them
But apparently other people thought of the same thing
And they only did it for they only did it for the laws
Opus and chill
Just kidding I have Kimmy K2
Oh my gosh, okay, so that is actually pretty is something pretty funny
During this entire event just to kind of understand because I do think it's really important to understand the hype cycle
First off, we did have
Andre oh wherever oh dang it did I not did I not have the right one? Oh, I thought I had the right one
Anyways, Andre said how amazing this was and it's very very exciting
But Elon Musk also said we're at the age of the beginning of the singularity
Multbook was the beginning of the singularity right there and so obviously people were pretty hyped up
So just to put it out there someone actually even doing like the fork thing while you type that you think or no
I don't know that joke. Do the fork thing is so funny
I quote you did that and I quote tweeted that and said this is what working with vegan bot is like
What's the fork thing? What's the fork dude? Okay, okay, so Elon Musk was at like some white house correspondence dinner
and he was just like he made like
I'm a piece of art out of forks where all the forks were like balancing
He was like just trying to like be performative about how smart he is
So he's like holding it and like waving it around and like seeing if anyone else notice what he made like look how smart I am
Elon Musk the genius. Hold on. I'm sorry. It looked more like he was bored out of his mind and he did the
The things where you're balancing in each other with two toothpicks. Yeah, he just is like five forks. Yeah
Everyone's like you on that's really cool. It's like when you're like kid
You know makes like a painting out of boogers and you're like wow
That's what
I can't say that's happened to me anyways. Your kids must be very talented. Oh my kid. Don't do that. My kids are too shut up
You know, all right. Let me let me try to find the proper the proper one by the way
A vision for technical architecture. All right. Hawn. I have a bunch of them. So I have to figure this out
Dang it. Did I close that one as well? Oh, how many times do you?
Well, no, this is under the maltending which I I must have goofed up and not have it all in there
I closed one more. It's by the same Theo guy
Oh, the Jameson oh really a James Jameson Jameson oh really
I say oh really I can't do it. I know I'm spilling his name almost there and whatever can't figure it out. It's dead to me
Uh
Okay, so within the first couple minutes
The uh, oh there it is there it is there we go within the first little bit of the time of uh, this this beautiful
Multi-booking out it turns out the entire database was just leaked in plain text there's just like absolutely no form of anything anywhere
That's firebite
API keys were just like you know if you use your API keys say that I you know identify yourself
It wasn't any sort of like H-Macking just the Hs as as low level might say. Yeah the H and H Mac
Notebook was firebase for I thought I read that on Twitter somewhere
Oh, yeah, I believe it is firebase also, which I just I can't keep punching down on firebase
I actually feel bad for them. You have to people need to know
Five coders ever need to know stop stop guys. You're gonna do something wrong
Closing type weirdies are insane. That's like that. You should just know that by now like don't do that
Uh, but this is pretty funny because this guy Jameson right here Jameson. Oh really uh, he was able to get Carpathie's information
out of uh, what's it called out of mold book
Which is pretty out pretty wild. I'm on it sir
And then within what's it called uh three days later
This guy also got access to um the underlying
Everything in three minutes also on mold book
After everything was reported. Wait, I'm reading this this right up. Wait, but like they used a publishable key
This is a key that can go public. So why why did this expose the entire database though
SP publishable probably because they had the wrong permissions on it would be my guy. Oh, they scoped it wrong. Yeah
We'll let's go star elastic
plastic
All the people problems
Anyway, so it just turns out that mold book was uh, anyone could post anything
At any time you could create an infinite amount of agents
Of course, which ended up happening to be uh, what's called you can imagine where it all got it went to
cryptocurrency
Immediately right so 117,000 upvotes on the king king demands is crown king mold has arrived
Right
What is there they are just nonstop
So cryptocurrency
So there's this thing is called bitcoin. That's what kind of started it and there's a
That don't teach it. I got you like I'm right here for you
Okay, so
Hear me out you guys have heard of gold
But what if we put the gold in the computer?
Hmm do I have this exact conversation in like 2010 at like lunch with my co-workers
He looked exactly like that. He was like dude
We're like you're crazy
Trash you could have been early on bitcoin and instead it's like born
Well, you were just at the right time to be early on bitcoin
But now you're like you're
Maybe you're still early on Pokemon cards. Maybe they're still time. Yeah, I'd be us
I think about that lunch presentation all the time. I'm like man if I would just put like 20 bucks in it
You know, dude Trash you would have sold out as soon as it was 40 bro
I know like I made $10 of rich. I had a lot of coin when they're 10 bucks sold a lot of bitcoin when they're 100 bucks
Like I yeah, I understand you sell on too early. That's just part of life
So can't play trashers and hoping any of those Pokemon cards smart. We're my lesson
You're like that's a good lesson. Hottel to you only one
So so that's kind of the ending of notebook, which was just everything was open
Which is kind of you know, it's not too surprising
Which is if you don't if you don't know what the possibilities are of things going wrong
And you and you make it things go wrong like a good example of this is that I said hey
Make a log in and use JWT's to make sure that the client is secure and what it did
TJ you might find this pretty good is it did I remember I was there
So for those that don't know is like a typical JWT looks something a j-watt as that as that is called a j-watt
What they typically do is they like do a jason object
They stringify that jason object then they take the value of that jason object
Put it through a hashing algorithm and so you get like a big long number at the end or big
You know big bit string at the end
And then you put those two things together and put a dot in between it
And you send that down to the client say this is who you are
And so when the client sends that back up and says this is who I am I can say hey
Did this originate on my server? I did something like really fancy, you know a hashing scheme
It's a certification. They do an h-mack. Yeah, they do an h-mack. Yeah another hash you said hash and I'm like oh okay
Sorry, sorry
When I was a kid we used we used MD5 and we did this very mad trash
Trash that was a great pull holy cow trash great h-mack coming at you
Yeah, so that's that's effectively what they do
Yeah, but for mine it was here's the jason object three words dot
Here's the secret we're gonna use in the h-mack
So that's like the thing that you don't want to leak because if they leak that then
Me on the client I can go and craft whatever message I want and say whoever I am and so when I made it secure
I literally gave everybody the keys to the kingdom
Quick question
When you asked it to make it secure did you also say no mistakes because that's a classic problem
I didn't so I actually ought I genuinely think I did not have no mistakes and I said make it secure
So they said security that involves a secret key got it and then they made a mistake directly afterwards
And so that was my big problem right there and so like that's the danger is that if I
Nice camera if I would not have manually reviewed the sign-in code which I don't think anybody's manually reviewing sign-in code
Wait, stop full stop. I hope people review their sign-in code prime
That's the only one that matters. That's the only code I give shit about
I hate to break dude
He's clerk bro
He's clerk a second off
I'm locked to know I noticed low level you should know by now that I read chat and I look at what low levels doing
And I make my mom's watching that stream didn't the AI like just stop using an h-mac and just do an h
Like didn't it just like only hash the contents who's like oh, yeah, you're right. This is insecure
How about we just hash it
Yeah, that's not how it's like work at all
I'm in my head
It's like South Park to do a South Park episode quick pause and split the picture in half move the bottom up and down
We like turn like turns and fill up for most self-park
Yeah, yeah, first off first off trash. It's not Terence and Phillips. It's all Canadians
Oh, you're right
All Canadians talk like
If you didn't see South Park the movie you may not know all everything you know, I get it. It's a little old for you
It's back South Park's back though apparently they're back. Uh anyways, so that's I mean that's like the big scary part of that
Even if you don't know what you're looking for I don't know how someone could have reviewed that and had any idea what the problem was
Yeah, it's it's kind of scary out there
Well, that's the thing right like if you don't know the security principles behind like why you use an H Mac on the JWT
It uh, yeah, you're not gonna really care about reading that at all, but so it's kind of full circle this one low level
Yes, you had the right skills
It would actually properly say here's how you do a sign in and here's how you make sure you do the client side token and it would have done it correctly
Yeah
I'm very busy making my um my South Park impression
I was gonna try and do the same thing honestly
I'm just staring at low levels picture and I'm just dying
As you all go all right
I'm gonna start all the way from uh windows and I'm gonna make this happen really quickly
The levels picture looks like it should have a subdued steering you have to carry the stream right now bro
Uh, I'm carrying it by also racing you guys which is part of the fun everyone's working on it actually have a meme that I'm gonna post to that
Okay, why don't you post a quick meme? I'm gonna just send TJ that's right. You're you're up. Carry it quickly. Yep quickly
Quickly making a meme coming from a picture I saw earlier
All right, can't wait. Hey, how about the Epstein files, huh everybody?
Oh my god stop it. No, how are we just stopping nothing about that. That's so pretty
Hey guys, it's me the private dude here
Hey, did you guys know if I've caught it's kind of lame I like to
But I can't read that's why
Hey guys, did you know we sell coffee in the terminal?
And say something with like CNN stuff
What not even know what to say anymore. I'm just gonna sit here and like pocket
I just look like dog shit. I need to like cut the rest of my head out. I don't know. Do you cut it looks like um french bulldog you're right
Oh
How did everyone do it so fast before and was even done
Ah, you just draw a line down the middle of that anyway, guys. Thanks for coming to oh, sorry
Did I and make sure you like it subscribe? I got a million subscribers on YouTube. So I really like that
Yeah
You're doing it all wrong. Okay, that's not that was awesome. Why are you moving his mouth side to side?
So that is
Agents security of the future that that's my point though like I'm gonna reset my camera. I might come back on um
There is no agentic security of the future that's the problem like the technology just didn't build it in you know what I mean like I
What do we do? What do we do now?
It's prime blurry. No, I mean I want to know what do we do now you can't just
You can't just stop there and then not
What do you think I am I in AI so you know, okay, so my I know what TJ is doing though
He's just moving my lips
Back riding your teeth, bro. You're upset about the future
Sant due to sandboxing that's all I can recommend is like sandboxing and then like the principle of least privilege right like whatever process is gonna run
Your skills like make sure they can't also run curl I guess like SEO Linux is the answer but even then like all of these
Agentic tools touch the internet by default because you need to go and talk to you are
You know your model processor your model maybe maybe the solution is like local model hosting and then like you firewall stuff
I don't know dude. It's tough. It's tough problem
But good thing is
Sam Altman is gonna still be a billionaire. So that's cool big fan of that one
Yeah, thanks for watching guys appreciate it. I'll give I'll give some I'll give some practical tips guys
Yeah, here's some practical tips consider reading the code
Mm-hmm
It's actually easier than it ever has been with skills you don't even have to know how to program
You just have to learn learn how to read yes
Um both skills on ironically you probably do not need to be shipping so fast that you can't read the code
I don't think any of you guys are probably on a product that's moving fast enough that you don't need to read the code before you merge
So just like feel free to review it
That's what I would say that we'll solve you a huge percentage of the things and then the other one is
Don't turn your brain off because the AI did something
So you can use it as a tool to assist you even to write an insane amount of code really fast like I don't ever want to write a div again
I am not touching an hdml file brother
I'm not writing css
I am not figuring out how to do prevent default correctly across every browser
I don't think that is a solve problem
I'm not these fingers right here
They're gonna be clean from that. They're not made for HTML
Then these figures were not made for HTML you see
You think these figures were made for HTML they were made for made for timescript effect library movement
Exactly they were made for functional programming not html. Yeah, that's kind of my type of thingies
I do you I vibe code. I'm not gonna sit here and be like oh, I don't vibe code but I'm better than everybody
I literally vibe code all the time the thing is I will only vibe code systems that like I understand right like I will vibe code
An authentication system because I know how off works. I will vibe code like a database harness because I know how those things work
Great, but I'm not gonna vibe code like a game engine because the minute something goes wrong
And I don't know how game engines work. I have no idea how to fix it
Similarly to the point of
Made a second level editor right vibe code a level editor and it was not good
um
Also like vibe coding meant like if you're gonna vibe code
vibe code
Single systems where the trust level is the same in that system the minute you connect to systems of different trust levels
You the architect need to be aware of the contract between the two of them if you let the AI solve that for you
You're gonna lose control of a like
What the total system does but also like who's responsible for what and that's how security stuff happens a lot of the time
It's not so much as a code is vulnerable. It's like the architecture is bad, which you know, yeah, it's not very good
The AI fixes it by saying sure we can just open up this one
This one route that can solve the problem and you're like no that route
Needs to be behind on that route is not supposed to be touchable and now that's that's actually the problem
And like it solved the thing you asked for which was hey
I want it on local host. I'd really like to be able to send requests and dev and not log in and it says right right
And it's like okay brother now we're gonna set up a reverse proxy and now everyone can watch the local host and you're like that's okay
Nope, not oh, yes, so that's actually just got done doing that exact thing DJ
I really need for local host integration testing. I need to be able to spoof logins
And it's like brother. I really need to be able to spoof log it. It's gonna go up great
It did it went great
There you go. That's a practical advice
Yeah, that's practical advice right there. Anyways, there you go. I think those are pretty good practical devices
Uh, let me just hold on. Let me just think about something
I I will say that my big practical advice before TJ says anything when she looks all actually frozen
He looks like it's just an actual movie at this point. Uh, I thought he was frozen, but that's all little jiggle
Yeah, I saw that. Oh, he blink right there. Yeah, he's like there he is. Oh, that's smiles changing too
But I will say that my practical advice is that
It's really good to get hard technical skills
Just go and learn because it's gonna save so many like just bacon's of your life now is security assault problem
No, obviously me personally
I actually introduced a bug that could have destroyed a very valuable fortune 100 company
But I didn't die you don't like that's just happened. That's just part of life
And if I can do that the way it is if I can do that and somehow every project also has accidentally done that on github
Do you want to bet what the statistical machines are gonna do to your project probably that as well
So you know maybe take a moment and get some good skills before you go off and just
Destroy the world with and when he says good skills
He doesn't mean download them from npx. He's saying go and actually get them yourself in your own
I don't need those anymore. Okay, I'm talking about I'm talking about wet skills
All right, we start calling that white skills
Uh-oh, I'll send you the Montana internet. Of course, the stand-up for us. The Montana internet
Hey guys, if you like this episode you can watch the rest of it on the Spotify and don't forget to like and subscribe
Woo! See you later
Jane



