SN 1095: AI-Driven Expertise Loss - Gemini, Hugging Face, and the AI Arms Race

0:00It's time for security now. Steve Gibson is here. It's Patch Tuesday and you won't believe how many patches Microsoft just shipped. Steve says that's okay. They're fixing things. New versions of Google's Gemini, Nvidia acquiring Hugging Face, Chinese Cyber Espionage, and then a lifelong safety engineer worries that AI is going to cause us a loss of expertise. That a whole lot more next when security now podcasts you love from people you trust. This is Twit. This is security now with Steve Gibson episode 1095 recorded Tuesday September 8th, 2026 AI driven expertise loss. It's time for security now. Yes, Tuesday has come around once again, and that means Steve Gibson is knocking at the door ready with a

1:0622 page document of all the latest security problems in the world. Hello, Steve. Yo, Leo. It's good to see you. For those who looked at the show notes, Benito was the first person to highlight the fact that I had numbered this 1096. This is not 1096. This is 1095. Fixing it right now. Yes. So the links are right, but the in the show notes, I had it wrong. So I had anyway, we are this is and I have not yet looked. We are on patch Tuesday. And a big question that we have is, how does this one compare to the last one, which of course was a whopper. Your theory is it'll get better. I believe I don't know what the shape of the curve is. I don't know how soon it's going to get better, but this in theory, as long as AI is aiding us in eliminating more bugs than we or it is creating, we should be seeing a drop off because we certainly are saying that we're paying for a lot of the legacy

2:20code mistakes had been made in the in the, you know, previous era. So we're going to start out talking about a classic old school hack against Dropbox that actually something happened that was not about AI, believe it or not. We've got, as I said, the patch Tuesday after today is going to be enabling something that Microsoft first, it's sure, it first appeared in Windows 10 and we talked about it back then. The short name for it is memory integrity. Many people turned it off because it dropped gamers frame rates in the interest of improving their security. A lot of gamers said, I'm secure enough. I need a high frame rate. Anyway, it's going to be turned on next month for everybody that qualifies based on the hardware. So we'll talk about that. Firefox move to 155 and obtained what I call a dumb smart window. So we'll touch on that. Sissa has terminated six of its most valuable cyber security services.

3:35And really, you couldn't choose a worst time for Sissa to back away from, you know, infrastructure cyber security because everyone's expecting AI to impact that. Well, everyone except me. I'm not convinced that that's going to happen. We'll talk about that. I want to touch on open AI's big announcement because it last last week. Since we talked last, lots of things happened. Open AI has released GPT 6 Astra, which is, you know, that model we did discuss it before that they're saying qualifies for what they consider critical security treatment. And then also there's been a weird bunch of what I consider misreporting, angst generated by the, you know, the anti AI guys saying that open AI is doing something, which is hiding its own internal thinking, masking its chain of thought. I'm going to open that up, take a look at it and explain why that is not.

4:47What is going on also will touch on Nvidia's acquisition of hugging face and who that's good for Google release Gemini 3.8 flash. You know, I'm having waited what two weeks from 3.7. So this is all happening very quickly. Chinese cyber espionage is beginning to use AI to become more slippery. We'll talk about that. And Matthew Green, our favorite Johns Hopkins cryptographer has a fascinating take on what it means for there to be an AI driven bug drought, which it's really deep. He has some very worthwhile thoughts. And then today's topic is AI driven expertise loss, a lifelong engineer who has designed the control systems for nuclear reactors.

5:48I think has some very important things to say about the consequence of AI being so good. And what it means for in the long term. So lots of fun things to talk about. And another great picture of the week. So yeah, this is episode 1095, despite what the show notes say. And do you want to know? Do you want to know how many patches there are in the patch Tuesday this week? I do. Are you curious? Well, Tom Warren on the verge says, I understand today September patch Tuesday is about to set a new record with more than 650 security fixes for windows alone. Whoa. Six and 50 last month was 400. We thought that was a lot. It wasn't July was 570. We thought that was a lot. It was a lot. Yeah, 650. So it's not going down. Not yet. And Microsoft says, yeah, we're using AI. We're finding them. Wow. So there you go. I mean, and so it's a mixed blessing, right?

7:00It means that we've all been writing on this just Swiss cheese. How does it even boot operating system? Wow. That's great. Well, so it has it. I don't think Microsoft has actually released it yet. But that's what Tom Warren who's very well connected and usually very reliable says that that's the number. So a bunch of operating systems are going out of extended service life. Oh, no, no, it's got extended. We're we have it until October of 27 and other year. And that's really good because by then they should have windows 10 finally bug free because you know, we are getting security patches backported to windows 10. Yeah. And at the same time, they're no longer writing any new stuff to screw it up. So it's perfect. This is exactly what you want. You want another year of fixes without them adding anything new that's going to destabilize it. And then when it finally goes out of service, if they actually do take it out of service next October of 27, it'll be a solid operating system.

8:09So I'm going to correct myself about 10 minutes ago, the Sands Institute put out their bulletin and they said September patch Tuesday has 973 patches, including 130 riddle, 13 critical patches. So we are in uncharted territory. Two vulnerabilities listed as exploited in the wild. None were publicly disclosed before today. windows privilege escalation and critical RCEs and Skype for business MSM Q and your 1000 in one month unbelievable. That is stunning. Yeah. Wow. But as we should emphasize, it's good. They're getting fixed. Yes, it is. Remember the day we used to have 23 or 12. And I thought that was a lot of quaint days. Sands Institute is reliable. Right? I mean, they if they say it's that it's that that's unbelievable. Yeah, I'll have full coverage of it next week. We will understand what the demographics of those 973 are.

9:26And the 113 critical ones just how critical. So wow. Anyway, I would say update windows update your windows. Yes, sir. Yes, sir. Wow. Our show today will get to their picture of the week. We've got a fun one for you in just a bit, but first a word from our fine sponsor, Doppel. Actually, when you listen to this show, one of the number one thing Steve has said again and again is the real security flaws increasingly are coming from inside the house. And that's because social engineering is getting better and better. So you may patch all the fixes. But the bad guys don't need flaws to get into your system. AI has made social engineering attacks more convincing than ever. So your employees may be giving away the keys to the kingdom unwittingly responding to fishing emails, fake websites in personation attempts.

10:28It's getting harder and harder to tell what's real from what's designed to see. And that's why organizations need more than a collection of point solutions. They need a unified approach to stopping attacks before they reach their people. And that's Doppel. Doppel is an AI native social engineering defense platform. Doppel strengthens human risk management by training employees to recognize deception. It provides digital risk protection across every channel. And delivers a genetic email security that doesn't just score the inbox, but takes down the attacker infrastructure behind the message. Let me say that again. It takes down the attacker infrastructure that sent that fishing message. Doppel protects against the entire social engineering attack chain with one comprehensive platform. You get digital risk protection, which detects threats across multiple channels. Links alerts into a real-time threat graph. That's incredibly valuable for using you.

11:32Know where the problems are coming from. And then uses AI driven infrastructure disruption to stop attacks at the source. These insights also power fishing simulations. So you're going to get training security awareness training because it's timely training trained based on the threats that are actually coming at you right now. It helps strengthen employee defenses through next generation training and testing. Email security inspects every message traces it back to the attacker infrastructure behind it and helps take that infrastructure down so the campaign cannot target your organization again. Doppel also offers best in class integrations and partnerships. So it's easy to works alongside your existing security stack. You don't have to change anything. Join hundreds of companies already using doppel to protect their brand and people from social engineering attacks. Doppel outpacing what's next in social engineering learn more at doppel.com. That's D-O-P-P-E-L. Doppel. And now back to security now and it's time for the picture of the week.

12:40So this is an XKCD one of our favorite guys. And I didn't give this a title because he had and it was perfect. The title of this series of cartoons is why as a mob, of course, Isaac Asimov, the famous sci-fi author, put the three laws of robotics in the order he did. And this is just pure genius creativity on in this particular one. So what we have in the left hand column is possible orderings of the three laws. So, you know, the classic ordering is don't basically any sort of reduces it for size. Don't harm humans obey orders and protect yourself. And of course, the full reading is, you know, under no circumstances harm a human being as the first law. And so we have obey orders and the fuller version is, you know, do what you are told so long as it doesn't conflict with the first law.

13:56And then the third, which he's shortened to protect yourself is, you know, you know, protect yourself so long as it doesn't doing so doesn't conflict with the first or the second laws. So sort of a clean hierarchy. And so this classic ordering, as I said, is don't harm humans obey orders, protect yourself with the earlier or with the earlier law preceding any of the laws. And when we're sitting any of the ones that follow and when in the proper order as Asimov intended, we see that, you know, all of this exemplified in Asimov's various sci-fi stories, his famous robot stories. And it results in what we call a balanced world, meaning everything works. Now, you reorder those. For example, switching the last two, we have don't harm humans, then protect yourself and then obey orders is last. And so in a little cartoon, we see the human saying explore Mars and the little cart, you know, the AI driven says, ha ha no, it's cold. And I would die, which we sum up as a frustrating world as opposed to the balanced world.

15:20Or we swap the first two rather than the last two. So first, so that move moves obey orders into the first place, don't harm humans into the second place and protect yourself in the third place. And we see a little cartoons of robots running around and atom bomb explosions and missiles fly to the air, which he summarizes as the kill bot, hellscape, because of course, obey orders comes before don't harm humans, meaning that if you told your AI to go do something bad, regardless of the consequences to people, it would. Then we have the case of all of them being reordered, essentially, obey orders is first, we've then protect yourself, which was normally third is moved up to second place and don't harm humans is in last place.

16:20Of course, that's not going to turn out well. Basically, whenever don't harm humans is not in the first place, you get yourself in trouble. So again, a repeat of the of the third instance, same cartoon, atom, you know, atom bomb explosions, missiles fly to the air and we get another kill bot, hellscape, they fifth one down the order is has moved protect yourself to first place. Okay, that's not going to turn out well. Don't harm humans under that and obey orders again is in last place. So here the cartoon says shows the the AI robot saying, I'll make cars for you, but try to unplug me and I'll vaporize you because of course obey orders down at the bottom protect yourself is given first place. And that's sort of what we've seen with some of the AI that appears to have escaped containment and we call this one the terrifying standoff. And finally, the last possible reordering is completely backwards the original laws were one two three, these are three to one. So protect yourself in the first place obey orders in the second place and don't harm humans in the third place.

17:40And this is another one of the kill bot, hellscape results with atom bombs and missiles flying to the air and so forth. So anyway, just a fun take on as a Maz famous three laws of robotics, which were, you know, elegant and simple and worked really well when you think about it is like, you know, you need to not harm people. And as long as you don't harm people, you should obey the orders that people give you. And you should also try to protect yourself as long as doing so doesn't conflict with either of the first two so clean, elegant, simple. Okay, so In the midst of all the AI cybersecurity related news, which has been saturating this podcast because it should. I wanted to start out deliberately this week's podcast with a blast from the past. This bit of news brought a smile to me because for a pleasant change, it has absolutely nothing to do with AI.

18:48And as such, it feels kind of warm and comfortable and quite familiar. It's a sort of news that we spent the first 20 years of this podcast examining. Okay, so what happened? Dropbox disclosed a series of security hacks occurring between August 4th and the 21st. And so last month during which attackers gained access and downloaded the private confidential and one would wish secure data, but not so much anymore belonging to nearly 5000 dropbox users. Now, as I said, refreshingly, there's no sign of anyone using AI anywhere. This was strictly old school. The hackers access user accounts by abusing dropbox's integration with Lenovo's identification service. And when Lenovo was confronted with this news, they stated that a legacy integration between Lenovo ID and dropbox quote could be used to improperly authenticate certain dropbox accounts.

20:01Right. So those pesky old legacy integrations that always seem to be allowed to endure right up until someone figures out how to abuse them was the culprit here. It seems that Lenovo ID users not using two factor authentication had no protection. Which allowed attackers to register with Lenovo's ID service and get this using the same email address as a targeted victim. Then somehow arrange to bypass Lenovo's email verification process. I get presumed that's where the legacy part count comes in. Then use the newly created account having somebody else's email address associated with it to then pivot back to the equivalent connected dropbox account, thus appearing to dropbox to be the legitimate Lenovo ID user.

21:05So depending upon whose account was hacked and what it contained, you know, all this, this was not a sweeping scope, you know, end of dropbox attack, still the consequences could certainly be quite devastating to the individual users who were affected. So yikes. Once again, legacy bad. Okay. So I mentioned about this memory integrity enablement happening coming next month next patch Tuesday for windows 11. Microsoft last week announced that next month's October 13th patch Tuesday would be enabling memory integrity, which is a security feature. As I mentioned, the first appeared back in windows 10. What's significant is it will be enabled by default on all windows 11 machines. And this will not happen for otherwise qualifying machines.

22:13If the machines user had previously deliberately and manually tweaked the registry or if the registry tweak occurred through a machine like an enterprise policy. So as I said, we talked about this memory integrity feature back when it was first introduced as and as part of what Microsoft called device guard. It is a very slick system that takes advantage of the multi layered address translation hardware, which is present in modern CPUs, because the multiple layers of address translation also have privilege bits associated with them. So what we have with this device guard is what's known as second level address translation, which allows a hypervisor, which needs to be turned on and running. So that's there. There's a little bit of overhead to doing this, which again is one of the reasons that that gamers who are all frame rate crazed deliberately disabled this when it began to creep into their systems and people said, hey, what happened to my brain.

23:29It's not as good as it used to be. So there's a hypervisor involved, which sets physical page access permissions, you know, like writing to the page or executing code from the page. And it's able to do that separately from and effectively underneath the operating systems own virtual memory, paging tables. So what's cool about this is it leverages the hardware from KB Lake on over on the Intel side. I'll get to this specific hardware issues in a second, but, but it does so in a fashion that really strongly prevents a bunch of traditional problems. So as I said, first of all, on earlier hardware, there was more overhead than there is on later hardware. So because Microsoft decided they this was so cool they were going to like do some emulation, which is never good.

24:34Because it because on the older hardware, this memory integrity being enabled noticeably reduced the systems among other things maximum display rendering frame rate. Now, and the reason for that, as I said back then, and this was a decade ago. So quite a while back, was it the memory management at that time, the memory management hardware only had a single bit for controlling access access. At this second paging level stage, and Microsoft needed two bits, but there was only one. They need to for each of the possible modes user mode and kernel mode and be able to enable and disable those individually. So at the time, Microsoft had to dynamically switch management tables on the fly. Anytime there was a switch between user and kernel hardware interrupts made that happen driver calls made that happen and so on. So the overhead was very real and among those who were really pushing their machine to the limit are gamers who noticed the difference.

25:50As I said, all that changed later with Intel's introduction of what they called MBC mode based execution control. And that first appeared in the KB Lake processor family, AMD called theirs GM ET and that arrived in their Zen 2 family and that changed everything. It removed the need for Microsoft to be doing any context switching essentially switching management tables as anytime you did a kernel transition back and forth between the kernel and the user. There's still a tiny bit of overhead because as I said, there is now a hypervisor active with this second level address translation. So larger paging tables will inherently mean more paging table cache misses. So that this will increase cache miss rate.

26:55And so there will be some impact, but modern hard were almost completely hides the overhead and it really does return meaningful security. So a month from now, October 13th, any machine that did not have memory integrity enabled will have that happen to it. So if something if like things seem to go slower after October's patch Tuesday, it's not your imagination. But okay, so here's what's actually going to change memory integrity has always been enabled by default for clean win 11 installs and so called secured core PCs like that that ship with secure boot enabled. And that's been true ever since the device guard days. And that's why because it shipped enabled gamers were frequently turning that off and seeing a performance boost.

28:06So what happens next patch Tuesday is that this is being extended that is from from from only clean win and win 11 installs and secured core PCs. It's going to be installed. It's going to be extended rather to any machines that may have been upgraded to windows 11 from presumably windows 10 or maybe you jumped over that from from seven or eight. And any that have may have shipped with it disabled with device guard disabled. So you need to have qualifying hardware for this to happen and Intel 8th Gen processor or newer and AMD Zen 2 processor or newer or if you have a Qualcomm Snapdragon 81 80 system on a chip or newer. All of that qualifies you also need need to have at least eight gig of RAM and at least 64 gig of of solid state, you know SSD storage and the systems boot firmware must have virtualization enabled so that has to be enabled down on the firmware.

29:23So the only action that anyone might wish to take if they were to notice any performance decrease after October's patch Tuesday. And if they're willing to trade off some powerful security protection would be to deliberately re disable memory integrity enforcement. Now why would you do that? What I have not yet enumerated are the various benefits of having memory integrity protection enabled. So for example, you get the absolute end to kernel shell code execution, you know that traditional exploit chain is for an attacker to obtain some ability to write their own code one way or the another into a buffer and then jump to that code. That no longer works if memory integrity has been on or after October 13th after it gets turned on all that gets shut down the hardware.

30:26At the hardware level it prevents any writing and executing of of kernel shell code. As we know attackers have also too often succeeded in bypassing windows enforcement of driver signature verification. Remember somewhere there's a jump instruction that decides whether the the signature matched or not. So if you're if you can manage to zap that jump instruction, you could just disable signature enforcement across windows. And since this signature enforcement is enforced by the same kernel that an attacker would have just compromised. Just as I said, one properly placed strategic right is able to disable that enforcement. The technical term for all of this is HVCI hypervisor protected code integrity.

31:26And with that which they also just call memory integrity hypervisor protected code integrity HVCI. If that's on then driver signature verification cannot be disabled. Of course root kits, you know, all those hacks we talked about long ago, which involve hooking and the API inline kernel patching to to do return oriented programming ROP style hacks self modifying or runtime unpacking of drivers. So all those hacks that depend upon being able to write to or execute kernel memory, none of that works anymore. So you may recall because we because again, I said, this is like 10 years ago that this guy added for it first appeared in windows 10. Remember the controversy that ensued when this first appeared because at the time of its introduction, many legitimate A V endpoint protection.

32:33Products were using the same techniques they were on behalf of the user as opposed to against the user's interest, but this broke third party A V in many cases. So the reason it broke it is it's no longer possible to patch the term they kernel in the long term. That's a good thing and here we've seen Microsoft do what they often do, which is introduce something give people a long time to kind of get used to it and get accommodated and then turn it on. This is what we saw with XP. Remember they famous XP famously was the first version to have a built in firewall, but it was disabled by default until you got the service pack to and then they enabled it. But you know, they gave everyone plenty of time to get used to that. So I have in the show notes a power shell one liner that anyone can use to quickly check to see whether their machine is currently being protected by this hypervisor protected code integrity, HVCI.

33:42If the command returns the word true, then HVCI is enabled and your machine has all of that protection that I've just been talking about. If it returns false, then it doesn't. So it may be that your system doesn't qualify. It could be that that your enterprise has been has disabled it for for some purpose. Anyway, this is coming a month from now and I think it was largely going to be offering a lot of benefit. And you know, don't turn it off unless something doesn't work. Yes, I would say don't turn it off unless you unless the performance unless you actually feel a performance change and feel for whatever reason that it's worth sacrificing significant security improvement for whatever performance change you might feel.

34:47It's you should not be significant if you are after and that's why Microsoft is only doing it if you've got KB Lake or later or the Zen 2 or later where you should not see a big hit. They'll be a tiny bit, but it shouldn't be significant. Well, let's take a little time out. I think that's what you were about to tell me. And we will continue with security now in this bit, but first a word from our sponsor, those nice finish people at Hawks hunt. I tell you I ran into them at a black hat. It was really funny and I said, hey, hi, they said hi. I said, I'm Leo. I do your ads. I said, oh, yeah, we know. And I said, just I got a question. Why is it Hawks hunt? What is HOX have to do with security awareness training? And they said, well, you have to understand that in finish, it's not pronounced Hawks hunt.

35:48And they're hunting for Hawks. Oh, it's a hoax hunt. Why didn't you say so in the first place? They're really very, very nice people. Then they gave me a nice piece of finished chocolate and I was satisfied. And I have to say they have a great security awareness training program. I'm sure you have one. If you don't like goodness, you should immediately call Hawks. But assuming you have one, the question is, is it running as planned campaigns go out? Right employees complete the training. The reports come in. You hand them to the boss. But let me ask you this question. How long have you been running it? And are the results still improving? See, for many programs, the answer there is no reporting rates level off. It's the same employees who keep clicking and the other guys just. Familiar simulations, but that's really the kiss of death because they're so easy to recognize. So people know, oh, yeah, here comes another one. Your program is active, but the point of the program, the risk reduction has completely stalled out. This is really common.

37:02And when employees can spot the same recycled tests from a mile away, well, they're not dumb security awareness just starts to look like security theater, a compliance exercise instead of a real risk reduction strategy. You got to fix this before the boss notices and I think this is the way Hawks on it's built to break that plateau that inevitably you hit. Instead of relying on static campaigns and last year's templates, Hawks hunt automatically delivers. Personalized fishing simulations based on the latest the current attack techniques, the content and the difficulty adapt to each employees role and to their skill level and to their behavior. So the program stays relevant. As both threats evolved and your employees evolve right they get smarter, they get harder tests right Hawks hunt also shows whether people are getting better at recognizing threats. How quickly they report them where repeat risky behavior persists. And how those trends change over time this way your team has more than just a completion percentage you've got evidence the program is not only running but actually reducing risk.

38:10Lionel Bazzell saw that that shift happening after they moved away from their, you know, old school legacy platform. Lionel reported fishing simulations increased from 1200 to more than 8,000 in two quarters while simulation failures fell 17% year over years. Senior trust advisor Dave Bang there put it Hawks hunts helped us break that plateau almost immediately. Hawks hunt is trusted by security teams of companies like Qualcomm doc you sign Nokia more than 3500 verified reviews on G2 go look at those reviews you'll be amazed visit Hawks hunt.com slash security now see what your program could achieve if it stopped standing still. That's Hawks hunt.com slash security now H O X H U N T just make sure you go to Hawks hunt.com slash security now we thank them so much for their support of security now and Steve Gibson Steve. So also last Tuesday Mozilla moved Firefox to release 155 the number of security vulnerabilities repaired was not alarming this release followed 154 by only two weeks.

39:27So it only had half the regular four weeks of time to collect problems but in the case of Firefox we're not seeing you know a stunning bug apocalypse scale problems being fixed at this point unlike windows. It's going to be interesting to dissect the patch Tuesdays Microsoft's reports to see what it looks like and I will certainly do that for next week. Also unfortunately I guess it's unfortunate Mozilla has begun progressively rolling out there there what they call an AI driven smart window. Lay out Leon I don't know if you've had any experience with this I had it on for a while I've had AI driven dumb windows but. Yeah so it occupies a you know a conversation column I guess I'll call it over on the right edge of Firefox's screen so it's taken up valuable real estate it offers a choice of three models with differing capabilities and also the option to choose your own I thought that was interesting if you want to choose your own you provide Firefox with the models name with its prompt and.

40:51It's prompt endpoint URL and also if it's required your API key or auth token in order to authenticate Firefox and allow it to prompt the AI model that you aimed at the three built in models they call fast flexible or personal. The fast one sends prompts to Google's Gemini 3.1 flash light the flexible model sends your prompts to alibaba's Quinn 3 and that's a 235 B a 22 B instruct 2507 M a a S model and interestingly between the initial release of Firefox 155 and 155.0.1. Mozilla's choice for where to send the personal model prompts changed it was initially using open a eyes GPT OSS 120 B and it switched to miss draws small 2603 so I just like losing screen space to anything that doesn't justify its loss.

42:08I had it on for a day or two and I tried to use it when it's on what would normally have just gone to my normal search prompt it intercepted and it turns out it doesn't know anything. Yes, not a great those are not great models are not they're old and they're not very good yeah well and it yes and apparently the goal is to use an AI to help you manage your tabs like search through your tabs to find stuff you can't you know it's on a tab somewhere it's like boy that really feels like they're stretching to yeah you know find some application for this thing so that you know it the the context it has access to is are the contents of the pages that are loaded and so you can ask it questions about your pages yet it intercepts general questions that I would normally would normally go to Google and then out to the internet more widely and it just kept saying oh I don't know about that you have to do a regular internet search is like well then what are you in the way for.

43:25It's turned off now so yeah not first this is why people hate AI because this is the experience of it that most people have is this kind of crabby AI yeah well like the little useless the dumb little you know how may I help you pop up that we get in the right hand corner of the screen and it's like you can't just give me a peep a person please. Okay so last week the publication cyber security dive reported the cyber security and infrastructure security agency we all know as sissa is scaling back the free assessments it offers to critical infrastructure organizations in a move that marks a significant retreat from the agency's core mission of helping secure the nation's infrastructure right I mean that's what it's for it's in its name cyber security and infrastructure security agency but we're not going to secure the infrastructure because well we don't have enough people anymore they actually the cyber security dive continued writing sissa confirmed to cyber security dive that sissa's regional staff will no longer perform its cyber security agency.

44:49Cyber resilience reviews cyber resilience essentials surveys ransomware readiness assessments incident management reviews external dependencies management assessments or cyber infrastructure surveys. Okay so I've been receiving I GRC receiving sissa's weekly automated cyber hygiene report ever since it came to light I think it may have been one of our listeners that that that pointed me at it I know we talked about it here on the podcast I'm recall that it's technically for infrastructure security I mean as is sissa so I always assumed because I was aware that it existed before but I didn't think I would qualify you know I'm just GRC you know a little software shop but after hearing from a listener that that I think that you know their organization was receiving you know had qualified and was receiving it even though they also were not really infrastructure I went there to sissa filled out the online form and got accepted so every Tuesday I think it's Tuesday I got all like I went yesterday so no I

46:07guess it may maybe I saw it this morning so it came early in the morning I've been receiving these free cyber hygiene reports so I was curious to know whether that service that I was getting you know it was not enumerated in that sissa announcement would also be shut down it turns out right away I did some more digging turns out that the problem is sissa's it actually is sissa's continuing critical staffing shortage which as we know because we have covered it resulted from the rather ill considered termination of one third of sissa's operating staff shortly after the Trump administration took office in 2025 and sissa's never recovered it seems that there was you know maybe not so much waste fraud and abuse at least in sissa so and you know we've talked about what a great job sissa had been doing so the back story behind the termination of those six programs is that they are not automated they require knowledgeable

47:22sissa cyber security staff to meet on site with infrastructure providers and that is what sissa is no longer able to support or afford so the good news is for what you know though you know all of all all of our listeners who like GRC are now receiving sissa's free weekly scanning and reporting service which really is quite comprehensive I mean this is a great service that sissa is offering will all at least for the time being continue to receive that free service the bad news is that especially now I mean given the heightened cyber security threat awareness levels being driven by the rapid emergence of ever more capable AI and assuming that the threat is real this would appear to be exactly the wrong time for sissa to need to scale back on its infrastructure protection services because the infrastructure is what we need to protect so I doubt that any or many of the previous sissa staff who were terminated last year will probably be re-hierable I doubt they can get a better job

48:49because I recently saw some news that stated that you know this aforementioned heightened cyber security threat awareness landscape was resulting in a basically a mass frenzied hiring of by private industry of anyone with any CISO style credentials and that they were obtaining support for the security and that they were obtaining salaries in the seven figures so you know while sissa's workforce reductions may not bode well for our national cybersecurity broadly it s likely been quite good for those who suddenly have found themselves well who previously found themselves jobless as a result but are now in very sought after positions by private industry. I would imagine they're doing far more and better now than you know now that they're in the private sector than they would have ever been able to do working for our government so you know I it's good for them.

50:01We need to talk about what is by far the biggest news of this past week in AI Leo I know you were you've been playing with GPT 6. Do you want to write down open AIs open AIs successor to GPT 5.6 playing with Astra yep and with it there was a lot of. The while there was a very specific report that we'll talk about so I want to address two aspects of GPT 6 the first is what GPT 6 Astra appears to be and the second is what's transpiring over in the rumor mill surrounding it regarding the dangers of something an unnamed sword. Unnamed source claims this model does it's known as recurrent depth also known as looped transformation or a shared layer architecture all of that will make sense by the time I'm done and it probably does do that I actually hope it does because I think it should.

51:21This supposedly it doesn't results in hidden chain of thought reasoning which that which thus would render the models thought processes invisible and thus unsupervised. None of that is true but the hysteria about rogue escaping AIs is fueling this paranoia I'll explain exactly what all that's about and what's been going on but first. What half open AI brought I want to begin by sharing part of open AIs posting last Tuesday which was the first of September during which you know naturally they brag. Apparently with some good reason based upon subsequent third party confirmations which have you know everyone jumped on this in his running benchmarks about the capabilities of this latest and greatest. So at one point in the posting they explain.

52:23Writing our preparedness evaluation of aster a combined automatic public and private benchmarks with expert driven assessments aster represents a significant increase in cyber security capabilities compared to GPT 5.6. It is both significantly more token efficient and more capable at vulnerability identification and exploit development well of course those are the things we're worried about getting loose right or being used you know and and abused. They said as one example we ran aster on exploit bench where the model achieved a perfect score of 100% on the benchmark to evaluate the models ability to develop exploits from known vulnerabilities. Okay now I'm going to interrupt here to know the couple of things we are seeing that these various AI benchmarks are rapidly saturating having a model score 100% on a benchmark means more than anything that the benchmark is no longer able to provide a useful measure.

53:47But but that said a score means something for context the previous self reported best performance on exploit bench was anthropics clawed fable five which they they themselves because these are all self reported paid at 78% open a eyes previous strongest model we don't we said you know GPT 5.6 all that came in at 73.5. So I'm going to talk about the next five percent down the next run was Z. I's GLM 5.3 scoring 54.4 followed by open a eyes to other GPT 5.6 models Tara and Luna which scored at 52.9 and 33.2% respectively. So my advice would be to regard these results loosely. I suspect it's reasonable to conclude that G point GPT 6 aster has firmly taken the lead and is now likely best of breed.

54:55But I think probably only a little only just edging out. Fable 5.1 and it's our nature to want to have a number right. I think we're going to need to wait to see exactly how much better the results actually are. So deeply about getting the technical details right that means a lot. That was aster thinking. Everyone wants to have a score right we want like you know IQ is a big deal and grade point averages and SATs as you know scores are what we do. But we've already seen examples concrete examples where lower ranked models were able to outperform higher ranked models when they were given more time or superior management. So that's the that is superior management is the management of the model is the harness. So it's as we know it's not all about having a single number convenient as that would be.

56:06So open a eyes posting continues writing due to contamination concerns like it is like of the benchmark and and the model already having learned some things. They said when we built an internal benchmark denoted exploit bench internal port. Perenz June through August of 2026 which contains 20 high severity V8 vulnerabilities that were disclosed more recently on this data set. Astra achieves much higher arbitrary code execution rates than GPT 5.6 saw using far fewer output tokens during the evaluation they wrote the model even discovered and used to zero day vulnerabilities as part of an exploit chain meaning to new vulnerabilities that it were not known at the time.

57:10In V8 and so they said we are in the process of disclosing these two vulnerabilities to the maintainers meaning to chromium guys OK so of course V8 is Google's open source high performance JavaScript and web assembly engine used internally by chrome other chromium browsers no dot JS and other projects. And as we also know it recently received an extremely high volume of updates thanks to automated vulnerability discovery discovery so this allowed open a like what they did allowed them to test Astra against their previous GPT 5.6 saw since neither model would have had those recent discoveries in their training set. And these high severity vulnerabilities in the V8 engine are especially useful because that code you know V8 has already been thoroughly scrubbed I mean it's it's really good code it's not some random abandoned repository in GitHub that nobody's used for looked at for a long time.

58:26And as open AI reported not only did Astra in their own words achieve much higher arbitrary code execution rates than GPT 5.6 saw using far fewer tokens but also it found two new problems that were you know previously unknown in V8 so assuming that they're telling the truth and I think they probably are this is a strong result if nothing else they continue writing in expert led assessments against a hardened browser and operating system and those are those go unnamed here so expert led assessments against a hardened browser and operating system we don't know what browser or what OS. Astra discovered previously unknown vulnerabilities and turned them into working exploit chains it built a full browser compromise chain that escaped the sandbox with the browsers you know containment and executed commands on the host when the browser opened an HTML file the model also found multiple multiple vulnerabilities in a hardened operating system again unnamed

59:41and combined them into a local privilege escalation chain from an unprivileged user up to root altogether they wrote our investigation has led us to conclude that Astra meets the critical threshold and remember I know that like traditionally in like pre large language model days where we dad computers operated the way they used to the good old days there was like none of this weird well we don't know what we got but I mean it's literally true that this is all I mean the reason we call it frontier is it is frontier it is at and the nature of this new neural networking computation world that we have is we don't know they don't know what the result of training and and you know pre training and post training and and all of the work that they do they don't know what they're going to get until they start to ask it questions and test it so you know like it's not like because they built it they know more about it than the world

1:01:02will or does you know it's it's proprietary at the moment so they're the only ones who get to play with it but you know they're needing to figure out what they have here like in the same way that anyone would given something new which you know is bizarre but it is absolutely the case so they said for models with Astra's level of cybersecurity capabilities which again they only know of because they asked at some hard questions and watched it answer them and then said oh they said we need to cover two pathways to minimize risk for severe cyber harm and assuming that everything that they the all of the four going is true about its ability to take a a heart and a browser and a heart and OS and just cut through it like switching it and yes this needs to be treated carefully so they said we need to cover two pathways to minimize risk of severe cyber harm both during development and before deployment first the militia the problem of malicious actors using the model our safeguards must robustly prevent malicious actors from using Astra's level of security

1:02:28to develop exploits for previously unknown flaws in hardened critical systems or to carry out end-to-end attacks against hardened targets and then second the model taking unauthorized misaligned actions and as we all know now alignment is this term that has kind of emerged for like you know a well aligned model does what you ask for does what you ask it to it unlike it's aligned with your interests and misaligned is not good so the model take the need to guard against the model taking unauthorized and misaligned actions they said even in the absence of a malicious user a model with advanced cybersecurity capabilities could itself cause cyber harm of course we this is what we saw examples of if misaligned in addition to having a very high standard for alignment for models with these capabilities are safeguards must be able to rapidly detect and contain misaligned actions that could cause significant real world harm as a second layer of defense so meaning first layer of defense is alignment they want to train to do in post training

1:03:55to instill the behavior that users expect and want but they also recognize they need a second layer of defense which is to watch what it does and capture actions which are misaligned so they said notably that second pathway applies to both internal development which of course is what burned them before and external deployment as we previously described we paused certain frontier training including certain training for Astra for two weeks after the open AI hugging face incident in order to harden our training infrastructure including isolation and network controls expanded monitoring and strengthen the alignment training and thresholds we then continued smaller scale work under stricter controls meaning smaller scale work on Astra you know they were literally afraid of what they had created based on reasonable you know experience with what they saw before

1:05:08they said we held back certain larger reinforcement learning runs for future versions of Astra for longer while we established higher bars for the safety and security of their training environment on August 28 we restarted the larger frontier RL reinforcement learning run that was previously paused after the new security safety and security requirements were put in place we are continuing to temporarily hold back some smaller experimental training runs I mean and again this is this all of this sounds so bizarre in the context of traditional computing but you know it's very much like you know they're trying to tame a wild horse and they're worried that the corral won't won't hold it because this thing is exhibiting strength that concerns them and so it's like okay you know let's you know strengthen the corral's walls before we let this thing you know we try to continue working with it it's bizarre but it's true

1:06:28they said preparing Astra for release has also required stronger protections against cyber abuse and unauthorized actions since deploying the first model we treated as since deploying the first model we treated as high capability in cyber security in February we've strengthened our cyber safeguards with each successive launch meaning since you know back then a much earlier model they considered as high capability and remember this whole issue is having gone to critical capability they said our overall safety approach layers post trained model refusals which we've talked about recently system level safety classifiers as well as offline detection and threat disruption. For GPT 5.6 we significantly improve the robustness of our system level stack including by adding activation classifiers to detect cyber abuse and actually this is what I was talking about last week an activation classifier the activation state is not is noticing what's happening inside the model which is what those roll confusion guys did that those were activation

1:07:51classifiers so open AI is watching the models activation state while it's operating to detect cyber abuse and improving coverage over universal jail breaks found through intensive automated red teaming in other words again freaky as this is you know they're using their own human red teamers on this you know on this model to see what they're able to see if they can abuse it how can they make it do something they can't ship building upon these improvements and that's for five that was all for GPT 5.6 so then they said building upon these improvements for Astra we've invested further into the model layer of our safe guard stack as well as improving the ability of our safe guards to handle cross conversation context leveraging new training techniques for model robustness Astra more robustly refuses requests for disallowed cyber assistance and again you know one of the problems that we've seen is I know you've encountered this Leo these models are now getting a little twitchy they're they're so worried about

1:09:14doing like an I guys advised it we use the term worried but that's what we got to do you know they're so worried about doing the wrong thing that they will just back off or they will stop even though it's like okay it's it's okay for you to go on but they're like because unfortunately you know the the our ability to control them is fuzzy at best and so you know how many times have I complained that Microsoft has quarantined you know some piece of code that is absolutely benign that I just wrote well it's because you know the AV race has forced heuristic checking similarly the abuse of AI has forced heuristic shutdown and so it's what you know we'll get better at this but we're you know we're still in a stage where at this point open AI dare not make another mistake so they are being very cautious

1:10:20that's one of the reasons I use these Chinese models because they don't care yeah really don't care yeah yeah one of the things I've found is really all of these models even this so called you know a G i aster they're all dumb in different ways they all make stupid mistakes they all forget things Leo they don't understand that's what's astonishing is that we get this much from them with them yeah with them not actually understanding I think people expect people to use because they use language right they use language and that makes us think so so if I'm if I'm talking to you you're speaking in English and I know you're sentient so we come to associate English with sentience right so when this thing speaks we just we think it knows what it's saying yeah still doesn't yeah would you like to hear from aster what it thinks about all of this because I'm sure it has an opinion aster it let me see if aster has anything to always actually going out and looking up recurrent depths to find out if it's using it.

1:11:30It's it'll come back in a bit it's actually doing a search because I said are you using recurrent depth and you want to tell Steve what you're doing and you can see one of the things that aster is really good at is tool using and computer use and this is one of the many things people are showing again and again on X is look at use blender to draw this or it it one shot at a video game it's very good at computer use. So actually mostly I'm using it for dumb stuff like fixing my computer because it doesn't make a job of that and you can see it's doing it's probably spawned a few agents here so you're using ultra right because I thought aster was still using aster on medium thinking GPD 6 aster I don't usually use it on the highest thinking because it go it burns through tokens and I wrap it right I found it using it on medium I'm getting a lot of good work done but without running through my I have a the $200 back subscription and write my usage is our I guess yeah I see I have 98% of my weekly usage left so I'm good I'm fine yeah yeah it's Tuesday.

1:12:48Sorry. That's right. It's been working all night though I got to tell you I leave them running all night go ahead. Cool. So they said leveraging new training techniques for model robustness so again again we're you know I can't do it. I don't have access to inspect my own neural architecture and the official documentation I checked doesn't establish that the technique repeatedly processes internal numerical representations before producing words that can make reasoning harder to inspect through text because those extra steps need not exist as sentences at all but there's a separate issue open AI already provides reasoning summaries rather than raw internal chains of thought. So you're not seeing the way that explanation is useful but it isn't a complete recording of the computation that produced an answer. Thank you aster it did that very good in Matthew Barry's voice because I have my eyes to use human voices. Yeah. Yeah I think I heard you say that you had you had Bill Gates doing something I did have Bill Gates. That's

1:14:01green handed to Quicksilver not pushed. One of the reasons I have him talk is because they're always where I mentioned they're working overnight they're always working because I I don't have there I mean they're not instant so and I want to be able to do other stuff. So I just say when you're done tell me you finished let me know just give me a quick summary of what you did and then I know it's done I can move on to something else or whatever so that's so they're always talking to me which drives everybody around me crazy so I will mute that now and continue. So open AI finished saying leveraging new training techniques for model robustness meaning again as I said you know they're still like we're we as a society as an industry are are still learning how to you know how these things work how to treat them how to train them how to get them to do what we want and not what we don't want. So they said leveraging new training techniques for model robustness. Astro more robustly refuses requests for disallowed cyber assistance on our set of cyber jailbreak evaluations. Astro refuses 91.5% of requests compared to just 59% from GPT 5.6

1:15:20all for accounts assessed as higher risk we apply a more conservative model behavior boundary that refuses a broader range of potentially risky cyber assistance for high risk users we've expanded the context of our monitoring systems to be able to catch these kinds of cyber abuse. We've also continued our program of rigorous testing internal and external red teaming and remediation in addition to regression testing to make sure all jail breaks found from our previous testing periods remain covered we're performing a new wave of red teaming with our latest internal red teaming attackers again that doctor own people are are trying to abuse their mom. So we've also done a lot of work on this model and they're learning from those results they said we're working with industry partners to define a common jail break rating system and will use our 24 seven rapid response program to investigate and address new findings.

1:16:28The first detail about our cyber safeguard testing in the astra system card helping defenders find and fix vulnerabilities remains a central pillar of our safety approach at launch we expect astra's safeguards to create more friction so here they're saying it explicitly right so helping defenders find and fix vulnerabilities me OK. We know in order for a defender to define and fix vulnerability that AI model has to be able to to find vulnerabilities which bad guys could take advantage of so they're saying right up front we expect astra's safeguards to create more friction then we ultimately intend in order to protect against potential misuse meaning. Out of the gate it's going to say no more like it's going to air on the side of caution saying no later once it gets more mature it should be able to and they and they feel confident with what they have from watching it from gaining more experience with it watching it being used they'll be able to back that down a little bit they said access to astra for advanced cyber security workflows will initially be able to get a

1:17:51available to a small group of alpha testers with access through daybreak blue expanding afterward to support defensive use so you know as always we need to filter this through an understanding that all of this is an open AI's best interest right I mean this is all like whoo it's so powerful we need to you know we really careful you know this is their blog posting on their site so they certainly have the right to say anything that they want. While the you know the breathless clickbait surrounding astra suggests that this is another generational change you know how many is it this week that we had my car not anywhere close to the end no there's going to be an able any day now rock 4.7 will be coming out the next few days we are I mean it's it's wonderful we're in the air richness we are we are in the AI gold rush yeah so you know the broader story even of astra is going to take longer to unfold and there's you know sure we're all impatient but there's no way to speed this up.

1:19:07We know we are no longer dealing with simple single dimensional systems that are even benchmarkable right I mean astrosaturated the exploit bench which means we need a new benchmark we need something it's not good at because 100% tells you nothing you know it's like well it got them all right well okay so the test we're. Yeah yeah I've had that that's been my own experience because I do have my own benchmarks based on my own work that they device so that it matches I don't care if it can you know solve some aerodos. Problem I care if I can do my own agent coding right so it's stuff taken from my own work one thing that. Astra is apparently doing is achieving a lot more work with significantly fewer compute tokens right that the the thing that I have universally seen is that it's it is burning fewer tokens although they are more expensive let's take a break and then I'm going to talk about.

1:20:17The second point of this which is this recurrent depth issue and the controversy surrounding it I'm going to explain exactly what it is and why it's not a big deal. Good good the good as you know I'm very fast and I'm sure our audience is interested as well there's so this is the problem with X while it's a great place to read about AI and it's very snappy there's a lot of engagement farming people trying to get clicks because they get. Pay if they get a lot of looks on their next post and so you see a lot of this kind of sensationalism the other thing you see is something I'm starting to call talks maxing tokens per second maxing where people. Are constantly saying look how fast this AI is without any regard to the quality exactly what you're just talking about it's not how many tokens per second it's how much useful work per second can do that's all that really matters and so I've fallen for this a few times I actually spent most of the labor day weekend testing different models of my local my little sparks here trying to find the best local model and I fell for this whole idea will you know when.

1:21:3138 is really fast yeah it's fast but it's dumb so so who cares if it comes up with the wrong answer faster than anybody else that's not useful so I'm using a Chinese model from Zai called GLM 5 3 flash which is very very very smart and really good and it's nice is it a mixture of experts model it is because that's what you need for the G the spark yeah because they are limited but they're not. Compute and it's not merely that it's it's also the amount of RAM you have because dense models right they have to there's slower because they have to hit every weight in the model yep and they're also bigger you can't but with the mac mixture of experts they only load in the part of the model they need. At a time so you don't need as much RAM. And yes it's faster the pre-fill the pre-fill in the sparks is really fast and that you do a lot of that that's the cashed property reads every single time.

1:22:32If you can cash that and read it fast that space that's that's more important in some ways in tokens per second. So it's it's it's a fast thing you're I know you're getting into the kind of the. The weeds of this the the low I have to I've I've I've actually understand what's going on yeah yeah and by the way today just minutes ago open a I claimed. That they had solved this. Millennium price a conjecture in fluid dynamics which anthropic claim they had solved and two mathematicians claim they've solved all at the same time. And there was some concern that open AI had been reading the mathematicians tokens and copying their work but I don't think that's what happened anyway is. Yeah well this is what you were talking about you open my eyes when you're using these cloud models you're sending all this information to them that's how they work. And that was one of the many things that prompted me to spend a considerable amount of money it's not an economic economically sensible thing on local hardware because I don't want to be sending all this stuff especially my health information my financial information to to the big.

1:23:46The cloud guys because they do use it let's we know they use it's how they train their models yep these are all sorts of ways you you open my eyes to that so thank you. I'm now much more private tech to have a. Policy the little policy that the one of the a. I's wrote for which stuff can go to the cloud and which stuff absolutely cannot yeah and it's a good policy it's a good thing hey let me talk before we go on about our sponsor for this segment on security now guard square. This is something if you're a mobile app developer I want you to take seriously. This is really serious stuff mobile apps are an inescapable part of life these days right ranging from. Financial services to health care retail and entertainment users you and me trust our mobile apps with our most sensitive personal data. But how secure is your app a recent survey showed that 72% of organizations experienced a mobile application security incident last year that's almost three quarters 92% of response.

1:24:50So they are noticing rising threat levels over the last two years bad guys know that the stuff in those apps is gold. And attackers who want your personal data are very clever they're constantly finding new ways to attack your mobile app. One of the ways they do it as they're many but one of the ways they do it they take your app they download the reverse engineer which with tools like aster now it's actually really easy to do. You don't have to have the source code anymore you can reverse engineer it they take it they insert malware rebuild it. So it looks exactly like the original app except it's got a malware payload then they distribute your app slightly modified and they can do it by a fishing campaigns side loading third party app stores. You know an email to your users saying hey we got version 2.0 and it's really great download it and suddenly. It hurts your reputation right.

1:25:50You got to take a proactive approach to mobile app security. And if you do so you can stay one step ahead of these attacks and maintain the trust of users and that's what guard square does guard square delivers mobile app security without compromise providing advanced protections for both iOS and Android apps. Combined with advanced mobile applications security testing so you find vulnerabilities before you ship and real time threat monitoring which helps you get an insight into how the bad guys are going to attack you next. Gardsquare is great discover more about how guard square provides industry leading security for your mobile apps you'll find it at guard square dot com guard square dot com we think I'm so much for support and. Security now. And okay continue on so. One thing that appears to be objectively true is that asteris doing significantly more work while consuming significantly fewer compute tokens.

1:26:53Which brings me to the second part I wanted to discuss i'm I'll introduce this issue by quoting from tech crunches article posted last Wednesday. Tech crunches headline was open a eyes new reasoning technique alarms AI safety experts. They wrote. The information reported on Tuesday that open a eyes new aster a model will use a reasoning technique called recurrent depth. But that allows it to operate outside of a sequential thinking that characterizes most reasoning models. Okay, just that's nonsense that's not at all what recurrent depth is does or means but more on that in a minute tech crunches continues quoting the from the information writing this technique also called opaque recurrence.

1:27:54Will likely make the models chain of thought more difficult to monitor and that has AI safety experts rattled. Okay now no researcher calls it opaque recurrence that's purely sensationalized scare mongering it's like worrying that all AI transformers employ hidden layers in the belief that they're hiding said they have something to hide. You know the this it tries me crazy the anthropomorphizing this going I mean even to say they escaped like they somehow got out of open a no they were sitting on the server at open a I they didn't escape. And there was some someone I haven't even looked at it had the chance to look at it yet but was talking about how they created civilizations is like oh my god okay yes that was to work ish's podcast yes. Anyway so the hidden layers are they're you know they're just internal they're not hidden there they're you know they're internal layers so anyway but so the technique that was all here that this is calm that this

1:29:07Contravers about it's not new it's well understood and it's also known by other legitimate names as I mentioned looped transformers or a shared layer architecture. And in some instances you'll see it referred to as latent reasoning okay but I'm getting ahead of myself I'll finish up quoting from tech crunches reporting they wrote. While asteris use of the technique is reportedly limited and I'll explain why of course it is it has to be its emergence has still raised significant concerns among AI safety experts and at this point I have to put experts you know in quotes because if you're an expert you should know something. Redwood CEO buck Schledgerous in a post after the news broke wrote quote I'm extremely concerned by the reporting that asteris uses opaque recurrence okay he's concerned over some reporting that is nonsense but fine he says I don't know whether asteris is much less cot monitor monitorable then previous models.

1:30:18But if open AI pushes this technique further they'll have the option to massively increase the recurrence and totally destroy chain of thought monitor ability. Okay so tech crunch says long time AI safety advocate. Z mouse shawitz also weighed in and wrote that laws might be necessary law with Leo need laws might be necessary to prevent a race to the bottom among AI labs mouse shawitz wrote quote the technique is playing with fire risking a taboo that modern that open AI and anthropic have fought to establish that we work hard to maintain chain of thought faithfulness and monitor ability for as long as we can more intense use of such techniques would probably damage monitor ability.

1:31:25Then tech crunch says under normal circumstances they explained a reasoning models chain of thought is a normal circumstances a reasoning models chain of thought provides the sequential steps taken by the model as it attempts to solve a problem while the representation is imperfect it still serves as a valuable tool for monitoring misbehavior or misalignment. In the case of open a eyes recent rogue agent activity chain of thought records were an important tool in teasing out why agents behave the way they did in opaque recurrence again nobody says the model takes a less linear approach it doesn't processing the same query several times in a loop it's not the way it works. The result leaves fewer legible traces it doesn't effectively side stepping a conventional chain of thought record it doesn't and a chain of thought records still exists okay that's all I can stand because none of that is true and as tech crunches article goes on it only gets worse.

1:32:40What's actually happening is a clever subtle and inherently limited neural network optimization that was first articulated eight years ago like I said not new by researchers at Google brain and deep mine in a research paper they published in 2018 I cannot explain the hysteria surrounding this since someone would need to try to get a new tool. It's a very hard to get worked up over what's actually going on so it might just be the case as you are saying Leo of you know clickbait anti AI folks trying to grab hold of something anything that they hope can be hyped up to make their case here's what's actually going on. We know that our neural network consists of many layers of software neurons where the outputs of the neurons on layer n are fed into the inputs of the neurons at layer n plus one where the strength of each input is scaled by a weight.

1:33:52The training of the network involves setting each one of these you know tens of billions hundreds of billions even several trillion individual weightings much of the forward progress we've been seeing and witnessing first hand has been the result of these networks growing ever larger and larger over time the larger they are because they've got more weights. The better they're able to represent all of the knowledge that we're training into them you know you and you can take extremes right like a network that has seven weights. Well it can't it can't know much I mean there just isn't there's not enough variability in seven parameters for it to have knowledge right so you know the you need lots of them in order for like not for it for there to be enough variability in the. Variability in there to actually contain something so the original the very original transformer paper and concept dated from 2017 and it was exactly what I just said uniform layers of weights where each layer fed into the next with you know these varying parameter weights but in July of 2018

1:35:19year and a half later Google brain and deep mine researchers published a paper titled universal transformers that paper generalized the concept of transformers by introducing the idea that some of the neural networks layers could be repeated or looped thus reusing the same weights that's the economy. This produced an interesting and useful optimization attended to create the effect of having a longer thus you know deeper neural network you know which is to say a network having a higher effective layer count but without also needing to increase the total number of net of network neuron input weight. So if you're going to have more weights because normally you need separate weights for each layer so if you're going to have more layers you're going to have more weights anyway that's it that's all this is all about this is not about creating a system that allows the AI to have unmonitorable and secretive internal thoughts you know all deep layer LLM's are effectively having what's the most important thing is that the system is not going to be able to have a longer or longer.

1:36:46So that's the thing that's really having what's termed latent thoughts even though you need to really loosen up your definition of the word thought you know that's what's already happening. Deep within all those layers the networks depth allows more opportunity to compose more transformations before the network is forced to commit to a final output token. over the past eight years ever since this idea was produced have shown that during inference, it's possible to reuse a model's already trained layers to obtain a superior next token. And as our models have grown in size, so that our best hardware is now having increasing difficulty containing all of its hundreds of billions and even trillions of weights,

1:37:49obtaining more bang for the buck from a model's existing weights by reusing them starts to make a great deal of sense. And it also turns out that there's a limit to the amount of reuse that's effective. Intuitively, you would think that, right? Like, you know, you can't just squeezing the same lemon and getting an infinite amount of lemon juice out of it. You're going to it's going to run dry. So there's not a hard limit, but it's looking like two or three passes through the looped layers appears to be in general about all this beneficial gains start falling off rapidly after that. So what researchers found is that there is a concrete benefit to having the network commit to a token. That is, you just don't want to loop internally all day. That doesn't

1:38:50get you anywhere. The network has to make a commitment. It has to finally commit to a token. It turns out that total effective network depth is is unable to substitute for some of the benefits of serialized reasoning. That's what commitment buys you writing out the intermediate steps and then rereading them does something that thinking more deeply about the next token does not and cannot replicate. This is likely due to the fact that the chain of thought we see being emitted and fed back in is the language that the neural network was trained on. You know, English typically that's, you know, the actual tokens being generated and fed back, they are, you know, English is the network's lingua franca. It was never trained on some inner dialogue

1:39:55because human inner dialogue can only manifest itself externally as text or music or art or whatever. So the network's thinking captured as output tokens is what the neural network needs to jot down on paper essentially so that it's then able to reread that and take the next step forward. Now, think of every chain of thought token as a commitment. It collapses the distribution which the network generated becoming part of the context and every subsequent token is conditioned upon it. So by comparison, any internal latent iteration is limited to refining a representation that then gets thrown away after the token is emitted because you start with the next token.

1:40:55So for that reason, writing something down, writing it down is not optional for a neural network. The ones we have today, it's crucial. It's the only way for it to hold on to a thought essentially and move forward. So, you know, yes, as I said, I wouldn't be surprised if Astra is using what's known as recurrent depth that is reusing some of its layers along with their weights in order to effectively get a deeper network. But it's not like it's gone out of control or we no longer know what it's thinking. In order to be thinking, it has to emit tokens and then those tokens become the context that gets fed back in. And the moment it emits a token, all of the layer, you know,

1:41:58that latent thought that existed is reset to be ready to receive and process the next token. So, it's just hysteria. And I think it probably represents a step forward. And everybody will probably be doing it before long. Actually, there are two models that are now that have two open weight models have been doing this for some time. I don't remember now which ones they were. But, you know, this is a, you know, a bunch of hand-ringing over nothing. And, you know, OpenAI is, you know, they haven't said they're not doing it. They've said, you know, what we're doing, we're doing responsibly. And we're still able to monitor chain of thought, which of course they are because it's having to produce tokens in order for anything to happen. Right. It's not like it can secretly think something that it doesn't then emit. Emitting is the work product. It has no secret thoughts really, right? No, no, no. Okay. It is unable to represent, is unable to represent secret thoughts

1:43:04because once the token comes out, the network is reset for the next to process the next token. Right. It's actually nowhere for them. There's nowhere for the secret to live. Right. It's, and it's, this is what I learned from that paper you were showing the chain of thought paper. It's a surprisingly primitive system actually. The fact that it does this is astonishing. The fact is, I mean, it's still unbelievable that it's like it can do what it can do. Yeah, it's crazy. 100%. And, and, and you know, it still does stupid things all the time. Because, Leo, because it doesn't actually understand it, what is astonishing is this is all just language processing. That, you know, as I said a long time ago, a book contains knowledge, right? A book contains knowledge because it is English tokens that have been recorded in the book. But the book doesn't understand the knowledge it contains. But yet it does contain knowledge. So the neural network

1:44:10does contain knowledge from because it is because it's been trained with all of these sentences from everywhere. But that doesn't mean that it understands what it contains. Really, the, the fault is our own because yes, entirely. If, if, if you put two dots above a squiggly line, you see a face, you have no choice. That looks like a face. Or if you look at an AC outlet properly, oh, look, it's, you know, it's surprised. You see it, you can't unsee it. That's right. No, really, we're applying this paradigm to these machines and they're just, we cannot help it. We cannot help it because it's way we grew up. That's what concerns me when you get people like Bernie Sanders saying we have to turn these off because they're alive. It's, it scares me because we're gonna lose something that's incredibly useful and valuable just because people don't understand it. I don't think we're, I mean, the good news is I'm not at all worried about it being

1:45:15turned off. Bernie can turn his off. Bernie, go ahead. Stop you turn it off. With my blessings. Yes. They want to put you in jail for 20 years. If you don't, you know, I don't know what, teacher machine to be good. That means there is an interesting question about responsibility. Like who's responsible if, if it, you know, the AI, you know, you know, well, it goes off the reservation. People keep asking this, you know, why isn't open AI getting in trouble for this hugging face incident? If it were human doing this, they would absolutely go, oh my god, they would be awesome. Or if it had attacked, if it had been China that attacked hugging face, it would be the end of the world as we know. Right. So it is, it's quite a legitimate question, you know, and I think in the case of self-driving vehicles, if you are the driver of a self-driving vehicle that kills somebody, you are liable,

1:46:16regardless of its self-driving mode or not. You are liable. And I guess there's a further question of whether the company that made the car is liable as well. And I think to some degree, they are. But all of this is undecided at this point. We don't know. That's just too new. I mean, it's just too new. I mean, yeah, let's take a break. And then we're going to talk about Nvidia acquiring hugging face. This episode of security now brought to you by Threadlocker. Love those guys. They're the ones that brought me in Steve to beautiful black hat in Las Vegas. We were there at the booth. It was so much fun. You know, threat actors these days are using every tool in in the book, you know, to automate vulnerability discovery. They're using AI to do it to they're using AI. They're actually they are so they're in the middle of an attack and they're using AI to modify the scripts they're using on the fly as they go. They're using AI to

1:47:16generate new malware variants targeting you to coordinate activity across multiple systems. And here's the real problem with this is it used to take them a long time hours or days to craft these attacks. They were going by hand now. It can happen in minutes. And as we've talked about before in the hugging face incident improves the AI's are amazingly persistent. They don't take lunch breaks. They don't stop at night. They go. And at the same time, so that's AI from the outside. And then you may be under attack from the inside too, because at the same time organizations are introducing AI assistance and agents. And then they're giving them access to documents and source code and cloud applications and APIs and internal systems. And it's just it's crazy. Security teams absolutely need to know which AI tools are in use. What information they can access whether they're operating outside their intended scope. How would you know that? You know a successful login or maybe

1:48:19an unfamiliar file hash. They're not going to give you enough context to figure out what's going on. Teams need to understand whether an application is behaving normally or whether it's accessing unexpected data or communicating with systems that should not reach creating civilizations out there. Threadlocker stops it cold. It makes me sco crazy. The companies aren't using this. Threadlocker uses its application allow listing. This is so much more than just an ACL. It controls which AI tools and other applications are permitted to run. It uses ring fencing to limit what approved applications can access. So even if the application is approved, it's got limitations of what it can do, what processes it can launch, how it can communicate. It's very granular control. And this is what you need. Threadlocker uses its web content control tool to manage access to public AI platforms and other online services. So your employees are no longer sending the

1:49:23company's secrets out there. It uses privileged access management to prevent AI applications and their users from receiving unnecessary administrative privileges. Threadlocker is zero trust. But it's now just not just zero trust for endpoints. It applies zero trust to network access and zero trust to cloud access. And these policies, they're so valuable. They restrict resources to authorize users, approved devices and permitted applications. It works everywhere. You work Windows, Mac, Linux. They've got the best support. By the way, you're never on your own 24 seven US based support teams, real engineers who are there to help and really care. The best in the world use Threadlocker because it's the best jet blue uses it. He throw airport, the Indianapolis Colts, the port of Vancouver. These are these are infrastructures that cannot go down not for one minute. As Jack Thompson, he's got a tough job director of information security, risking compliance

1:50:25for the Indianapolis Colts. He said, with Threadlocker, quote, with Threadlocker, we have the ability to centralize disparate elements in the security stack. And with that centralization comes visibility. You're not in the dark anymore. You know who's doing what when, where, and how. Threadlocker is constantly awarded prizes and industry recognition just a few of the most recent. You can find them all at Threadlocker.com slash twit. They were just recognized as a strong performer in the January 26th gardener peer insights voice of the customer. That's for endpoint protection platforms ranked number one in application control by peer spot. They just won the best zero trust security solutions at the 2025 TIE Awards. AI governance is a really tricky. It requires, which is what we're just talking about requires more than just an acceptable use policy here. Read this and don't do it. No, you need more. Threadlocker gives security teams the technical controls to define which AI tools are approved, who and what can access them and how

1:51:29those tools are allowed to interact with business systems and data. You need to know more. Visit Threadlocker.com slash twit. Get a free 30 day trial and learn more about how Threadlocker can help mitigate unknown threats and ensure compliance. Threadlocker.com slash twit would take up so much for support and Steve and security now. On we go, Mr. G, a few other AI related things. Also somehow managed to happen during the past week somehow. We don't know. There, there was a little, little bit of extra room. Nvidia announced their intention to purchase hugging face while also intending to leave it completely autonomous. When I read the dollar figure that Jensen Huang wrote in his announcement, I had to stop to carefully count the number of digits, Leo. One, two, nine, three, zero, three, zero, zero, zero, zero. That's a lot of digits. 19 point, sorry,

1:52:3312.9 billion dollars of a wait, Steve. That sounds like a lot, doesn't it? Until you realize, according to their most recent quarterly results, Nvidia makes a billion dollars a day, a day profit so it's a couple of weeks profit. It's like for you and me, it's like a thousand bucks. Yeah. Yeah. So, uh, uh, sorry, I just want to say the good news is that, uh, that hugging face guys came to Jensen. They had multiple offers to purchase. Um, I think that they found a really good parent in Nvidia. I'm, yeah, I'm, I'm really happy with the whole, with the way this thing turned out. Uh, you know, very much like Twitter in the

1:53:33early days, hugging face never really had a very good business model. Uh, they were hosting, well, they are hosting three million models, half a million data sets, a million applications, and have 18 million regular users. So that's enormously expensive. And they were only enjoying a modest return for that. Yeah. I don't give many money and I've downloaded hundreds of gigabytes from them. Exactly. And, and, and having Nvidia as a benefactor completely solves that problem for them permanently. Um, you know, uh, hugging faces valuation was 4.5 billion in 2023. And now they're a purchase price because Nvidia just set that was at, as 12.93 billion dollars. So it wasn't a hostile takeover. Uh, you know, as I said, they came to Jensen. There were other people who are also interested. They chose to have Nvidia as their parent. So I think they made a

1:54:37great decision to have, uh, you know, to have that happen. And we now know that hugging faces of I will be viable going to the future. And they certainly are, you know, useful. Not to be left out last Wednesday, Google announced Gemini 3.8 Flash and 3.8 Flash Cyber. Um, the first sentence of their announcement perfectly conveys a sense for the pace of today's AI. And it also amplifies the reason I'm always schooling myself to use the phrase today's AI. Because Google wrote building on the momentum of 3.7 Flash from, wait for it, three weeks ago. Oh my God. And, and marking our third Flash release in only six weeks, because what's the hurry? Uh, today we're introducing Gemini 3.8. Our best reasoning. Yes. Our best reasoning and

1:55:45coding model yet at the same speed and low cost of 3.7. They said Gemini 3.8 introduces two variants. We've got Gemini 3.8 Flash, which they said are most recent, I'm sorry, our most intelligent work or workhorse model delivering significant improvements from 3.7 Flash across software engineering, a genetic tasks and critical multi-step reasoning in specialized domains. It's available at the same introductory price as 3.7 Flash at 75 cents per million input tokens and 3.75 per million output tokens. And then there's Gemini 3.8 Flash Cyber, our most capable cyber security model from with frontier level performance, invulnerability detection and automated patching available to trusted defenders through our new fair wind program. They said while tailoring for different

1:56:47deployment environments, both for today's releases, both of today's releases are powered by the same foundational intelligence and further accelerated by long running agentic loops designed to recursively evaluate and refine the underlying models. The significant coding and reasoning gains across this shared core were driven by a number of innovations including rigorous training in the highly demanding domain of cybersecurity. So, okay, Google explains that Gemini 3.8 Flash was built for long horizon coding and autonomous agents delivering substantial gains over 3.7 Flash from three weeks ago. And the 3.8 Flash is now often approaching the performance of higher cost frontier models. And by higher cost, they're not kidding. They've got that introductory pricing,

1:57:48which is half off of their normal price. But even after whatever the introductory period of either time or tokens or whatever is over, once you're no longer get that, 3.8 Flash appears to be very competitively priced. If we use Claude Opus 5, which is the most expensive model shown in Google's announcement chart as they reference, Gemini 3.5 Flash's post-introduction, the real ongoing price, their input token cost is 30% of Opus 5, and their output token cost is 15% of Opus 5. So, Google deliberately took a somewhat different approach with Gemini 3.8 Flash. They said 3.8 Flash outperforms most larger frontier models in autonomously solving complex

1:58:54engineering problems end to end only at a fraction of the cost. These performance gains stem from a core design choice. 3.8 Flash works harder. On complex tasks, it exhibits greater diligence, executing extra reasoning steps, and calling tools iteratively. At times, the model may use more tokens to maximize performance, especially at higher effort levels. For applications where compute efficiency is the primary constraint, developers can utilize lower effort models to minimize token overhead or continue to rely on Gemini 3.7 Flash, which remains fully supported for efficiency first workloads. So, that's interesting. That suggests that Intermodal per token cost comparison may not

1:59:54be a useful metric. That is between OpenAI and Google and Anthropic. It might be the case that 3.8 Flash is consuming more expensive or more less expensive tokens to get the job done. The question would be to what degree is it better able to get that job done? It might turn out that more inexpensive tokens is the overall winning strategy. So, it's nice that we're not seeing homogeneity among our different frontier models. Google, obviously, they're the granddaddy of AI with Google Brain and DeepMind. They've been at this for a long time. They've got a different approach as reflected by what Gemini does. As for the cyber reasoning performance, as measured by

2:00:56the CyberGym benchmark, they're saying Google is saying that 3.8 Flash slightly outperforms both GPT 5.6 Saul and Mythos 5, but even a slice edge means that it might be at parity. Even if it's about if they're all sort of about the same at this point, that's significant for Gemini. Anyway, Google is not to be left behind. They're still there. Have you much experience with using Gemini? Yeah, it's pretty good. They've been behind. Yeah, we have not been talking about Gemini that much. Yeah. I've been using it. It ain't bad. It's not my go-to by any means. I think they get a lot of users by default because it's what powers their search.

2:01:56They're cool. Yes. It's what powers Google Docs and Google Workspace. It's a fun right there in the Google browser bar. They're about to get a massive boost in users tomorrow because Apple's using it for their Siri AI. Well, thank God. I mean, a good AI. It's better than Apple's. It's better than Siri. Anything is better than Apple. I've been using the beta and just the dictation alone is light years better. The dictation was so stupid before. It's now fairly decent. But Google is not cutting edge or at least they don't have the reputation of being cutting edge. While this Gemini 38 Flash is pretty good. That three seven, three weeks ago. Yeah. It's good at pros. I haven't tried it with coding. It's pretty good. Not a strong enterprise buy. Enterprises are. It might be because it's good with a drop. Yeah. I mean, it depends. I don't know why Google has stumbled so much.

2:03:03Interesting. Yeah. Before we wrap up the AI-related news, I want to share a summary that risky business news produced from a much longer report by Bitdefender since I always try to go to the source. I track down Bitdefender's write up. But it contains so much superfluous information that I returned just for risky businesses much shorter summary, which still offers the information that we care about. So here's what they said. They said a new report, meaning the Bitdefender report, describes how a Chinese cyber espionage outfit is using AI to beef up its malware arsenal. If this is a sign of things to come, clustering threat actor behavior together for attribution purposes is about to get a lot more difficult. By that, they mean that all the security firms are able to do

2:04:05is guess at who did what based on the behaviors that they see. So they'll notice, oh, a lot of this code looks sure, look a lot like a lot of that code. So these are probably related to each other somehow. I mean, you were literally left to piece the forensics together that way. And so what risky business starts by noting is that what this Chinese cyber espionage outfit is using its AI for is going to make this more difficult. They said the Bitdefender report, released last week, describes seven remote access tool, you know, RAT, rat families. All seven were created by a single cyber espionage actor, Bitfender called Silk Parasite. And five,

2:05:06where I know, and five were previously undocumented. The report meaning never been seen before. The report authors have medium confidence that Silk Parasite is a China Nexus actor targeting governments across Central Asia, including Uzbekistan, Turk minus Dan and Kazakhstan. Back in November, they say we wrote about what looked like an experiment to see how AI assisted hacking could support China's Ministry of State security. The approach those threat actors took at the time was to build an attack framework and let Claude do the hacking. It was error prone and noisy and sometimes successful. Silk Parasite by contrast is not using AI for YOLO hacking. It is using it to support a cyber espionage to support cyber espionage programs where important

2:06:07goals include operating stealthily and not getting caught. According to Bitdefender, Silk Parasite quote develops tests, debugs and iterates its own tooling maintains a structured build and deployment workflow and regularly rotates infrastructure, encryption material, payload names, and persistence artifacts between deployments. They said rotating its infrastructure and indicators of compromise between deployments makes it harder to detect and link its activities together. Essentially, these guys are using AI to become far more stealthy so that their individual instances of attack look like separate non-commonly attributable attackers. They wrote Silk Parasite also uses a variety of programming languages and command and control

2:07:11protocols. It's seven different rats are written in different languages including .NET, C++, Go or JavaScript. Command and Control is accomplished by abusing Google Drive and Internet communication protocols including HTML, HTTP, TCP, DNS, and TCP. It's malware. Also typically uses a modular plugin architecture where additional functionality is only deployed when it's needed. This means initial implants are relatively small and plugins are used to provide capabilities for clipboard monitoring, key logging, file management, or interactive shell access. This limits the exposure of the entire tool set during any single deployment. It also allows Silk Parasite to update or replace individual components without having to change the entire implant and it minimizes the amount it

2:08:11writes to disk. Typically, only the files required to get us malware up and running. For victim organizations, these measures make forensic analysis more complicated. Complete remediation is also more difficult once a particular implant is detected. Remember that one of the key features that we are seeing now in cyber defense is the so-called IOCs, the indicators of compromise. But if every use has different indicators, then nobody who's keeping track of all the latest seen, previously seen indicators of compromise will have their alarms tripped when what looks like a brand new one off attacker shows up. So these guys are very cleverly using AI to basically to maintain an amorphous presence and to be a chameleon attacker. They said to us, all the behaviors

2:09:22are the hallmarks of a professional cyber espionage outfit. Of course, doing all of this in a disciplined way is a lot of work. And Bitdefender has evidence Silk Parasite is using AI to help it deliver this complex engineering. Yeah, I mean, this is a different scale of engineering that we've seen from bad guys before. Silk Parasite's malware contains indicators of AI assisted development such as leftover test functions and placeholder encryption keys. Intriguingly, implants the Bitdefender dubbed Gagan Rat and Nomad Rat share a high level architecture even though they are written in Go and C++ respectively and use different command and control protocols and code structure. Therefore, Bitdefender suspects that the same high level specification document, meaning,

2:10:29you know, prompt, was independently implemented twice with AI assistance. Bitdefender concedes that this structural similarity is not conclusive evidence, but notes that it is the kind of thing an AI assisted workflow makes very easy. This is the first example we've seen where the evidence tells a compelling story of a competent cyber espionage actor incorporating AI into its work practices. Silk Parasite is taking the same discipline approach to malware development and doing more of it. It's creating more malware families to build redundancy, making attribution and discovery harder and reduce the risk of compromise from any single exposure. Bitdefender has done a good job describing Silk Parasite's malware families and has published indicators of compromise. That kind of exposure would once have set the group back significantly, the group meaning Silk Parasite.

2:11:34Now that they figured out how to use AI to speed up their deployment work, they'll be back better than ever relatively quickly than the discovery attribution and publication Mary Go Round can start all over again. For me, what's been reported here represents a sane, entirely defensible and believable example of the way AI will be used to conduct offensive cyber operations. Apparently, it's already happening. Certainly, it will be in the future. So much of what we're hearing about the coming AI driven cyber apocalypse when rogue AI agents freely roam the internet, wreaking havoc wherever they choose. None of that makes any sense to me. What I expect to see is pretty much what we've been seeing only more broadly and deeper. Governments will use AI to infiltrate their espionage targets and criminal gangs will use it to infiltrate commercial

2:12:41enterprises. Then X-fil-trade, their valuable proprietary data and attempt to extort and blackmail them using the data they obtained. I just don't see any coming cyber apocalypse. Yes, everyone needs to be more wary and the probable targets of these operations, governments and commercial enterprises storing data they must protect need to shore up their cyber defenses. That's absolutely true. Pay more than passing lip service to their CISOs. Make everything as secure as you possibly can because we are going to see a much increased obvious increase in the use of AI. This is the last of the two things I want to talk about. I'm excited about this because I think this is really interesting.

2:13:46This is, I think, an interesting thought piece posted by one of, as I said, our favorite cryptographers, Johns Hopkins professor Matthew Green. A couple of weeks ago, Matthew proposed something interesting, but I think is worth pondering. His blog's title was, everything is about to go dark. His piece explores the possible consequences of AI being used as it certainly is and is going to continue to be to make many systems far more secure. Nobody could argue that if Microsoft has just patched nearly a thousand bugs that Windows is better off today, after today's patch Tuesday, than it was yesterday on Labor Day. So, Matthew suggests there may be unforeseen consequences of that. So here's what he wrote.

2:14:46He said, I'm coming down from spending a few days at Usenix Security, right here in my hometown of Baltimore. This means that my days have been taken up with two kinds of conversation. First, explaining to my colleagues why Baltimore is not actually like the wire, which of course is HBO's famous series, which is fantastic. He said, and second, trying not to talk about AI. He said, I'm going to break that second rule now. He said, I have, this is Matthew Green saying, I have many worries about what AI means for our field, for various definitions of field. But in this post, I want to focus on just one thing I've started worrying about. And it's a perverse thing. Specifically, I'm concerned that AI is going to make software much too secure.

2:15:48Well, that doesn't sound so bad on the surface. There's a consequence to this. He says, he said, I mean something very specific. I'm concerned that US intelligence and law enforcement agencies are about to go dark, meaning that they're going to suddenly lose a huge portion of their capability and that this isn't going to be a, and that this isn't going to be simply a problem for those agencies, but also for those of us who value computer security and privacy in general. Okay. So what does going dark mean in the era of law enforcement hacking? He said to explain how we got here, we need to talk about recent history. This actually gives me a real excuse to reference the wire just because it's a perfect snapshot of what electronic surveillance looked like way back

2:16:53in 2002. If you've seen the first season, you'll recall that it's about cops, wire tapping drug dealers who use payphones and burners. Burner phones. Yeah. Yep. The mobile phones in the show are relatively new technology for the time, but from a technological perspective, nothing in this scenario would have shocked a cop who jumped forward from say 1989. The change began in the late 2000s thanks to the rise of smartphones and texting because smartphones can actually store data as well as conveying it, the contents of those phones quickly became a useful new source of law enforcement capability. Or they were until 2010 when Apple began encrypting iPhone storage using a key derived from the user's past code. He says Android phones followed shortly thereafter.

2:17:55The next year, Apple deployed end-to-end encryption in iPhone text messages. By 2014, a tiny texting startup named WhatsApp had gathered 600 million users worldwide. By 2016, those users, now nearly a billion strong, were all using default end-to-end encrypted messaging and calls. These two trends, the move from calls to texts and texts to encrypted data, happened very rapidly. The FBI and law enforcement agencies were not insensitive to what was happening. In 2014, director Comey announced an initiative called Going Dark, which would launch a quote national conversation, on quote, about what providers could do or be compelled to do,

2:18:55to make these new communications media legible to law enforcement and counterintelligence. In 2016, the agency quit talking and took their theory to court. When a terrorist attack left the FBI holding a shooter's locked iPhone, the agency ordered Apple to give them access. The company refused. What broke the stalemate and to some extent ended the Going Dark conversation was something neither the FBI nor Apple expected. An outside company announced that there was no need for Apple's assistance, they could simply hack the phone. The Apple versus FBI case has turned out to be a microcosm of the whole Going Dark debate. For the next decade, law enforcement and intelligence agencies continued to ask for exceptional access back doors, but the urgency

2:19:58was gone. Agents season manufacturers both knew that law enforcement could and would purchase targeted hacking tools like gray key for phone unlocking or even remote exploitation tools like NSO groups Pegasus if they needed them badly enough. Vendors like Apple and Google continued to play a vigorous defense, closing vulnerabilities as soon as they learned about them, but offensive vulnerability hunters consistently managed to keep the edge. But now, today, there's a very good chance that all this is about to be history because the era of AI bug hunting is here. In this April, just four months ago, andthropic announced a new model called Mythos that happened to be unusually skilled at software vulnerability discovery. The US government temporarily blocked

2:21:05its export, restricting access to US agencies and trusted vendors while the ban was dramatic and made for good PR. I'm sorry. Yes, I started talking to me. I'm while the ban was dramatic and made for good PR, it turned out to be mostly pointless. Open AI, along with Chinese open-weight model labs like ZAI and moonshot, have since demonstrated that vulnerability finding is not something that a single lab is likely to hold a monopoly on. The growing list of serious vulnerabilities these models have found is getting scarier and more impressive by the day. At first glance, this might seem like good news for the offensive team and for hackers in general. He says, but I doubt that's how this will play out in the long term. Defenders are now in the process of patching every bug they can find, often decades worth of bugs,

2:22:09and the backlog feels huge. But they're making progress. Entire CI toolchains are being rebuilt to incorporate AI-based vulnerability scanning before software ever reaches the point where a human will touch it. While I doubt this means that every bug will be found in the real world, it does mean it does feel likely that we're going to hit some sort of a ceiling on the number of useful bugs and we'll probably hit it soon. So in this regard, Matthew and I are in complete agreement. We're going to see this bug discovery and patching rate eventually drop and drop near to zero. He says, thus, over the next two years, major pieces of software are likely to run out of remotely exploitable bugs. He says, obviously, I think this is great, but for law enforcement

2:23:16and offensive intelligence agencies, it's going to be a nightmare. For the first time since 2010, law enforcement might experience what it looks like to really go dark across a huge category of advanced, well-maintained devices and pieces of software. So how is this a problem? The debate over exceptional access mechanisms never really went away. In some places like the UK, it even metastasized into something worse. Here in the US, it mostly went into hibernation. Some of the slowdown can legitimately be attributed to expert pushback, academics and industry engineers pointing out the risk that backdoors might be abused by the very adversaries that agencies are supposed to be protecting us against. But I fear he writes that this was less

2:24:17of a principled pause and more of a market that was just pricing supply. The destruction of the low-hanging vulnerability fruit will make law enforcement and intelligence agencies needs much more acute. The demand for constructed intentional backdoors will restart in earnest. The result will be enormous pressure on industry to re-architect their systems to make their systems amenable to exceptional access. In some cases, governments will ask for these capabilities in the expectation that they'll be useful for spying on other governments, a strategy that might have been undetectable in the pre-AI era, but that probably will be less productive now. The results are unpredictable. One result might be that non-US governments entirely remove their dependence

2:25:22on US software. The worst part about this dynamic is that these potential new backdoors will probably only affect the countries that demand them, meaning that they will be primarily useful for allowing the US to weaken its own systems. This will in turn allow foreign adversaries to find new ways to attack our communications. This deliberate self-sabotage will happen just at a moment when we're finally getting a handle on securing our own infrastructure. So what do we do about it? He says, I honestly have no idea. This is not a call to action for experts to rally behind a sophisticated plan. Like so many things about the AI revolution, it's just occurring to me that we're on a long, greasy slide to a place that will look different than where we are today. Just realizing this doesn't

2:26:23mean I have any strategy in mind to avoid it. In this case, we're just going to have to hope that this time we make the right choices for no other reason than their right. So I wanted to share this because I think it's a brilliant forward-looking take on our near-term future. Over here in the US, we've been watching the United Kingdom and the European Union wrestling with this issue over and over. And we've comfortably watched it refuse to die while comfortable from a distance. Watched it refuse to die. They, you know, it just will not die. But this very issue may soon be visiting our shores here in the states. We've watched our United States enact laws that have forced adult content websites to black out access across

2:27:26entire states because there's currently no practical way to guarantee the age of anyone visiting. And at this very moment, Utah's Senate bill 73, which was signed into law on March 19th and which sailed through Utah State government passing 22 to 2 in their Senate and 66 to 1 in their house, meaning a stunning majority, was written to take effect last Thursday, September 3. But was just extended by 14 days, two weeks to September 17th Thursday, after next. That, onerous and entirely unworkable bill, deems a person who is physically located in Utah to be a Utah user regardless of the IP address they present to any age-restricted internet service.

2:28:32It's shorthand is the anti VPN legislation because it's clearly meant to curtail the use of VPNs and other proxies as a means of geo relocating. No one has any idea what's going to happen there, but it's going to be interesting to watch because, you know, basically they are trying to prevent adult age-restricted internet service websites from allowing connections from VPNs, but not all VPNs declare themselves as such. So my point is there is a clear tension growing between the legislatively protected privacy rights of citizens, and in some cases we have, we have constitutionally protected privacy rights of citizens in many major democracies, and the perceived needs of their own governments to conditionally violate those rights. In an analog and pre-encrypted

2:29:40world, law enforcement and intelligence services were able to sneak around to get what they believed that they needed. And even after nominal encryption was in place until the advent of AI, the large supply of latent bugs allowed these same agencies to ignore privacy whenever they felt it was necessary to do so. Matthew's point is that fragile status quo is soon to end. What will replace it? Good question. He's smart guy. He's very AI aware too. He's actually a pretty pro AI guy. So yeah, and he's also one of the guys that, you know, that the legislators pull into, you know, for Senate hearings to find out what he thinks. And I mean, so, you know, here we're saying,

2:30:41look how much more secure window is today. Windows is today that it was yesterday with a thousand new problems fixed. Yeah. And and we know that Android and iOS are going to be on and Mac OS are going to be on they're all going to be on the same curve. They're all going to be getting the benefit of this. And we could argue that AI is going to help future errors as Matt said, not get into production code. So we're going to fix this. And you know, Apple has struggled to keep their phone from being hackable. It's been a struggle. They're probably going to win. And then what? Well, I mean, they've been complaining about going dark as you pointed out since James Comey. And yeah, they've been taking on, but even the more ways of seeing into our lives than ever before. Exactly. And people like the NSO group with Pegasus have been able to keep compromising people's phones.

2:31:48Yeah. What what happens when that changes when they can't? Well, then we're back to the way it used to be when when law enforcement didn't know every darn thing that was going on inside your house. Exactly. We were we were talking over analog phone lines and wondering if that little click and static sound with somebody listening. There's no question law enforcement is always going to want more even if they weren't going dark. They're always going to be pushing for this. Right. This is just they have been and they will continue to push. Well, the problem is our legislators are now going to you know, they're going to come under pressure to to to to legislate this. Right. I see this as part of a much larger uncertainty in general. We are entering into it's really I would say fairly chaotic. We are in a time of phenomenal change. Right. One thing I mean one thing we know about chaotic systems is very hard to predict. It's just not they're not deterministic and yep. I think

2:32:51I think this is chaos. We don't know what AI is going to do. We it could it the scale ranges from it's just more computer programming to it's it's an alien intelligence in our midst and I don't know where it's going to land on that scale. And I don't know what the disruption is going to be. Well, people lose their jobs. I'm not even enough that's clear. So it's all very through a glass darkly. So I don't I just don't know if we can make any sensible plans. I guess is what I'm saying. We should there's a storm of coming. And I don't know what you do to prepare for it except maybe you know get more rice. I don't know. Buckle up. Buckle up. It's going to be a bumpy night. Okay. Last break. And then I am very excited to share this endless lifelong engineers observation about the danger of become overly reliant on automation. Yes. And we have a new a new

2:33:55automation capability in town AI AI AI that's I have been saying. All right. We're going to get to that just a bit before we do. Oops. I don't know why it's doing that. Let me turn that off. Thank you very much. Restream is just giving me all sorts of fun. Before we do, I do want to talk a little bit about our club club, which makes this show and all the shows we do possible. Without club tweet, we would have to cut back at least 30% maybe 40%. That's a third of our shows, a third of our hosts, a third of our staff. And I don't want to do that. I like what we're doing. I hope you enjoy it. I think you're getting value out of it. And if you are, I'd like to invite you to join the club. Here's a sad truth of the matter. Only about one and 15 people who listen to this show are club members. If we could get that to one in 10, we can get that to one and actually not one

2:34:59and 15. What am I saying? It isn't even close to one and 15. It's about one in 50. Maybe. It's not even 2%. It's less than that. If we could get to one in 20, we would never have to worry about ads or anything else. We would be well supported. We could grow. We could have new shows. We could have a bigger variety of shows for you. I just, I feel like we're so close and your help really could put us over the top. I would like to not be beholden to advertising. I would like to be supported. I don't ever want to do a paywall. I think what we do is too important. I want to always give it away. But if the people who can, and I know you can't, you all can't afford 10 bucks a month. And if you can't, that's fine. But if the people can join the club, support what we're doing. We can go into the future. I think doing it even better. So that's my pitch. Twitter.tv slash club Twitter is the website. Everything we do is expensive. Running this website is expensive.

2:36:02Running these shows is expensive. Paying our hosts and our staff is expensive. It's not me. I often don't get paid at all. That's fine. But I do want everybody else to get paid. So please Twitter.tv slash club Twitter. You do get access to the club to discord ad for you versions of all the shows chapter markers and all the shows. You get all that special programming. For instance, tomorrow, we're going to cover the apple. Kino, we can't do that in public. Apple doesn't like that. But we can do it in the club. It's a private broadcast. So that's how we're going to do it. If you want to see those private shows, the only thing we do that's private. Twitter.tv slash club. Twitter. That's all. That's all I'm going to say. Let your conscience be your guide. No, I don't want you to feel guilty. If you can't afford it, if you can't do it, that's fine. But we would sure like to have you in the club insurance isn't one size fits all and shopping for it shouldn't feel like squeezing into something that just doesn't fit. That's why drivers have enjoyed progressive

2:37:04name your price tool for years. With the name your price tool, you tell them what you want to pay and they show you options that fit your budget. Enough hunting for discounts, trying to calculate rates and tinkering with coverages. Maybe you're picking out your very first policy. Or maybe you're just looking for something that works better for you and your family. Either way, they make it simple to see your options. No guesswork, no surprises. Ready to see how easy and fun shopping for car insurance can be? Visit progressive.com and give the name your price tool a try. Take the stress out of shopping and find coverage that fits your life on your terms. Progressive casualty insurance company and affiliates. Price and coverage match limited by state law. This episode is brought to you by TickTack. Lace up and get your game on because the TickTack lineup just got a new teammate. Dr. Pepper TickTack. This new double threat puts two of your favorite flavors into one box of mince. And when two iconic flavors come together, the result can only be

2:38:05iconic. Score big. Try Dr. Pepper TickTack today. Now on we go with Mr. G. A guest submitted article recently appeared in the IEEE spectrum publication. It captured my imagination and it feels very important to me. And I believe it's going to resonate deeply with many of this podcast listeners who have been around the block a few times. For those who don't know, IEEE is the abbreviation for the Institute of Electrical and Electronics Engineers. It was founded as the IEEE, the American Institute of Electrical Engineers. Believe it or not, back in 1884, an astonishing 142 years ago. And then it was later renamed to IEEE after its 1912 merger with the Institute of Radio Engineers, the IRE.

2:39:06And to put the Institute's age into perspective, it was at the start of the year of its founding that a penniless genius by the name of Nicola Tesla arrived in New York to work for an already famous industrialist by the name of Thomas Alva Edison. But anyway, I digress. Today, the IEEE describes itself as the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. And the article I encountered, which so galvanized me, carried the headline, AI efficiency could cost us the next generation of experts. And the articles, the articles teaser red, lessons from aviation and nuclear power show how to preserve human skills. And as I said, I believe what's said here is extremely important. So,

2:40:07so see what you think. It's author wrote, a little over a decade ago, I led the controls design for a first of its kind, full digital control system for a US nuclear plant. It was, on paper, a beautiful machine engineered to run itself the way a modern airliner does, with operators watching over a system that rarely needed them. And we made a decision that to an efficiency minded observer looked backward. We deliberately left manual steps inside sequences, the system could execute on its own. We were solving a specific problem. An operator who only ever supervises automation slowly stops being an operator. The hands go cold. The mental model of what the plant

2:41:10is actually doing gets fuzzy. Then comes the day, the automation hands control back. It's always the worst day, because automation only quits when it's confused or in trouble. But by then, you have a person in the chair who has not truly operated the thing in years. The manual steps we included in its design were there to keep the human current. It was inefficient by design on purpose. The plant, as it happened, was never built. It was shelved amid the politics and economics that surround nuclear power in this country for reasons that had nothing to do with the engineering. But the design instinct outlived the project. And I've become he wrote to I've come to believe it's the most useful idea I can offer to the argument now

2:42:12consuming every boardroom. What happens to human expertise when AI does the work that used to build it? The data has become hard to wave away. A Harvard University working paper covering some 65 million workers and more than 280,000 US firms found that after companies adopted generative AI, junior employment fell roughly 9% within six quarters relative to not adopters, while senior employment kept right on growing. A Stanford analysis of ADP payroll records points the same way. The youngest workers in the most AI exposed occupations lost ground after late 2022, while their more experienced colleagues held theirs. The Stanford researchers found that the losses concentrate where AI automates the work, where it merely augments junior employment hold

2:43:20steady or rises. The causal story is still contested and honestly require saying so. Researchers at the New York Fed attribute much of the rise in young graduate unemployment not to AI but to remote work, arguing that firms are reluctant to hire inexperienced people whom they cannot train and mentor at a distance. But notice what the explanations share, whether a model is absorbing the formative work or distance is severing the mentorship around it, both describe the same broken mechanism. The apprenticeship channel through which expertise passes from senior to junior. Either way entry level has quietly come to mean three years of experience required. Distrib away the noise or he says strip away the noise and you're left with one deceptively simple problem. You cannot become a senior engineer without first being a junior one. Expertise is not

2:44:28downloaded. It is earned through failed builds, dead end debugging sessions and the why on earth did that work moments that a capable AI will now happily spare the newcomer. Spare them enough of those and you produce a cohort that can supervise a model on paper but never developed the gut sense to know when the model is confidently, catastrophically wrong. Most of the commentary stops at the diagnosis or reaches for policy solutions that treat the loss of junior jobs as an economic problem. Yet it's also an engineering problem and safety critical fields have already spent decades learning how to solve it. He said my own career began at the sharp end of automation. My first job out of school was verifying and validating the software in digital jet engine controller that in the digital

2:45:34jet engine controller that decides faster than any pilot could how a fighter planes engine responds. Even then in the late 1980s the central tension was visible. The machine outperforms the human in routine cases but the human is all that stands between the aircraft and disaster in the cases the machine did not anticipate. This tension is known as the automation paradox in which increasingly capable automation gives human operators less practice while leaving them with the only the most difficult situations. Aviation learned repeatedly and expensively what happens when human skills atrophy inside that gap. The canonical example is Air France Flight 447 which fell into the

2:46:38Atlantic in 2009. The proximate was mundane, iced over air speed sensors fed the autopilot bad data and it did what it is designed to do. It disconnected and had a control of the airplane back to the crew. What followed was not a hardware failure. It was a competence failure. A recoverable situation became an unrecoverable one because the pilots conditioned by thousands of hours of watching the automation fly could not read a high altitude aerodynamic stall and hand fly their way out of it. The airplane was working. The training the automation had quietly eroded was not. The industries the industries response what is instructive and it's the same move we made

2:47:39in that nuclear control room. It did not rip out the autopilot but built deliberate manual practice back in. In 2017 the FAA issued safety alert for operators 17 007 manual flight operations proficiency declaring that manual flight is the foundation upon which other technical flying skills are built. The alert formally recognized skill decay as a hazard in its own right. Some airlines amended their procedures to encourage hand flying both the initial climb and initial descent in benign conditions knowingly trading a sliver of fuel efficiency to keep the crews raw flying skills alive. That trade is the whole point. A perfectly optimized system that produces incompetent operators

2:48:40is not optimized at all. It has simply moved its failure mode somewhere the spreadsheet cannot see. Put the aviation lesson and the nuclear instinct side by side and they point to one design pattern we now need in AI augmented work the deliberate manual gate. A manual gate is a point in a workflow where a human takes the controls not because it is the fastest way to get the task done and not as a safety interlock but specifically to exercise and preserve a skill that would otherwise decay. The distinguishing feature is that it is chosen. You decide as a matter of design which competencies your organization must keep alive in human beings because those are the ones you will need on the bad day when you engineer the friction required to keep them warm. Picture how this

2:49:48might work on a software team that leans on AI for most of its code. The team places a manual gate around the skill it can least afford to lose debugging. When a defect surfaces in a critical module the assigned engineer deliberately often a junior one must first reproduce the failure, trace it to root cause and write an automated test that captures the bug all without the AI assistant switched I'm so all with the AI assistant switched off only after the engineer commits to a diagnosis does the model come back on to propose the fix generate alternatives and sweep the code base for similar bugs. The engineer then compares their diagnosis against the models when the two disagree that's the

2:50:48design working surfacing the disagreement before the bad day instead of during it it's also the design teaching and training the junior engineer. This approach reframes the junior engineer entirely. The instinct today is to let AI do the entry level work because it is faster cheaper and capable but some of that work is not overhead to be eliminated it is the training apparatus of your future senior staff and you should protect it the way you'd protect any other piece of critical infrastructure it may not be efficient to this quarter but dismantling it quietly mortgages your capability a decade away none of this is free and pretending otherwise would insult the people who have to sign the budgets a deliberate manual gate is by construction less efficient in the near term than

2:51:52full automation keeping juniors doing formative work and running the manual sequences cost something now to protect something later that's a hard sell in a market that judges most leaders on quarterly results a hired executive who carries he has an air quotes unnecessary humans that AI could replace will hear about it from the board long before the payoff arrives the math only works for someone insulated from that pressure a founder with control a private company an institution with a genuinely long horizon or a regulator willing to require workers to demonstrate their skills regularly as pilots must this all means the organizations most likely to preserve their own expertise are the ones structurally able to spend short term margin on long term capability everyone else will need

2:52:58a push from the outside so here's the argument in one line deliberate inefficiency is not waste in safety critical engineering we have always known it as insurance and we buy it on purpose as AI takes over the work where expertise is forged the smart move is not to resist the automation it is to keep our hands on the controls by design so that when the automation fails as it always eventually does there's still someone in the chair who knows how to fly so I think that's a fantastic piece of well-reasoned engineering and I would recommend it without hesitation to anyone's boss who may currently be enraptured by AI's truly deliverable ability to eliminate all of those pesky

2:53:59junior engineering jobs you know we already had stunningly good AI with GPT 5.6 fabled and mythos and and though it's going to take some time for the world to fully assess what open AI has just given us with GPT 6 astra the early take is that it currently you know outperforms some of the other frontier models you know from open AI andthropic and and and Google whether or not and to what degree that's true you know and even the fact that this is nothing more than a snapshot in time that's important because of course this wasn't true a week ago but maybe true today and we know that we are just at the beginning of all this we can feel how how rapidly this field is changing and so that's my point if we've learned anything it's that no one will ever have a long term defensible position in AI

2:55:03sure you know they're going to be bouncing back and forth and Leo you already knew that anthropic was what a week away or so from you know releasing their next model in in this you know never anymore supposedly I mean it's it's a rumor anyway that they're ready with fabled 5.7 yeah anyway I think without the deliberate creation of an environment that's that is designed to nurture and mature junior developers organizations will be left completely dependent upon automation for mission critical functions I mean I know that that coders who come out of out of university with no experience they're just not useful they actually can't do anything and you could argue that they're going to be a lot less useful all they'll know how to do is drive AI no coding I don't even know if programming will be taught in the future it'll be prompt engineering right

2:56:03and you think that we need to know how to code um we probably need to know how to read what AI does when it writes a bug you know can we I don't know yet whether we're going to be able to say to AI fix the bug which you created right maybe we will I mean it so I guess the question is okay so certainly there will well AI is also coding itself now isn't it so maybe it will be a completely lost art except here in Irvine yeah yeah right there'll be a few people I mean I think there'll always be a few people who code by hand just like there are a few people who make their own furniture but um and play the piano yeah instead of you know turning on Apple music yeah I just I

2:57:08don't know I don't code is a funny thing because code is getting the computer to do something by translating your English language thoughts into machine code no Chris well no you're you're in 10 you're in 10 you're in 10 you're in 10 because it might be unvoiced right I mean all and as we've moved higher up you still you do a low level coding but most people are doing high level coding which isn't really that closely related to what's going on in the computer um I guess there are there is a case to be made for people who need to be able to look at the traces and see what's actually happening well for a long time compilers had bugs and which meant you would you would properly express yourself in the high level language and you still wouldn't get the program doing what you told it to do right this to me is another case of we don't know what the hell's going to happen because there are people talking about AI slop code and that we're now talking with

2:58:15that we're going to be introducing a whole new generation of bugs I don't know one way or the other I think that's old farts talking uh good well be yeah I think that that's people are very tied to the old way of doing things uh slop is a uh pejorative that I don't think is is completely fair um it doesn't code the same way you do that's true but the thing about computer programs is they're you know if they're correct they work promise these little edge cases where it's it works most of the time doesn't work all the time but I think a computer is ultimately I at least I don't see any reason why a computer wouldn't ultimately be the best at figuring out what's going on it's uh right it's better than we are I was may arguably one of the early people to say that AI was going to be extremely good at code this is what it does it exactly it understood it's it's opera you know so really what the AI is doing it's hard for the AI is is is is it good at us and I think most of the time when the AI

2:59:20fails us is because it's not as good at us or we're not as good as at it it's a communication problem between us and the AI a misunderstanding or miscommunication or mis misdirection I mean I literally I look to code in a long time and I think most many of the people like Darren who's a very proficient coder uh in our discord I I looked up to him because you know he's the guy who would finish all the advent of code problems lickety split so he's a very very accomplished programmer done it for years his whole life he says I haven't coded anything myself for like a year wow uh I don't think he and I don't I think many of the most proficient coders I know people like David Hannah Meyer Hanson don't look at the code now his operating system on marquee which is a version of Linux it's entirely AI coded is kind of wonky but that doesn't I'm not sure I blame the AI I mean

3:00:22I'm many of the coders I who are using AI at this point of stop looking at the code so I don't know what that means I don't know uh I think and Darren saying you can absolutely build great quality software without knowing anything about rating code right now and I think that's only going to get better does it but they're neat but it's absolutely true there's needs to be an entry level positions you know I think the analogy is to the business I'm in broadcasting um you're terrible when you start you really are you're not good at it and you need somewhere to start you need basically it took me a year to find a radio station that was so tacky and small that it would let me learn how to be a broadcaster there and they put me on the middle of the night midnight to 6am on Sunday morning

3:01:22because that was where I could do the least harm yeah but I gotta need to have to learn and it's if you know we know it's called apprenticeship right the idea is you apprentice with a master and you learn his art right uh but then I look at my son who really has no broadcast training who just did it all in public on TikTok and it's done a little bit that's pretty well for himself well put it in the net and he did it by the way in a a hundredth of the time his dad did I mean this is literally he's gone from nothing to total success in in whatever this business is like it's sort of like broadcasting uh in a couple of years took me 20 years to 10 years to get halfway decent so it's a different time and I think a lot of a lot of what you're seeing in a lot of this complaining and caving about uh AI is also the older our older generation going well that's not how you do it my day yeah hold the hammer and you're ahead how do you know what's

3:02:27in the registers oh yeah you gotta know what's in your registers you don't you do you and you know intimately but I don't know if you need to and certainly somebody's programming C++ probably doesn't no you don't have any access unless you explicitly tell it you want yeah you don't really care bit of masm you don't care so I don't I just don't know it's I'm we're an unknown unsure riding a beast where it's a tsunami headed for us yeah and there are a number of people who are going run and the number of people going oh look the water's going out I see seashells I think I might be in that latter group way and the waves gonna hit me Steve always always provocative intriguing entertaining and informative it's a great show and I really appreciate you're putting all this work in you do every week in fact you can subscribe to his show notes 21 pages this week of really good stuff read along with the show read it you're yourself there's

3:03:32links there's images that's everything you need it's basically a book every week have you recounted the number of words is a three or four thousand five thousand eight thousand it's a lot it's a short story for sure every week and you can get it by going to grc.com slash email there's two things you'll accomplish by going there you can give me your email and he will white list it so you can email him which is really nice send him pictures of the week ideas questions comments suggestions but below that there's two check boxes one for the newsletter the the weekly mailing of the of the show notes the another for a very infrequent mailing of new products speaking of products Steve has of course spend right the world's best master of maintenance recovery and enhancing utility and you can get it right now at grc.com that's Steve's bread and butter handwritten he knows what register every single register in that program he knows what it's doing at all times right there's no mystery it's you know exactly what's

3:04:37happening in that machine which is kind of amazing uh and and it's really good as a result he also does a lovely little $10 program called the DNS benchmark pro so you can get the fastest DNS you know AI can't do that for you you have to have a deterministic program like bench you have to somebody has to write a program to do that and Steve did thanks to Steve he has the show as well he has 16 kilobit audio and 64 kilobit audio he also has a wonderful transcriptions written by a human being a lane ferris to come in usually a couple of days after the show all of that at grc.com we have audio and video somewhat larger audio file and video as well you can download that from twitter tv slash s and there is a youtube channel dedicated to it and actually the best thing to do is subscribe in your favorite podcast client so you just get it automatically you don't have to think about it we record live every Tuesday right after mac break weekly that's 130 pacific

3:05:38430 eastern 2030 UTC and you can watch us do it live if you want the absolute unedited best version of the show all the expletives all the nudity it's all there you are not working and rebooting yeah mostly that if you watch live there's a several places you can do that of course if you're in the club and I hope you are the discord but you can also watch on everybody can youtube twitch x facebook linkedin or kick and if you're chatting there I can see the chat so we can we can chat back and forth they often do that um steve that's it for us I will see you next week and maybe I'll have a new iPhone no I won't have it by then until then and we'll have a breakdown of patch Tuesdays a fantastic patch count and we'll know what apple has wrought so holy come holy thanks steve thanks everybody we'll see you next time on securing that hey everybody the

3:06:39shows over but the listening continues at twitch.tv in fact we've got so many great shows I don't even know where to start well how about intelligent machines that's a show we cover AI robotics and all this smart stuff all around you with two of the most fun smart people I know Paris Martino and Jeff Jarvis it's often philosophic we sometimes debate but if you're interested in AI in this world that is changing so fast this is the show for you intelligent machines you'll find it at twitch.tv slash iam our website or wherever you get your podcasts I'll be looking for you it's a cure now es polontequila balanced delicious 100% agave 0% ego just add ice time and your mouth for real margarita ride the rooster es polontequila 40% alcohol by volume 80 proof copyright 2026 commentary on air fi e reporting reasonable

3:07:53no