Why AI engineering needs old-school discipline

0:00ThoughtWorks is a leading global technology consultancy that delivers extraordinary impact by blending design, engineering, and AI expertise. For over three decades, we've led in technology innovation, and today we're at the forefront of AI-powered software and data engineering. Welcome back to another episode of The New Stack Makers. I'm Friedrich Klauding-Wyme, the senior editor for AI at The New Stack. And I'm Nimesha Estegiri at ThoughtWorks. I'm the data and AI advisor. Well, Nimesha, thank you for being here. Thank you, Friedrich, for having me. Absolutely. It's a pleasure. Now, one thing I was thinking, ThoughtWorks, I've heard the name, but I'm thinking quite a few folks in our audience may not have heard it yet, they may not be familiar with what you're doing. So, maybe give us a short background on what it is that ThoughtWorks does. Yes, of course. So ThoughtWorks, we're a global digital consulting firm.

1:02We help our clients around the world with various things, from strategy, design, engineering, and these days, really AI, and our phrases, it's not just AI, but AI that works. And why that is is really taking up, taking our decades of experience, bringing thought leadership to the industry and bringing the latest and greatest ways of thinking about building software, building it right, and building the right things. So, everything from our product, thinking, capabilities, and skills, as well as agile and engineering and platform engineering practices. What is it in your experience right now that these companies are looking for specifically? Like, what are they struggling with still? Well, I mean, these days, there is, so since 2023, in November, 2022, this whole wave of generative AI, and then the next year of doing a lot of proof of concepts and really

2:08exercising it, trying to now upskill their own employee force, companies are looking for understanding how do they actually bring a lot of their proof of concepts to production. They're also looking for how do we get our development work that we're doing, so it's not just accumulating code, but it's actually viable code and code that can really execute and run successfully. So, I think Gartner was saying, like, 40% of agentic projects will be canceled by 2027. So these are staggering numbers and just imagine the billions of dollars that are being put into the industry and then therefore waste in some ways. Not a waste in terms of learnings, there's been a lot of learnings and rapid learning with the pace that is there in industry, but definitely in terms of business ROI, one may not actually achieve that. No, absolutely. And I keep hearing the same numbers kind of, I don't think it's really the needless

3:08and move there all that much, but what is it that these companies are getting wrong? Why are 40% of these projects getting canceled? Why aren't we better at getting value out of these tools, which are really powerful? I think it's the question that is being asked. The question that we're hearing a lot from executives and others is, how do we go faster? How do we go faster? How do we keep relevant? Where I think the right question or another alternative, better question here might be, what do we build, given the latest technology that we couldn't build before? So reimagining and reinventing what we couldn't do before, and that's where we focus our energy rather than everything. And then secondly, how do we build differently? Now that we do have AI, and it's not just as an assistance, but actually even a part of the team with human and machine agents working together, what does that look like? That is not, therefore, just a tool change, but a systemic systems change, rethinking

4:12your models of what to build and how to build. And that also then implies how do you measure? And the measurements in the past, where we might have just measured output and production of how quickly are people bringing in pull requests or code into production. Instead of that, it could be more about, for instance, iteration cycles and interactions with your AI and with others. It was collaboration and interaction metrics that are also, how your first past acceptance rate, right? How that code generated, how quickly can it be accepted and with minimal rework? As you mentioned, systems thinking, they're talking to me a little bit more about that, what needs to change there, how the units and both the change that's practical, but also change kind of in the culture, I think, of how these projects develop over time.

5:13Exactly, exactly. So I think it is the typical people process tech coming together, but I think when we're thinking, rethinking this, I think there's a lot of thinking about the organization at large and you're thinking about the technical platforms that you have in your organization and what platform capabilities you may need to bring in that become strategic assets. And so therefore, that then supports the system at large and you're using, therefore, your platform as paved roads that people can use to accelerate their work, but also becomes that the conduit to the governance that you also need to apply and the efficiency gains. So all of that kind of coming together, and for ThoughtWorks, we've been a progenitor of a lot of platform and engineering excellence capabilities, and that is now, once again,

6:13coming to the fourth front, where yes, there's a lot of fundamentals from the past that need to be actually, and at this point, reinforced, if anything. And so then you start thinking about, if you have that harness and the technical system in place, then that's where we now, you're elevating the people to say, hey, use more of your human judgment. Let the other things that we have been jaded in the past of repetitive work that we may have just, that's what I do when I come to work, but kind of rethink that. So there's a little bit of unlearning so that you can learn this new way and where your human judgment and the higher order, you know, a value of your human effort can come into play. Yeah, yeah. In practical terms, as we're thinking about some of these kind of standards that we've had for many years, like, is it that we need to bring back their focus on, you know, what are some examples that you've seen? Yes.

7:13So for instance, it might be everything from like mutation testing, right? And so test-driven development, right? So we're really thinking about creating those feedback sensors for AI. So that the AI, when it's now with autonomous coding agents, and there's been a drastic change right in December with the latest models and so forth. So and there's a lot of, therefore, keen interest in generating and designing these autonomous coding agents. But you want to, what do you want to bring back? So I think there are things like mutation testing, or so a lot of our testing principles, and there might be things with test-driven development, for instance, designing that type feedback loop for the AI so that it can continuously learn and evolve, so that being a key component, I think the other thing would be even the metrics, such as even like door metrics. So I think those types of things with deployment frequency, lead time, change failure rate, these are our lagging metrics that will then ensure that, okay, yes, we are moving the

8:17needle in the right place. But I think there are also other things like zero-trust security architecture, right? And really thinking about the ensuring that we have proper identity management and security as well. And we're seeing the propagation of agents, and now they're coexisting along with you when you're working on your laptop and your desktop, and a lot of that changes that are happening. Zero-trust architecture is critical, and being able to know who did what, as well as the authentication and the authorization of the work that is happening. A lot of traditional, fundamental ways of thinking about engineering discipline, but just really becoming now back into the forefront. Yeah, it's about me through a little bit, but what's the best case scenario here that you've seen? Like, the company that has done this really well, and if what does that look like?

9:21Yeah, I think that this is where we're going back, right, to the systems thinking. I think the companies that we find who aren't thinking about their overall strategy, right? And designing that, and so there's a little bit of, let's think this through before we go ahead and jump and require top-down mandates. I think those are not as successful, and those were finding as anti-patterns. So it is more about when there are successful, are doing the due diligence, it's hard. It is hard work, but to provide the literacy and the enablement within your organization for the people, and then to really leverage the ROI of your work, a lot of strategic thinking as well, to think about where do we invest as much as, you know, how, and with what tools?

10:21Yeah. Yeah. I feel to me, like, in this time of AI FOMO, strategic thinking isn't always at the forefront. Is that something that, you know, really needs to, would you say a lot of these companies need to slow down a little bit potentially and just, you know, think over what they're doing and whether maybe AI is even the right tool for them at this time? Yeah. I mean, definitely. Yes. And I think there is a responsible AI perspective here as well as responsible leadership and technology aspect to it, but why now more than ever is because you can lend yourself to create generating AI Slop, or the AI is going to produce a lot of, you know, what you tell it to produce. And so, and also what you tell it not to produce, you know, there is without proper feedback loops in place. So, I think that is why, like, once again, even the engineering discipline we talked about, there is strategic disciplines as well.

11:23And, you know, bringing back the disciplines that might be in place about what is your competitive advantage, right? I mean, you don't need to do what the joins as they're doing just because they're, that's what they're doing, right? So, how do you want to differentiate and where do you invest your money? So, yes, definitely that's also part of that. And that requires getting the, this is where the human judgment comes into play. For sure. For sure. Now, when we haven't talked about agents specifically yet, but we should talk about that a little bit because that's, you know, basically becoming the default now for at least development teams. How's that changing how you're talking to your customers and the problems they're facing? And you mean coding agents, right? Yeah, coding agents. Sorry, coding agents. Yes. Specifically coding agents. Yeah. So, I guess seeing that in especially two different ways, but one is the, you can think

12:25about it from the topology shifts that we're seeing as well as architectural shifts. And so, I'm part of this global team that puts together the, that works technology radar where we're, you know, finding trends and we're learning from on the ground actual experiences from thought workers, from our thought workers globally of 10,000 plus employees. And one of the things that we're finding is that we're using, we're putting together techniques to ensure that the code base is not shifting or drifting away from architectural principles. Right? So, those types of things are very important when we're also thinking about coding agents. And so, I think there is the latest technologies with agent skills as well as agents at MD was from before, right? So, a lot of these things that help provide the context to the agents.

13:26We're thinking about that as essentially forward engineering, right? Like, forward constraints for them in terms of the guiding principles. What are those architecture decision records that we would ask the agent itself to document so that the humans could review that and could reference that? Because at this point, like humans reading all the code that's going to become more just, yeah, unsurmoned. Yeah. Yeah. So, having those ways to be able to validate that. And on the other hand, the team topologies are also changing because now we have humans and machine agents working together and collaborating. And so, there we're seeing these teams of coding agents where it might be a lot more strategic and intentional, deliberate design, you know, who might be orchestrating very role-specific, okay, this machine agent for backend versus front end and whatnot.

14:28So, that we are finding and we put in our radar as a technique that people can assess. But the thing that we put on the technique as people to maybe caution and watch out for and just don't jump in without some other additional experimentation and testing or the coding agent's swarms that are coming out where it's like hundreds of agents being tasked to do the same thing potentially. And then you have to think about collaboration and, you know, they might run into conflicts that they have to resolve. And so, they're, you know, right now we're like, it's still maturing. So, for organizations around the forefront of trying these things out, then okay, fine, go ahead and continue to evolve and provide more practice, best practices for the industry. But for others who have maybe more regulatory compliance requirements and other things, there might be more cautionary steps towards it.

15:28So, overall, I'll just say that really this is still an evolving field and so it's dramatically changing. And one of the great things about the tech radar is that it just allows you to give, it gives you a glimpse of what are the things that we're all testing, experimenting versus what are the things that have evolved much further that you can, you know, choose to adopt. Yeah. As you've looked at that radar lately, anything else that stood out for you any surprises there? Um, surprises there. Yeah, that's interesting. I guess the, the, the, the, so this was my third time being part of this team that put this together. One of the biggest things I'll say was, we used to call this, we used to coin this term two complex to bit blip, which is still the case sometimes because our blips are very short and sweet and very quickly get a glimpse of a new technology or new technique or older techniques

16:29and technologies that we want to reinforce. Um, but this time there were a lot of too young to blip. Okay. So it's moving so great. Everything is moving so quickly and we're, and like a thought worker may have put something out there like because everyone's also experimenting on our organization and others. So they might put out a blip that that were like, oh, wow, this might, this is an interesting open source project that tackles this white space in the industry, but it just came out two weeks ago. So we're like, okay, well, we publish our radar maybe two months after we have this meeting. So, um, would it be, you know, is it the right time to still anticipate the maturity of this? Or is it, um, you know, one of those things that comes out and then, I mean, not last. So I think there's, there's definitely, um, yes, something about that as well. Yeah. Two weeks is a long hype cycle right now, a lot of these projects. Yes. Yes. We get $50,000 in that time.

17:30Um, one thing I just wanted to go back to you said, you know, I think it's another fundamental issue we're all dealing with is, uh, a lot of code is being generated now a lot more than before with all of these tools. And one thing that keeps coming up in my discussions with people is how that creates new bottlenecks all across kind of the life cycle from code reviews on rate really after that. The code has been written like, what are you seeing there? How are people dealing with that and what's your advice? Yes. Yes. Yes. And we blip this actually in the past radar and also, uh, I think we reinforced it here as well as, um, first of all, I think a perspective of cognitive load, cognitive load for the human agents as well as for the machine agents. And, and this is where once again, good architectural principles like, um, thinking about the boundaries of your code, um, and, you know, from before we've talked about modularity and, you know, from 1970s, you know, partners as paper like once again, it becomes very evident

18:35here as well. So why that, that as a technique is important is that for the machine agents themselves, I think for their own, you know, what we feed into their context, uh, windows, um, but also for human, uh, cognitive load to be able to review and understand the code, write a lot of that modularity, um, helps and comes into bear. Um, and I think, uh, uh, once you have that, then you can start also thinking about the, the harness that you are developing and that harness, including, you know, those guard rails or architecture guard rails, what are those feedback sensors, right? So, um, in addition to, right, your, the, the, the, the feed forward of your context that you provide your agents with the feedback with the sensors and the, um, you know, the tests and linteres and a lot of those common practices that come. Sure. Sure. Those are not going away. They're not going away. And if anything, it's like, yes, how might we do better?

19:37Hmm. I do want to share though that I think the other thing is to maybe rethink a little bit. This is maybe a little bit more advanced, uh, uh, advanced in the industry right now, but rethink of how we even think about code itself because, yes, um, the, the quantity of code is, is, is, is just going to, you know, dramatically increase with how quickly AI is able to produce it and then humans become the bottleneck in that case. But I, I think, uh, there's an opportunity to rethink where does code matter and the volatility and durability of that code. So, what I mean to say is, first of all, this is why strategic thinking is important about what's actually valuable for the organization, where do you want to build versus by? How do you want to actually differentiate your organization, right? Should we even bother building it? So that is very important because it's so easy to build these days that it's, it's going to get even easier that because code is, you know, going to become a commodity to generate

20:38that you don't necessarily need to spit it becomes, there's a lot of dark code, right? Like we have a lot of dark data. We've been collecting a lot of data through our big data initiatives. But now, you know, there's a lot of dark code more, more so than even before. Um, but, uh, not that there wasn't dark code. The second thing is, uh, so viewing, you know, should this code even exist, I think the other thing is what is the volatility of the code? So, because of it's so quickly to also create POCs, how might you architecturally or from your system standpoint, think about documenting that code as having a, a, a life cycle that would then eventually be, you know, get deleted, you know, so the, the, the, um, retainment of that code, um, being very explicit about it. But secondly, um, uh, the, um, thinking about code that could be dynamically and more affirmatively generated, right?

21:40Because like someone, like a user wants to be able to access a particular, uh, interface, let's say, or an API, well, if I don't have the agent skill for it or if I don't have that built already, and it's, uh, not a necessarily reusable, you know, feature, then why not go ahead and just dynamically generate it for that particular single of it for purpose use and then you're done. So, so I think there's, it's going to be that perspective of how we really think about code differently. Yeah. Yeah. Yeah. It's interesting. It came up, I think first in discussions a year ago or so and the models weren't quite there yet. I think at the time that people were thinking about it already, like, maybe it's the spec is more important than the code because I'll just regenerate the code as I need it and the model gets better and then just have better code at the end and just don't worry about all of the. Yes. But then we're going to still shift the needle then to the spec and the spec is to load, which we're also seeing as well. Um, and for that, I think we do have a technique by the way that our thought workers proposed,

22:44um, we were calling about progressive context disclosure. Okay. Um, where, you know, progressively, uh, because remember, like I said, the machines cognitive load as much as important as our human ones, um, and, uh, and the progressive discontext exposure, you know, we are being explicit and intentional, I'd say, about, um, what matters for this particular request. Right. Right. Because that's super spec is also going to become difficult and then you want to think about that modularly as well. I think I got to spec a mono repo of specs. Yeah. They'll keep us, they'll keep us pretty for a while. Um, for those who want to learn more about the radar, is there a place they can go to is that public or is it only for your clients or with this? Oh, no, no, no, no, definitely, it's, it's public. I mean, our, um, our goal is to take a lot of the learnings that we're having and then the escalation and then share it with the industry. Um, so if you go to thoughtworks.com slash radar, you'll be able to see our latest one.

23:46Uh, we do have our, uh, the, uh, the, uh, the next version, the next edition is coming out next week, actually, April 15th. So volume 34 will be out. So, um, yeah, if you just wait a few days and, and, and, and see that one, then you'll get the latest and greatest. The end. Awesome. Perfect. Well, the Misha has been a pleasure. Yes. Thank you so much, Patrick. It was great to, to be here. Same here. Same here. Thank you so much. Okay.