Numerics.NET with Jeffrey Sax

0:00What does it actually take to rent a remote lodge in the Amazon or guide travelers across Antarctica? On the Adventure Life Travel Show, host Jason Maynard sits down with the expedition leaders, lodge owners, and trip planners who make adventure travel happen, the real stories behind the destinations. New conversations every month packed with insight you won't find in a brochure. Search Adventure Life Travel Show on iHeartRadio and start listening today. This is Matt Rogers from Lost Culture East. This is Matt Rogers and Bowen Yang. This is Bowen Yang from Lost Culture East. This is Matt Rogers and Bowen Yang. You know when people try a new food and suddenly it's like, okay, hold on. I got a new favorite food. That's the reaction a lot of people are having when they first try Cupid Mayon. Yeah, it's the one with the red cap and the little baby on the bottle. You've probably seen it at the grocery store. And this mayo is different. Most mayonnaise uses whole eggs. Cupid only uses egg yolks, which gives it this rich umami flavor. It's smoother, deeper, almost buttery.

1:00Once people try it, they start putting it on everything. Eggs sandwiches, fries, burgers, chefs use it, restaurants use it, people who really care about flavor use it, put it on just about anything. Then you'll understand Cupid, the original Japanese mayonnaise. How'd you like to listen to Dott and Het Rocks with no ads? Easy. Become a patron. For just $5 a month, you get access to a private RSS feed where all the shows have no ads. $20 a month will get you that and a special Dott and Het Rocks patron mug. Sign up now at patreon.dottnetrocks.com. Hey and welcome back to Dottnet rocks. I'm Carl Franklin and I'm Richard Kebble. We have been doing this stupid show for two thousand and nineteen episodes. What is wrong with us?

2:01I don't know. It's a pattern. This, you know, yeah, can kind of anticipate that sometime next week might be another one. I say it's a stupid show. I got to admit, it got a lot smarter after episode 100. I don't know why, but I don't know why. It was really stupid before that. There was a period. I remember drawing you out of end diagram. It's like, here are the people who like a technical interview. The people that like music and comedy and so forth. The intersection between the two is small. Very small. Yeah. Okay. So I mean, two shows. Yeah. Spoiler alert, we are fixing to make a new Monday's. Oh, don't give it away, man. I just did. By the time this thing publishes, but you know, maybe, maybe knows. All right. Well, this being episode 2019, we were going to talk about what happened in 2019. And by the way, Jeffrey Sachs is with us. And I just want to invite Jeffrey to jump in if he wants to during this whole. He probably remembers

3:04to 2019 too. It was not that long ago. It wasn't that long ago. All right. So believe it or not, I have a few good news stories for 2019. Nice. Get tired of being the harbinger of Doom, that your style at all. I am. Yeah, especially last 2018 was just dark, just depressing. Yeah. So I will have some bad news too, but first some good. The world's first malaria vaccine began rolling out in Africa. Exciting. Malawi, Ghana, and Kenya began vaccinating children with RTSS, the first malaria vaccine used in a large scale public health program. Our friend Bill Gates. And Melinda Gates. We're behind that. Yeah. I don't know. They're actually our friends, but we've seen them. Yeah. I'm mad. I'm about 360,000 children per year were expected to receive it during the pilot. So cool. Yeah. Ethiopia and Eritrea's peace efforts received the Nobel peace prize. Very good. Those two have been at longer heads since their formations. Yeah. And see, it is

4:09possible. Nearly 100 captive whales in Orcas were returned to the wild Russian authorities gradually released animals from the notorious whale jail in the country's far east. By November, the remaining 50 beluga whales had been freed completing the operation. Wow. Rinse me star trek for. Algeria and Argentina eliminated malaria eliminated. How cool is that? Well, there's a bunch of ways to tackle that, right? Like you can do it by just controlling mosquitoes. Yeah. But, but also, you know, make me, you know, handing out nets like there's all kinds of tools to try and and it all works together. Ethiopia launched a massive tree planting campaign. Millions participated in the country's green legacy, reforestation effort. The Ethiopian government reported that roughly 350 million seedlings were planted in a single day in July. Part of a campaign aimed at planting billions of trees. The exact single day figure wasn't independently verified, but the enormous

5:11public participation was real. So now let's get to the depressing stuff. Well, maybe not depressing, but you know, Donald Trump is impeached for the first time. Depressing? It's up to you, I think. Hong Kong pro-democracy protests. Do not go well. What began is what's that? Do not go well. No. What began is opposition to an extradition bill grew into enormous demonstrations demanding democratic reforms and resistance to Beijing's increasing influence. The protests continued for months and frequently resulted in violent clashes with the police. Brexit crisis and Boris Johnson's rise. Richard, do you remember we were sitting in a restaurant somewhere and you looked over and you said, hey, that's Boris Johnson. Do you remember that? It was a London. We were at NDC London. And you're like, that's Boris Johnson. And I said, he was in the restaurant. I said, who? So I didn't know he was the mayor of London one point and he was kind of funny when he was the

6:14mayor of London, the old working guy. I think this was before his his further rise. So I really didn't know who he was before he was prime minister. Yeah, but you know, we did see him. Teresa may repeatedly fail to get her Brexit agreement through parliament and resigned. Boris Johnson became prime minister, negotiated revised deal and then won a huge conservative majority in December. Christ church mosque massacres bad news. This was in New Zealand. Your your home country. A white supremacist attack two mosques in Christ church on March 15th killing 51 people. But the massacre led New Zealand to rapidly tighten its gun loss. Yeah, what a concept. Noter-dame cathedral set ablaze on April 15th was an accident. Yeah. On April 15th, the world watched as the 850 year old cathedral in Paris burned its famous fire collapsed and much of the medieval roof was destroyed, although firefighters managed to save the main structure and many important artworks

7:14and relics and they rebuilt it. It is rebuilt now. I haven't seen the new one, but it's on my list. And you can still see the guy in the tower going, yeah, laughter. Boeing 737 max crashes in worldwide grounding. Was this the beginning of the end for Boeing? Oh no, this was the towards the end of the end. Towards the end of the end. When you get to the point of actually building an unsafe aircraft and ignoring the fact that your engineers are telling you it's unsafe until it starts killing people, you are very lost. Yeah. Just a couple more. The US China trade war escalates. ISIS leader Abu Bakar al-Baghdadi killed. Mass shootings in Al Paso in Dayton, Ohio. And worldwide protest movements erupt beyond Hong Kong 2019 saw enormous anti-government demonstrations in Chile, Lebanon, Iraq, Algeria, Sudan, Iran,

8:16Bolivia, and Haiti among others. So people not happy. Yeah. You done? Yeah, I guess I'm done. Let me give you one more, Grim one. Okay. Right at the end of December 2019, authorities in Wuhan China report an unusual pneumonia. Interesting. Yeah. That mud had been COVID. Might have been. Yeah. Yeah. Cause that's when I really start. All right. You ready for some space stuff? Sure. Go for it. January, the new horizon spacecraft that wanted gone to Pluto. Still going. Okay. Had its next close fly by to a Kuiper belt object. Today we call that Kuiper belt object Arakov. Although the time they called it Ultima Tule. So that's over a billion kilometers beyond Pluto. That's how fast that thing was moving. And was basically a primordial solar system object. It was literally two blobs of rock and I stuck together in a very low gravitational space. But you know, just extraordinary seeing back to the early history of the solar system that way.

9:17That's 2019. It's been seven more years. And the new horizon is still whizzing out at very high speed. Yeah. Also in January, China's Chang E4 lander lands on the far side of the moon. First time ever in the von Karman crater. It has the U2-2 rover on board. I think that means little rabbit. It will start a exploration in the previous year. China had also put up their relay satellites so they could communicate to the far side of the moon. Right. In February, back on Mars, the opportunity rover finally ends its mission after 15 years to design about the size of a golf cart design for operations for 90 days or for 15 years. Now that was in 2019, it was declared its mission over. Its last communication was actually in 2018. There had been a major dust storm in 2018 on Mars. And that was always the challenge with these rovers is that there were spirit and opportunities. They are on solar base. And so the dust covers up the solar panels. But the

10:19atmosphere of Mars occasionally would create dust levels that would clean those solar panels off. Classically, one of the at that time opportunity had its own Twitter account. And so the operators would take its telemetry data and turn it into text as a tweet from opportunity. And so opportunities last tweet was my battery is getting low and it's getting dark. And of course, the reality for a device like that is that if it doesn't have enough power to keep its electronics warm overnight, those will freeze and it will not power back up. And that's what happened. The opportunity by my in March, the crew drag and vice-space X does its first flight to the International Space Station. It's completely autonomous. It actually does its own docking. There's nobody on board. It works perfectly. It then leaves the station in D orbits and lands in the ocean and is recovered successfully. All good news. Here's the bad news. The following month, that same capsule is now going having a abort system test. This was an unmanned test tethered to the ground firing the super drake hose. And something goes terribly wrong and the vehicle is destroyed in explosion.

11:23Bummer. There was a leak in the helium system for the super drake hose that created a mass detonation. NASA actually considered it lucky. It was a huge gift. You know, you'd have that that occurred during an actual flight. It'll delay things for the rest of the year. They won't fly until 2020. Although that will actually deliver crews successfully. They'll do a new abort system use tests, including an in flight unmanned abort. Also in April, also on Mars, the Insight Lander will get its first telemetry by measuring a Mars quake to start mapping the interior of Mars. So opportunities done, but the Insight Lander is doing its thing. Also in April, the Israeli nonprofit company SpaceIL launched the bear sheet lander. The first commercial privately funded lunar lander ever flies on a Falcon 9. It originally been part of the Lunera X Prize, which started back in 2017 and 2007 and it ended in 2018. So it was actually over. It did make it to the moon, but failed on its landing, which not that unusual, because it's really hard to land on the moon. In fact,

12:28later in the year in July, India will launch their chander and two to the moon and their orbital work perfectly, but their victim lander will crash on its descent towards the South Pole. Also in April, the event horizon telescope publishes the first image of a black hole. Oh yeah. This entry to the galaxy Messier 87. I remember. So this is using a technology called very long baseline inforometry where they actually harness data from a dozen or so different radio telescopes around the world to create a sort of super radio telescope and then they composite the image into what the black hole looked like and it turned out looked exactly that the one in interstellar, which there's a series of that Christopher Nolan hired a Caltech physicist named Kim Thorn, who actually figured out what it should look like and the EHT actually validated that. And it was very low resolution if I remember correctly. Of course, but you know, you're literally talking billions of light years away. So it's miracle it works at all, but it's just that, you know, we're getting new superpowers. It may space us launches the very first batch of Starlink satellites

13:33in array of 60. Yeah. An August SpaceX does their first flight of Star Hopper. This was a funny little looking can of a thing with some legs on it. Flue just a short distance and landed again. It's the beginning of Starship, but the really important thing is this is their new Raptor engine, a methane liquid oxygen engine. And it's also full flow combustion. So that was the first time that a full flow combustion stage combustion engine had ever flown and it'll get bigger for that. And one little piece of additional Boeing news alongside killing people in their danger, 737, which admittedly is now fixed. Yeah. And I have flown on it. Boeing flies their Starliner to the International Space Station for the first time except for that part where it never got to the space station. Right. Software bugs in the system cause the vehicle to decide that it's actually farther along the mission that actually is and it tries to correct its orientation inappropriately and before they can send commands to get it to stop doing that, it burns up so much of its maneuvering fuel that it's unable to make it to the space station. They do get control

14:36of it, fix the problem with the software and successfully do your work and recover the vehicle. But that will only be the beginning of a whole series of terrible problems for Starliner that are not resulted this day. Hey, when you mentioned Starlink, you were using Starlink, right? Are you using it now? I was at that point. It would be later. I would apply for the beta and get into the beta in a couple of years after that. But are you using it right now? I'm not right now. I have, I have, it's my backup. I have fiber now to this house in the middle of nowhere, which is ridiculous. But yeah, it is ridiculous. Starlink is the back. Okay. Okay. On to the computing side of things, the whole bunch of stories around OpenAI. So February of 2019 is the release of GPT-2. And remember the mantra of OpenAI had been do AI out in the open, do it also open source, right? That's why it's called OpenAI. But they come to the conclusion that they're largest model in GPT-2, which is a 1.5 billion parameter model. Oh, how quaint. It's too powerful and that it shouldn't be shared publicly. So they only release a very tiny version of it just to show

15:38what it's, it's potential. And they are starting to reconsider what the company's all about. Part of this is they're, they're working on a paper which will publish in 2020 about the scale necessary to build, build these language models and that being enough for profit is just not feasible. It's not the way they're going to be successful. And so in March, Delery Structure, OpenAI had become what they called a capped profit company, which is funny because this is all around the time that likely they're talking to Microsoft because in June, Kevin Scott wrote an email to, to Bill Gates and, and Sachin Adela called Thoughts on OpenAI, where he talked about the fact that OpenAI and DeepMining Google Brain were well ahead of Microsoft and that it was an opportunity to invest in OpenAI and put them over onto Azure and maybe have a relationship from that, of course, will happen in July, Sam Altman and Adela will be on stage together. Will there announce an exclusive cloud partnership? I do remember that. And finally, in December, Microsoft will put a billion dollars into OpenAI. Although how much of that they'll

16:40spend buying Azure, we don't really know for sure. Yeah. Related to one of your stories early on in about China, in March, the UK publishes a report about Huawei concerns about Chinese security, you know, with Kill Switches on 5G networking and things like that. The UK has a group that's actually getting into the software that drives all of those things, but they consider the software such poor quality that they can't even verify its reliability or security. Wow. And, you know, consider it unsafe to use. That will result in the May in May, the Trump administration will literally say you can't use Huawei stuff full stop and it'll spread around all of the Western world, really. That's around the time the first 5G networks actually get deployed in the US in April. Also in April, but on April 2nd, not April 1st, probably deliberately, Google shuts down, Google plus can't do that on April 2nd. Stop funny. No. Okay. September, Visual Studio 2019

17:41with framework 4.8, but also dot net core 3.0, which is really the point where you kind of hit parody. So the features in the old framework that we wanted in the open source platform version, they were all there now and more or less complete. That's where I started using it. Yeah. I started using it at three. Yeah. And it was the point where it's like, hey, this is the end of the road. There won't be a dot net core for there won't be two versions of a dot net after this. There'll just be five, although you know the old framework. Yeah. Well, 4.8 will never go away. Yeah. And in December, they'll put out 3.1 and fixing some of the problems. Two more stories. October, Google's Sikamore processor. So it's the first real public demonstration of what you call quantum supremacy. This was one of those chandelier soaked in liquid helium style, quantum computers, very fragile. It had 54 qubits. Only 453 of them were functioning. Did a 200 second run doing a random number sampling task. They Google argued that this would take 10,000 years with a traditional computer and they finished it in 200 seconds. IBM, of course,

18:42challenged the numbers and then demonstrated various stratnon quantum strategies to get pretty good results. I think they've fastest when they got it down to a couple of days. Okay. But whatever. But it is in an old wave of advancement in quantum computing. Yeah. And last but not least, because I do think it impacts a lot of the conversations we're having today. In December, at Amazon's Reinvent conference, they talk about Graviton 2, which was custom arm processors for workloads in EC2. They're promising 40% better price performance. Up until this point, for the most part, the cloud had been regular machines bought from Intel and AMD chips and so forth, bought by the big vendors already pre-racked and so forth. But now the cloud companies, Amazon being one, but also Microsoft and Google as well. Google was already going into their tensor flows and things like that are starting to build their own hardware to optimize for cloud workloads. And that's going to change things going forward. Yeah. Those are my stories, friend. All right. Well, that brings us to better know a framework awesome or something better know

19:46something. Maybe we should call it bad or no, it's something that would be great titles. All right, man. What do you got? Okay. So for those of you paying attention to my other shows, Rocky Latka and I started with him as cohost on code it with AI, which is presented by DevExpress. And we just put when 3.8 local to the test, when 3.8 is a model and I ran it on Alama and we had done an episodes 29 and 30 with Jeff Fritz. We had done comparisons between local models running in this hardware that I have right in Alama. And Quen 3.6 was the winner by far. So I was thinking Quen 3.8 going to be great. Yeah. Not so much. Interesting. Yeah. Well,

20:48maybe it was just Rocky thought it was running out of context, memory context because Alama allocates context depending on the amount of VRAM you have. So I got 3035 or 32K based on the amount of VRAM that I have. But I don't think it was enough because it was doing the confusing things that the other model said done earlier, which is telling you that it couldn't reach the API, giving you some weird nil error and stopping. So Rocky tells me about llama.cpp. And llama.cpp is an alternative to Olama that it allows you to host models. But the cool thing about it is that it can take the compute heavy stuff and run that in VRAM. And then it can use your system RAM for the rest of it. Now, I don't know if it swaps things in and out system RAM VRAM or it just runs some things that don't

21:55require such a heavy load in system RAM. But that's it. So then there are only certain models that can run. But there's a Quen model that runs that's bigger that I'm going to try for next weeks code with AI. Well, I say next week, we're recording this on August 21st or 20th rather. I'm sorry, we're recording this on August 21st and it won't come out till September 10th. So the episode would be 43. And that would be on last week, September 2nd. So if you want to see what that does, I'm going to have a live demo of it. I'm going to try it. But llama.cpp. Know what? Learn it. Love it. Cool. Who's talking to us today, Richard? I had to go back. We're doing mathematics today. So I had to go back to a mathematics show. So I went way back 1252, which is from 2016, our friend, Seth War is talking about his library, new Mac ML in UML.

23:02Because secretly, Seth War is as charming and entertaining as he was on Channel 9. Actually, a math guy got a real, you know, got a master's degree, super smart, you know. Great to explain the whole LLM wave too. We've done some cool shows with it. So this is a comment from Kirin O'Neal, who says another scheduling library along the lines of quartz.net. We were talking about that library as well. Worth mentioning is hangfire. The advantage I've gained from using hangfires that it runs in process. It doesn't require separate window service, depending on your use cases, can be a big advantage. And it's one less thing to think about when deploying. Because yeah, pushing out stuff to count on Windows services stuff is ugly. If you're going to try and make things work properly for you. But, you know, this was a whole conversation we had with Seth about how you do these long compute workloads when you're working out things like machine learning models and so forth, where they make run for a couple of days. And you don't want to just tie up the whole machine. You want to run them in the background. Yeah. So Kirin, thank you so much for your comment and a copy of music. Go by. It's on its way to you. And if you'd like a copy of

24:04music. Go by. Write a comment on the website at dotnet rocks dot com or on the Facebook. See public show every show there. And if you comment there, I'll read in the show. We'll send you a copy of music. Go by and if you want to just go get music to code by it's at music to code by.net. All right. We've taken up so much time with this intro stuff. I think we ought to take our break now so we can have an uninterrupted talk with Jeffrey Sachs. So we'll be right back after these very important messages. Imagine standing beside a glacier swimming with turtles in the Galapagos islands or being invited to dinner in a remote village that few travelers ever see. Adventure life specializes in custom journeys and small chakrisis that bring you closer to nature, local cultures and unforgettable experiences around the world. Work one-on-one with a trip planner who uses firsthand travel experience to tailor make your trip around your interests, travel style and budget. Visit adventurelife.com today to begin your next adventure. This is Ashley Akinetti from the almost famous podcast. You ever notice you and your spouse keep

25:05saying we need to get away but you never actually plan anything. That was us until we did something fun and spontaneous. We went to resort pass dot com. There are hundreds of hotel resorts, pools and spots and private beaches that you can enjoy. You can spend the day at a luxury resort, pool, spa massage without booking an overnight stay. Listen, I may have only been 15 minutes from home but it felt like it was a whole different world. I'm thinking this is exactly what I needed. So just go to resortpass.com, choose your resort, choose your day, luxury, resort day passes start at just $25. Once you post your daycation, people are going to ask where you are. Go to resortpass.com, slash almost famous and use the promo code almost famous to get $20 off when you spend $100. That's code almost famous at resortpass.com slash almost famous. And we're back at Statenet Rocks. I'm Carl Franklin. That's Richard Campbell.

26:06And also here with us is Jeffrey Sachs. Now hi, hi. Jeffrey Sachs is the founder and sole developer of numerics.net, a set of numerical computing libraries for .net. He has been building the library since 2005 when the first version was released for .net 1.1. Over the years, numerics.net has been used for applications ranging from calculating radiation doses for cancer treatment and developing truck braking systems to building support software for atomic force microscopes. Jeffrey's work on numerical software predates.net. Jeffrey, welcome. Hi. Thank you. Thanks for having me. You're welcome. Sorry about the long intro. If you're okay with it, we can go a little longer today. But my first question is what is numerical computing? I would say numerical computing is the task of computing values to a requested precision with a requested accuracy with limited precision.

27:12And do that as fast as possible. So it's basically just math equations in code, essentially like a like a matlab kind of thing or a yeah, that's what it comes out to. Simulink. Yeah. So so this is outside the spectrum of what the built-in mathematical functions in yes, in .net could do yes, way beyond actually. Right. Vectors matrices. All right. Probably in statistics, all kinds of fun stuff. So it's all based on the fundamental stuff that's in the framework, but then you do these abstractions on top of it that give you the higher level. That's right. Yeah, constructs. Constructs. All right. Let's dust off our math chopsy if we remember any of this stuff from school. Holy man. So I mean, just seeing your comment alone, you were talking about things like radiation doses. So can you talk a bit about what kind of math we're need there? It's a great thing to figure out. Actually, I'm not sure. I just hear a lot of stories from customers. Oh, right.

28:17These are just your customers doing cool things with your software. Yeah. So that's the most from part of what I do. I hear all sorts of things. Yeah. Yeah. And you heard some variety of what I hear. Your background is in mathematics then? Yes, it is. I actually. Right. So you just taking what you've learned is that like, how could I do these things with .net? Yes, I've been doing this pretty much my whole life ever since I got my hands on the computer. And then went to university, got a degree in computing science. So specializing in the numerical stuff. And then the rear. Yeah. After, after some time of building things privately, like just for my own enjoyment, I thought, well, why not turn this into my job and do it professionally? Wow. And that's how numeric.net got started. So you remind me of a product that when I was a crescent software, we sold. I didn't write it. You know, we were I just did tech support, but it was called quick

29:19pack scientific. And so in that was a bunch of assembly language, you know, things, calls that you could call from basic from quick basic, basic seven pds, any of the visual basic that did all these kinds of things. Yes. And I wondered even back then, why do you need all this if you have the fundamentals? And it's kind of like, you know, why do you need a database if you can open and read and write files? Right. These are layers of abstraction that help you as a non programmer, maybe a scientist that dabbles in programming, do stuff easily. Well, it's not just about the layers of a structure, although that can be useful as well. It's also about the limitations that you have doing math on a computer. And still doing things correctly. Yeah. So I said, it's we got to dig into this because I think it's fascinating. What what are the we never most time we don't bump into these limitations. What are the limitations? Well, the primary limitation is

30:23you can't represent every number from real life in a computer system. Right. Right. So we have integers and then we have what we call real numbers. But those are really the double precision floating point numbers as the official name. Right. Yeah. And I think the floating points are better term just because it lets you know this is only so accurate. Yeah. So the limitation is these numbers have a very specific format. They're an integer times a power of two. Right. And you only have a certain range to work with. So every calculation you do, you basically have to are able to use only those numbers. Yeah. And that's what makes it hard. Right. Right. So it's the precision. And you mentioned precision in your definition of numerical computing that you may want less or more. But I get more. Why less? Yeah. Well, in modern day

31:26computing, there's a lot of talk about reducing the number of bits required for all elements and so forth to do their calculations and still have good results. So they'll sample them from the original 16 or 32 bits down to some times as little as four two bits even. Wow. So that's why you would use less. In general, for calculations, you would use the standard for eight number formats. Because it's not the resources aren't as tight and the computation isn't as intense. Yeah. Okay. And so the point being when you use a higher precision, it's just going to take longer for no value. Yeah. But if you use two little precision things can go terribly long as well. Right. There's a famous example of an Ariane 5 rocket in 96. I think it was. Oh, yeah. They lost the first one.

32:27They lost the first one because some value was stored as a 16 bit value for the Ariane 4 rocket, the predecessor. Right. And they converted down from 64 bits to 16, got an overflow, which sent their navigation system into some kind of error state and they have to terminate the launch. So yeah, that's a very expensive booboo. Yes, it is. Like billion dollar booboo. Yeah. And so one of the things that makes it hard is because you have that limited set of numbers, whenever you do a calculation, you have to start by approximating the number you get from real life by a number that's in your set that you can work with. So you have to round it off and that already right off right out of the gate, you start with an error and then errors propagate

33:31like every calculation you do when you add two numbers, the sum of those two numbers may not be a floating point number again. So then you get additional errors and sometimes it can explode and well, lead to explosions. Right. Quite literally. Yeah. I remember watching that actually. And it's like suddenly the engines were trying to move the rocket in ways it really wouldn't want to move and then it just sort of broke into pieces. Yeah. That's right. I had a friend he's gone now, but he was a professor of mathematics and he called himself a philosopher of math. And I he was, I guess, into things that were subtract that nobody could understand them and they had to you had to just think about them completely differently. One of those things that I think of in vain are imaginary numbers. What the hell is an imaginary number? Right. Well, an imaginary number

34:38is a trick basically to make the math work. So there's a theory where when you compute roots of a quadratic equation, you always have two roots. If you have real numbers as roots, that's not always the case. But if you allow this extension by using the square root of minus one as another dimension, then you can always get two roots. And so there's all kinds of lovely, beautiful math that just falls into place perfectly when you do that. I still don't get it. Yeah. I'm remembering enough of this stuff to realize. As soon as you said imaginary numbers, like square root of negative two, that shouldn't be possible, but there's a fixed frame. Yeah. So it's not purely mathematical. It's actually some,

35:40most of quantum physics is also expressing in terms of complex numbers. So how real that is also questionable, but that's a different discussion. Well, you know, God doesn't play dice with the universe. Would you consider yourself a philosopher of math? A little bit, but not too much. Not the philosopher king. Oh, no, philosopher. The pure math and the computer math are actually very different. It's a very different story. Oh, sure. Yeah. You don't get into extreme abstractions that only handful of people understand. But as I understand it, like mathematics and computing in general has been a series of compromises based on the limitations of the hardware available at the time. Yes, that's right. And the language. Language too. There's actually funny stories about Fortran, the first programming language, which has like originally, it didn't have

36:44if statements, it had a conditional jump, right? It didn't have the logical operators and the comparison operators are written out with dots. Dot LT dot is less than right. The reason they did that was because at the time, computers didn't have less than signs, right? They asked you said hadn't been done yet. That's right. Wow. Was it Epsidic? Oh, you've been doing earlier. Another interesting thing about Fortran, white space doesn't matter at all. Right. You can put spaces anywhere in the middle of a name of a variable anywhere. The reason they did that is because back in the day, people wrote their programs on punch cards. And there were punch guard operators who translated or like who entered the

37:46the text in punch cards, did the punching, to then feed into the computers. And what they found was that they were very sloppy with spaces in the transcriptions. And so there were lots of errors because spaces were at the time significant. And so they did away with that so that their programs could actually run. It's a work around. And speaking of languages, how does JavaScript hit you for a math language? JavaScript is kind of, it's not. You can say it. It's okay. It's not the best language for a computing. The nice thing about it is that you have numbers and they're all the same. But that's when you do numerical stuff, that's also the problem. Right. Yeah. As we found out, we've talked a little bit about complex numbers so far in the library here. I don't know that I exactly understand everything we talk about space of complex numbers because I mean, I've always

38:51thought that not a number, speaking of JavaScript was such a computer thing. But is that something relevant in complex numbers as well? It's relevant in all of numerical computing. Okay. So a complex number just has a real part and an imaginary part. Right. And once you work with those two, it's just real math again. There's nothing complex about it. So the same issues that you encounter in real numbers, you encounter in complex numbers again. Right. And so problems like infinity and not a number like these are just things you have to do content with. Yeah. And that's one of the weird things. Not a number is a special encoding in the binary floating point format. And they compare false. So any operation, any logical operation with not a number returns false, including itself. So not a number is also not a number. Well, yeah. So you have to special case

39:55all kinds of quote to work with that to trap it. One of the places where you can get not a number is if you try to compute a square root of a negative number. Right. Does that make an imaginary number? That would make an imaginary number. But it's outside of the number format. Right. I've started to feel like in this context, no number is as is problematically like nulls are problematic where you have to test for them and they're hard to test against themselves. So they just have to build a set of exception cases around it. That's true. And it kind of it's more of an annoyance than anything. If you're careful, you won't encounter them. Right. But the other challenges is of maintaining precision. If you can do that, you're good. If you can't, then you can get things like divided zero by zero, which is also not a number. Not a number. No, that's fair. So, I mean,

40:59one hand, I'm losing this library because I have a particular use case where I have a particular requirement for precision that perhaps doesn't fall into what's built into.net. Does that make sense at a reason to go into using the library? That's one possibility. We have big floats that can do enormous ranges of numbers to whatever precision you want. Basically. How big is a big float there, Jeff? A Google? I think the exponent is about 10 billion. Oh. And you can have like up to a couple of billion digits, something like that. So you could calculate with Google's? Yeah. Wow. A Google actually fits in a normal double floating point number. Really? Wow. If it's 10 to the power 100, right. It's not exact because we don't have enough bits for that. But yeah, it is within the range. Nice. Okay. But then now, and I think about, you know, why would I want to

42:02a library like this? I immediately head towards calculus. Oh, because that's hard to do without a library. That's right. Yeah. Well, more generally, the reason you would go to a library like this is that the avoiding the loss of precision is a challenge. Right. So we talked about every operation introduced potentially as non-visitional small error. Right. There's also something called catastrophic cancellation, which is you have two numbers that almost have the same magnitude. And you subtract them. So the rest, the result is very small relative to those two numbers. Oh, right. Well, your precision will plummet. Yeah. I'd turn it to a zero when it shouldn't be. Yeah. Right. Okay. And then you get into other things like, one of the biggest challenges for me that I have to be always aware of is computer math is

43:06not quite like regular math. So for example, the order of operations matters. In normal math, when you add three numbers, doesn't really matter which order you do it in. No. But say you have one plus 10 to minus 20 plus minus one. Oh, I see the problem already. So one plus 10 to the minus 20, you don't have the precision to register that little thing you're adding. Yeah. So that remains one. Yeah. And then if you subtract again, it's zero. Whereas if you do one minus one and then add it's zero plus your small number and you have a different result. Yeah. Because the first item in the list is what defines the type that it's cast to. Right. So the programming language defines the order and you have to make sure that you do things in the right order. Right. And that can sneak up on you like a simple expression A plus K minus one

44:06where A is a real number, K is an integer. That looks innocent, right? Like yeah. Can't really go wrong with that. Oh, well, what happens if K, if A is a small number and K equals one, then you're going to have A plus one, you're going to lose tons of precision there. And then subtract one again. So you've just launched lots of precision in that simple, innocent looking operation. Yeah. You ended up at zero and it was should have been. Yeah. Or something, something close, but much less accurate than what you started out with. Right. Is there such a thing as a specialized computer and language that brings a computer math closer to real math? You could say that there are symbolic computer systems, but that's really kind of cheating. So I wouldn't say for practical use, not really. Okay. I mean, that wasn't that what Fortran was about was trying to get rid of these precision problems back in the day. Fortran was to make it easy

45:13to write out equations. It's literally derived from formula translation. Right. So eliminating the floating point traps and so on is really not practical to do in a language. You have to sort of choose your algorithm. Right. Right. And so you have to sometimes you get a formula and there's something that can blow up or that can where you can lose precision. And then you have to find a different path of obtaining the same result that doesn't have that problem. Sure. And that's the biggest challenge that that's like. Yeah. And so you no longer just working on the math problem. You're working in the constraints of the computer. That's right. You have to adapt your math problem to what the computer can do for you. That's right. Yeah. And that's why why it's hard. That's why so much time is spent on getting it right.

46:17And the problem is if you don't, if you're not aware of the problems, you just write an expression and in certain test cases, it works fine and in certain other cases, it ruins everything. Yeah. And the most cases you'll test are like mid-range normal everyday things. And sure, a little work fine with that. Like, yeah, you might lose a digit or two, but nothing relevant. Yeah. But then extreme cases happen and then yeah, you might run into trouble. Disasters. Should we go down some of the paths of these different math sets you've done here? I'm just trying to figure out what a Gauss Cronrod numerical integer integrator is. You're making that up, Richard. I wish I was. I'm just reading through the SPAC here. And go on. Oh boy. Yeah. Like, if you don't know what that is, you probably don't need one. Yeah. Gauss is the famous mathematician who pioneered numerical integrations, so confusing the area under a curve using approximations. You said Gauss, right?

47:20And Gauss. Yeah. Is that what the Gaussian blur is based on? That's also him. Like anything Gauss, like the Gaussian distribution, Gaussian blur, all those things are all the same person. Right. Cover guy. Yeah. Yeah. So one way to compute errors is you compute the same result in two different ways, one more accurate than the other one. And then you compare results. Right. And Gauss Cronrod is a way of doing that with reusing some of the calculations you've done, so that you've, so to minimize extra work. So that's what that is. I appreciate that. So now I'm thinking the software and engineer me is like, okay, we measure this particular problem-spaced two different ways and then use an integrator to get what we think is the most accurate possible answer. That's right. Yeah. Smart. And it's good to have a you could try and code your way around that. You probably screw it up, use the tool. That's right.

48:22And so that particular one is used for what's called adaptive integration, which means you like if you have a curve that's really oscillating a lot, right? Like changes very quickly. If you try to approximate it with a few points, you're going to be way off. And so by computing the error, you can say, okay, we're good in this part of the function, but here we need to take deep-remby more precise. So you do a recalculation there. What's there now? And you keep doing that until your result is accurate enough. I could eat enough. I imagine FFTs, Fast Fourier transforms. I only know a lot of this stuff because you know, the Gaussian blur using Photoshop and other things like that. And the FFTs, we use in noise reduction algorithms in Adobe audition. And I've been using them ever since I've been recording with

49:23digital audio. But from what I understand about a noise reduction, it uses a Fast Fourier transform and you select the number of points. And what it does in the audio world is it acts as, let's say it's 8,000 points. It's an 8,000 band compressor, right? So you have a compressor with three bands, high low and mid or high mid and low, whatever. But if you break that up into the frequency spectrum into 8,000 bands, then it can compress each band individually using a Fast Fourier transform. I don't know how, but I know it works. Yeah. For your transform, basically turns a signal into its frequencies. And so then you can filter out high frequencies or mid or low and

50:24or like apply equalizers and then transform back to get your new signal with the high frequencies filtered out. Right. Makes a lot of sense, especially we use an 8K FFT size for noise reduction, always have, and it seemed to be the sweet spot for audio, for speaking anyway. Just one of those things. Yeah. Wow. Should we, some other areas to explore here, Jeff? What do you, what would you like to talk about? Maybe some Jebyshev polynomials? Jebyshev is one of the luckiest people in the world. This is a mathematician I presume. He's a mathematician, I'm not even sure which century he's from. But he played with a certain type of polynomials that had, that looked interesting. And so they ended up being named after him. And they, these days, they're used all over the place. Right. Because their defining property, what makes them useful is that they don't, they're very

51:31even. They oscillate, yes, but they oscillate within a narrow band and it's the narrowest possible band. And so people, you, JPL uses that for their planetary models. They will give out positions of planets and asteroids and everything as a series of Chevyshev polynomials. Because you can very accurately model basically any curve using them. And what I was saying about controlling the growth of your errors, they're extremely favorable that way. So you will get minimal error profigation, which is again, why they're so nice. Come on. I looked him up. He was a Russian from the 1800s. Yeah. That's a lot of mathematicians actually, I think. And we said a time of defining

52:32all this stuff. So again, you come back to this a few times, Jeff, that's like you're trying to work through a set of data without and get an aggregate or some kind of valuation for it, without introducing a lot of error. And these are the kinds of tools you use to try and keep the errors. And like the problem is too, as I said earlier, as soon as you bring a real world number into your computing system, you have some deviation from reality. And there's another famous map because the computer does an approximation on it. Yeah. There's another famous and rather tragic example from the Gulf War in 1991. There was a scuttle missile heading towards a U.S. base in the Haran in Saudi Arabia. And they had a patriot system for air defense, but it failed it missed. And the reason it missed was Round of Air. They were counting time by incrementing a counter every

53:35tenth of a second. Right. So after about four days, the counter was at something like 3.5 million. And then they computed the current time by multiplying that counter by 0.1. 0.1 is not a floating point number, not a binocular point number. So there's this very tiny error there of about 95 nanoseconds. But when you multiply 9 to 5 nanoseconds by 3.5 million, yeah. That's a lot. You get about 3.5 a second. And missiles fly very fast and cover a lot of ground in that time. And so the interceptor missed. And unfortunately, I think 28 people died in that attack. So it can have very serious consequences. Yeah, they're kidding. We were talking about in the news stories Boeing and the problems that they had. And I wonder if, I mean, it was

54:35obvious incompetence in a culture of not reporting the truth and trying to appease the higher ups and all that stuff and sick of incy. But I wonder if any of that stems from having bad algorithms or bad math. Do you know? I have no idea. There have been mistakes. And like in with the area and rocket and the example you gave to earlier. Sure, which I think is a really interesting one just because they, you know, changed the precision on the number and then introduced it into a different rocket that behaved differently. And that number suddenly was a huge problem. And more importantly, the software doesn't content. You get an overflow and the software just treats it as a number still. Right. Not acknowledging that no, that's not a valid number. And you should stop what you're doing. Like whatever you do from here is going to be worse, not better. The end the end cast story, which was part of that 737 max problem is a complicated one,

55:39probably none appropriate here. But yeah, it was part, it was the decision on hardware and software and how you communicated to the pilots combined that creates that situation. And again, speaking of radiation doses, there's also a story about a system to administer the radiation, where some people got bad burns. And it turned out they looked into it, maybe as the operator not experienced and all the operators very experienced. And actually that turned out to be the problem because they entered the information so quickly that the system couldn't keep up and skipped an input. And that led to overdoses. So there's all kinds of like the numerical part is one thing, but there's also the good software practices of making allowances for exceptions and treating exceptional conditions.

56:48Addicantly and correctly. Yeah, it's interesting to try and write software that contends with that kind of thing because the consequence was so serious. You know, just sort of recognizing that the data doesn't make sense. What's reasonable before it actually runs. The Therak 25 issue, that was the radiation treatment where you entered the data too quickly. It's software went into a recession. And one thing they did, I believe in that instance, is the previous version of that equipment at hardware guards. So if the radiation was too high or voltage or whichever way they measured it, it would refuse to do that. But they took those off and they replaced them with software tests. That obviously didn't work. Yeah, it's created. Other consequences, no question.

57:49So your point is well taken. Math is kind of important to get right. Well, yeah, precision. I hope that we even now actually understand fully the compromises were made to store numbers and computers efficiently. They did a pretty good job actually with the standard I triply 7, V4 format. In some ways, it's quite ingenious. The numerical, if you compare the bit value, if you look at it as a 64-bit integer, yeah, it is in the correct sort order. So you can compare the numbers as integers and have the correct sort order, except for the, the not the number cases, which are an annoyance. And there's another problem that's not obvious at first sight. Because of the format, you always

58:54have an integer times the power of two. And so there are as many floating point numbers between one and two. There are between two and four, or four and eight in sixteen. So they become scarcer and sparser. The more the larger your numbers get, and they get closer and closer together. The smaller your numbers get. And then the question is, well, what do you do close to zero? And the solution they found was, well, let's just switch from a floating point format into two times a power of two to a fixed point format. And so all the numbers with exponent zero are called sub, sub-normal numbers. And they are a fixed point format. So they are that number interpreted as an integer times, I think, is two to the power minus a thousand seventy-five.

1:00:01So that when you get to zero, you have what they call gradual underflow. You don't just cut to zero straight away. But you have like a little bit of breathing room to get there. Yeah, this is way more complicated than you think about. Most of the time, we just use numbers and don't worry about it. Sometimes you want integers and sometimes you want a floating point and you're fine. But then there, this is here, I've just looked at this huge roster of capabilities that are in the all these exception cases. How do people figure out they need your tool, Jeff? Well, most people just need some kind of calculation that where they need the level of extraction right that our library provides. So if you have to solve a set of equations, okay, you can do it all yourself and try to figure it out how to do it efficiently. Or you can use my library and just say, okay, these are a number of delegates.

1:01:02And to be clear, your library is not free. No, it's not. Yeah. And not that I have a problem with that. It makes sense. As long as you've done a ton of work and it's worth it for the foe. If you need it, you need it and you should pay for it. If you need it, you need it. That's right. That's right. So just this is literally a tool save lives and stop spacecraft from exploding. Yeah. Talk about performance a little bit. Some of the, I mean, the .NET framework has gotten more and more performant with every release. Was there ever a time when you considered dropping to assembler to do stuff because it wasn't fast enough? Well, we've been doing this for 20 years. And we've seen a few performance concepts come and go. When the task panel library was first introduced around 2010, that seemed to be the thing to speed things up.

1:02:02Turned out there was so much overhead that it was limited in what we could do with that. So one example of where I did roll my own, so to speak, is Big integer. The initial implementation that shipped with .NET 4.0 in 2010 was adequate for small sizes for this zone as soon as your numbers got large. I got slower and slower. And so we did some work there to make a faster version of that. But I really have to give it to the .NET guys in the recent years. They've done so much to bring their numerical side of things forward into what's actually now to the point where we're considering giving up our Big integer.

1:03:06And just using the .NET standard version. Do you utilize span of T? I know that's a huge win. We use it all the time. Yeah, oh wow. All the time. Because one of the advantages too is a span can be over-managed memory. But it can also be over unmanaged memory. Like anything you get from a third-party library or application. And so using span allows us to use the same code for managed memory, so vectors stored in managed memory or in native memory. And then the performance too, it's very compact. It lets you. And it's very much optimized more and more. It's actually interesting when we look at our test suite. You can see we still support .NET 4.6.2 in .NET standard. And you can see

1:04:11in our execution times of the test suite how long it takes for each addition to run. And without even changing any code, you can see a progression by like every addition is like 10% faster overall. And it's just a runtime that does that. And overhead memory overhead is also going way down. Yeah. Just wonderful. Well, it's good to see that you picked the right horse. Yeah, 20 plus year run. Yeah. I'm really very impressed with the work they've been doing. And also in the library itself, the one again complex numbers for a game out also in .NET 4. It was really like your textbook or schoolbook implementation. It was just a formula without really understanding the intricacies of how to get precise results. Even in their

1:05:19when you computed the square root of minus one, which should be purely imaginary i because that's the definition of the imaginary unit. It was off. It wasn't correct. But in the last couple of years, and now with .NET 11, they've done a ton of work to clean that up. And we're not quite ready to give up our complex number implementation, but we're seriously considering it. So again, great job. Don't have team. She passed along to Stephen Taube, those guys are for Cumberland. They are. Well, Jeffrey, thanks for hanging with us. We went a little bit over, but that's only because we had such a long intro, but it was great just going through your interview without an interruption. And this is awesome stuff. And I'm actually a little impressed with both Richard and I for keeping up with you. I think Jeff is a great explainer. You are,

1:06:23which I really appreciate. Thank you. And good luck in the future. And thanks very much. Thank you. And we will talk to you, dear listener, next time on .NET rocks. .NET rocks is brought to you by Franklin's net and produced by POP Studios, a full service audio video and post production facility located physically in New London, Connecticut, and of course, in the cloud online at pwop.com. Visit our website at dotmtrocks.com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show number one recorded in September 2002. And make sure you check out our sponsors. They keep us in business.

1:07:29Now go write some code. See you next time. Imagine standing beside a glacier, swimming with turtles in the Galapagos Islands, or being invited to dinner in a remote village that few travelers ever see. Adventure life specializes in custom journeys and small chakrisis that bring you closer to nature, local cultures, and unforgettable experiences around the world. Work one-on-one with a trip planner who uses firsthand travel experience to tailor-make your trip around your interests, travel style, and budget. Visit adventurelife.com today to begin your next adventure. This is Ashley Akinetti from the Almost Amos Podcast. You ever noticed you and your spouse keep saying, we need to get away, but you never actually plan anything. That was us until we did

1:08:31something fun and spontaneous. We went to resortpass.com. There are hundreds of hotel resorts, pools and spots and private beaches that you can enjoy. You can spend the day at a luxury resort, pool spa massage without booking an overnight stay. And listen, I may have only been like 15 minutes from home, but it felt like it was a whole different world. And I'm thinking this is exactly what I needed. So just go to resortpass.com, choose your resort, choose your day, luxury, resort day passes start at just $25. Once you post your daycation, people are going to ask where you are. Go to resortpass.com slash almost famous and use the promo code almost famous to get $20 off when you spend $100. That's code almost famous at resortpass.com slash almost famous.