Data scientists be like... | Tina Huang

0:00Propel Fitness Water with Gatorade Electrolites, Zero Sugar and Vitamins. Propel hydrates better than water to help you get the most out of your workout and get back to your best self. What propels you? Propel with Gatorade Electrolites. Cray-worthy flavors in the autumn harvest bowl, maple glaze, salmon plate, and roasted bacon-bresil side. The season's most desirable menu has returned to sweet green. Featuring falls best dressed. Make your move. Order on the sweet green app. Chronic Migraine. 15 or more headache days a month. Each lasting four hours or more can make me feel like a spectator in my own life. Botox, on a botchaline, umptox, and A prevents headaches and adults with chronic migraine. It's not for those with 14 or fewer headache days a month. It's the number one prescribed branded chronic migraine preventive treatment.

1:02Prescription Botox is injected by your doctor. Effects of Botox may spread hours to weeks after injection causing serious symptoms. A lurcher doctor right away is difficulty swallowing, speaking, breathing, eye problems, or muscle weakness can be signs of a life-threatening condition. Patients with these conditions before injection are at highest risk. Side effects may include allergic reactions, neck and injection site pain, fatigue, and headache. A lurchic reactions can include rash, welts, asthma symptoms, and dizziness. Don't receive Botox if there's a skin infection. Tell your doctor your medical history, muscle or nerve conditions, including ALS, Lugeric's disease, Myasthenia Gravis, or Lambert Eaton Syndrome, and medications, including botchaline amtoxins, as these may increase the risk of serious side effects. Why wait? Ask your doctor, visit Botox Chronic Migraine.com, or call 1-800-44 Botox to learn more. This video is sponsored by Shortharm, but more about the leader in the video. Hey, could you post some quick data on our products engagement gaps? We have a meeting with leadership in like two hours. Oh, yeah, sure! Why don't I just go to our perfectly clean already, completely there, a product engagement gaps data set?

2:03Hey, I noticed that our metrics fell today by 1%. Do you know what could be causing it? Oh, yeah, I did notice that as well. If that's quite interesting, you know, that the metric fell recently. Yeah. Oh, the analysis. Yeah, let me take another look today. Oh, I guess it's actually important. I'm like slightly exaggerating here, but like not really. So in this video, let's talk about what data scientists actually do. The caveat here, of course, is that data science is a huge field, and the job of a data scientist really varies between different industries, different companies, and even within different teams. Kenji actually did a video earlier about what data science do, and I thought it would be cool to give you guys my perspective as well, specifically as a product data scientist in a big tech manga company. I divided this video into what I call five core responsibilities, which I spend most of my time doing. All right, let's go. The first core type of work I do is long-term projects. Long-term projects generally last at least a month. It can go up to several months where even years.

3:05I also divide this into two different subcategories. The first one is exploratory projects, and the second one is automation-sash-improvement projects. Let's first talk about exploratory projects. Exploratory projects, as his name suggests, are projects that are outside the scope of what the current team, as well maybe even the company is doing. It's intended to give direction to the team or even to the company on what it is that we should be doing in the future. What kind of projects that we should be looking into? It's kind of like exploring new lands. So you're kind of like door to explore, but like with data. Some examples of exploratory projects are looking at new technologies and see how they can be useful in your team or even in your company. Generally, the whole process is that you form a hypothesis, and you kind of start testing things out, poking around a little bit, and if you find something useful, you make a presentation and tell your team about it, maybe tell your leadership about it, and if they think it's really useful as well, then usually pass it along to engineers to start implementing. Exploratory projects are definitely my favorite type of project, because I really like looking into things that I don't know about, and just kind of like discovery things,

4:06which I find really exciting. Plus, there's less people talking to you and like asking you how things are, because it's kind of like outside the scope of what your team is already doing. So people generally leave you alone, which is quite nice. It is also a great reason for someone asked you to do something ad hoc, which we'll talk about later. It can be like, well, no, because I'm doing this thing right now. It is a good excuse. Another type of long-term project is the automation-slatch-improvement-type project. Some examples of these are like sometimes your dashboards are a bit shit, and nobody knows what it actually says. Maybe your models are just not very good, especially as time passes by. Maybe the process for reporting to your leadership about the progress of your team is a huge pain in the ass. Or maybe your data quality is not great, and your data just like isn't structured very well. These type of projects, there's also a lot of variety to them, but it's really just thinking about how to improve or automate processes that the team and the company are already using. For this type of project, you're generally also working with a lot of different members of your team. Usually people with different roles. Like for example, if you're fixing up the dashboards,

5:06then you're probably working a lot with the product manager, because they're going to be the ones who are looking at these dashboards the most, except for yourself. If your models are not so great, you're probably going to be working with mostly machine learning engineers and software engineers to see how you can improve them. And for work that has to do with data quality, you're usually working with the data engineer to look at pipelines, like can be improved pipelines that are broken, as well as restructuring the data so that it's more effective to query. So the second core responsibility that a data scientist spends their time on is adhocrequst. And adhocrequst are when people ask you to do things where like find figure things out that was not part of your original plan, or when things that are unexpected happen, and you kind of have to drop everything and focus on that. From my experience in product data science, adhocrequst, like people ask you so many things, because you're kind of just generally curious. So be like, how's this product doing? We're like, you know, what's the data say about this for this thing that I did two days ago, because I'm really excited about the potential results, like things like that. And you know, it makes sense because they're curious,

6:07and then they want to see what the data says, and if they want to see what data says, they're going to come to you because you're the data scientist. What I quickly learned after working is that the best way to distinguish between what is actually important and what is not so important is by the number of times that they ask you to do it. Like if they ask you one time, and they don't ask you again, then it's probably not important, and they probably forgot about it. But they ask you like two times, three times, and it's like, okay, like this is probably important. And then you go and do it. You kind of need to have this type of filter, because if you just did everything that everybody told you to do, you would literally do nothing else, and you don't have time to work on like your long-term project, for example. The other type of ad-hoc work is when something breaks, where something just goes like terribly wrong, and everybody's like freaking out, or something like that. And then this is when you drop your long-term project, and you immediately jump on whatever it is that that is very urgent at this time. So when things do occur, it's usually very much a team effort to go and investigate and to like work things out. Now let's take a moment to talk about our sponsor today. Short from produces nonfiction guides

7:08that are so much more than just book summaries. They start off by laying the structure of the book, and the concepts that are being covered. And if you're interested, you can also look more into the details. Short from has really become my go-to, when I'm being recommended a book, for example, and I'm not sure if I actually want to buy the book or not, or read the book, so I would go and short form, look up the book, and kind of see what the concepts in the structure of the book is, and see if I'm interested. Short from covers of a variety of different genres, including philosophy, learning, and productivity, and business. And business is a genre that I've been more interested in recently, especially how to work better in a team. As a data scientist, you propel fitness water. With Gatorade electrolytes, zero sugar, and vitamins, propel hydrates better than water, to help you get the most out of your workout, and get back to your best self. What propels you? Propel with Gatorade electrolytes.

8:28Order on the sweet green app. Chronic migraine, 15 or more headache days a month, each lasting four hours or more, can make me feel like a spectator in my own life. Botox, on a botulinum toxin A, prevents headaches and adults with chronic migraine. It's not for those with 14 or fewer headache days a month. It's the number one prescribed branded chronic migraine preventive treatment. Prescription Botox is injected by your doctor. Effects of Botox may spread hours to weeks after injection, causing serious symptoms. Alerture doctor right away is difficulty swallowing, speaking, breathing, eye problems or muscle weakness can be signs of a life-threatening condition. Patients with these conditions before injection are at highest risk. Side effects may include allergic reactions, neck and injection site pain, fatigue, and headache. Allergic reactions can include rash, welts, asthma symptoms, and dizziness. Don't receive Botox if there's a skin infection. Tell your doctor your medical history, muscle or nerve conditions, including ALS Lugeric's disease, myasthenia-gravis, or Lambert Eaton syndrome, and medications, including botulinum toxins, as these may increase the risk of serious side effects. Why wait? Ask your doctor, visit BotoxCronicMigraine.com, or call 1-800-44 Botox to learn more.

9:30You spend a lot of time working in teams. Me personally, I'm not naturally a good team player. I tend to be the kind of person that would just be like, it's probably faster if I just do it myself. A recent book recommendation I got is the five dysfunctions of a team. So naturally, I went in short form, kind of looked at the concepts and stuff, and the book seemed pretty interesting, so I bought the book recently, and I'm looking forward to reading it after I finished my current book. Short from Drops, new book guys, as well as articles every single week, and subscribers can vote for what book that they want to cover next. To get five days of free unlimited access, as well as 20% off the annual subscription, you can join short form by going to this link over here, also linked in description. All right, back to the video. So the next type of work that I spend a lot of my time doing as a product data scientist in Big Tech are metrics and measurements. We're very, very data driven, so what that means is that we always need a way of measuring the success or the impact that we're delivering. And the way that you do this is by developing metrics, and data scientists are kind of like the guardians of the metrics.

10:32For the first part in creating these metrics, you think about how you can measure the success of your team in relation to your overarching company goal, because you know, whatever it is that your team is doing, it should ultimately go up to whatever it is that the company cares about. You're also thinking about what counter metrics there are, and counter metrics are metrics that you want to make sure that you're not hurting as you try to drive up your metrics. For example, say you're like on the ads team, right? And then you're like, our metric is to increase revenue. So you're like, yay, that's like put ads everywhere. Yay, look at that. We are increasing revenue. But then because you have your counter metrics, which is maybe like number of engagements, like number of people who are using your product, and you see that dropping really low, that shows that this is probably not a great thing, because even though you're driving up your metrics, you're also hurting the company as a whole. For forecasting your metrics, you're usually looking at some sort of time series, machine learning model, while incorporating the factors that you know are important in driving your metrics. And finally, monitoring is a very long-term process.

11:33You're essentially just making sure that you're working towards your goal, and like nothing is going to be really wrong. Since you're a de-guardian of the metrics, if something does happen to the metrics, you're generally the first person that people ask. And trying to figure out what it is that has happened that may have impacted our metric, usually in a negative way, I have figured out that there's actually a process to doing this, so you don't end up wasting a lot of time. The easiest and the first thing you should do is go ask the software engineers and the data engineers if they perhaps did something and shipped something to production that could have broken the metric pipelines in some fashion. And like 95% of the time, that's usually the case. And then if we still don't know what's happening, the next step is to sit there for a few days and hope that it normalizes by itself. And finally, if that still doesn't work, and that covers like, I would say 97% of occurrences of your metrics dropping, then you finally launch an investigation and look at like the different components of the metric and where it is that it's dropping and trying to pinpoint it and diagnosing what the issue is.

12:36If you have unexpected increases in your metric, though, people generally are more cool with that, so they're not going to question you as much. So the next type of work that I spend all my time doing as a data scientist is experiments. I think the importance of experiments is quite specific to big tech companies. Mostly just because of your smaller size company, you don't have the data infrastructure or war enough data for you to do more than just simple A.B. testing of different options. However, in big tech companies, we take experiments very, very seriously. The reason that we care so much is that making changes to the products were to features, test dramatic impact, just because of how big the reach is to so many different people in the world. So if you're going to be shipping something, you really have to make sure that what you're shipping is actually a good feature or a good product. Also for ML algorithms, you have to make sure that they don't have unintended side effects. Like save your ML algorithm has certain biases and you release that into the world,

13:36then that's going to hugely impact people, right? Usually in an unfair fashion towards a certain population of people. While all companies should be making sure that the algorithms that they're shipping are ones that are unbiased and are not discriminatory in any way. For big tech companies, it's almost like especially important, again, because of the wide scope and the wide reach of the product. So pretty much if you want to do anything new, then you have to run an experiment and make sure that it's doing the things that you're intending to do, making sure that it's good, making sure it's not having bad impact. Data scientists are very involved and very responsible for this experimentation process. As a data scientist, it's really important that your stats are like in tip top shape. You've got to know your statistical distributions, your power analysis, sample sizes like that, and you also have to come up with the correct experimental structures as well as analyzing and interpreting the data that comes out of the experiment's property. You're basically the person who tells the engineers and the product managers whether this thing that they did is good or not,

14:37whether you should ship that to production or not. Something that I do want to point out is that school statistics is so different from real life work statistics. Like, I thought I had a pretty decent grasp of statistics from school and, you know, like, reviewing for interviews and just like learning it myself. But then I started working and I realized that I didn't have deep enough of an understanding to be able to come up with like specific, custom experimentation structures and analyses. School statistics always gives it to you in like a very specific way, right? You kind of just like need to make sure that you applied formulas correctly. But in real life, you can't just assume things. Like, there is a normal distribution where like certain parameters are satisfied. Oftentimes, you need to think about how to transform your data into a way for you to actually do your statistical analysis. So yeah, experimental processes really important and definitely this is the area in which I spend most of my time kind of like stressing out about and like double checking and triple checking my work just to make sure that I'm doing

15:39the experimental structures and doing the analysis correctly. The next area where I spend a lot of my time on as a data scientist is on figuring out what projects to do next where we should be investing more of our time and when we should be asking for more budget and how to ask for more budget. You want to make sure that the projects are well-scoped out, make sense to leadership as well as making sure that you have a way of measuring the success using metrics. If the rest of your team and you decide that you want a budget increase, you're also working a lot with the product manager to come up with that proposal. Showing the data and analysis that demonstrates why having more budget is going to increase your impact a lot. So after you do all the scoping or projects asking for budget, things like that and you know, we start implementing. Unfortunately, you can't just like go like, okay, and then just run away. Even after you decide on the projects with the product manager and the rest of the team, you're still working really closely with the team as the projects are becoming implemented. Making sure that we're progressing the way that we should be and making adjustments because unexpected things always occur. All right, that is all I have for you today.

16:39I hope this video was helpful, giving you some idea about what it feels like and what the work of a data scientist is like, especially in the context of Big Tech. Do let me know in the comment section if you find any aspects of these work appealing or any aspects of this work, not so appealing. And I will see you guys in the next video, Relief Street. No beefy, no sitting on keyboard. Very bad. Why don't you go and sit in bed? Why don't you go sit in bed? Fall has never looked or tasted this good. Sweet Greens Fall Harvest Menu is back with seasonal favorites dressed to impress and made to be devoured. Warm roasted sweet potatoes, crisp apples, maple glazed Brussels and crave worthy flavors in the autumn harvest bowl, maple glazed salmon plate and roasted bacon Brussels side. The season's most desirable menu

17:41has returned to sweet green. Featuring falls best dressed. Make your move. Order on the sweet green app. Have you heard that McDonald's spicy chicken McNuggets made with spicy tempura and a blend of aged cayenne? Ah, back. Remember to grab a few extra napkins. Bottom up, buh buh. For a limited time at participating McDonald's. When you shop pickup at Fred Meyer, you can expect the savings you love and fresh groceries selected just for you. Our associates are committed to getting

18:41every detail right. Carefully hand picking your items, checking for quality and freshness and packing your order with care. Because bringing you fresh, quality groceries is what we do best. And right now, enjoy $30 off your first online order of $75 or more. Restrictions apply. See site for details. Fred Meyer. Fresh for everyone. Close your eyes, exhale, fill your body relax, and let go of whatever you're carrying today. Well, I'm letting go of the worry that I wouldn't get my new contacts in time for this class. I got them delivered free from 1,800 contacts. Oh my gosh, they're so fast. And breathe. Oh, sorry. I almost couldn't breathe when I saw the discount they gave me on my first order. Oh, sorry. Namaste. Visit 1-800-contacts.com today to save on your first order. 1-800-contacts. Did you know that mosquitoes have killed almost half of all people who have ever lived? Today, people are fighting back. With support from the Gates Foundation,

19:42American scientists and partners around the world have developed a new generation of bed nets that can kill up to 90% of mosquitoes exposed to them, helping cut malaria rates in half. The result? A safer, healthier future for everyone. The Gates Foundation, partners of human potential. Search Gatesfoundation.org slash mosquito to learn more.