How often do you use 'the'? Meet Zipf's law

0:00Hey, it's Paige DeSorba from Gagley Squad and this episode is supported by Rothys, the brand famous for chic, comfortable flats. My listeners know I love to start an outfit with the shoes and there's nothing better than a great flat in the fall. Their timeless, they make everything look more polished and Rothys iconic pointed flat, the point, has been reimagined to be better than ever. It now has a cushioned bouncy insol with much more arch support, so it's comfortable enough to wear all day and the point comes in so many gorgeous colors, patterns and textures. Discover the point at Rothys.com that's r-o-t-h-y-s dot com. Hi, it's Paige from Gagley Squad and this episode is sponsored by Experian Boost. Summer glow up, check. Credit glow up, even better. Boost your credit scores instantly by getting credit for bills you're already paying, your phone, utilities, rent and insurance. I wish dating kind of worked like that. Connect your bank account, add those on-time payments to your experience credit file and your FICO score updates right away. You could instantly raise your FICO score by an

1:03average of 14 points with Experian Boost. Download the Experian app for free today. Result will vary users who received a boost, improve their FICO score 8 from Experian by an average of 14 points, see AppStore or Experian dot com for details. Result will vary, not all payments are boost eligible. Users who received a boost improve their FICO score 8 from Experian by an average of 14 points, some may not see improved scores or approval odds. Not allenders use Experian credit files and not allenders use scores impacted by Experian Boost, see Experian dot com for details. Granger Girl here, I'm a neon-foggerty. This week we'll talk about Zip Slaw, a pattern that shapes how often you use your most common words and why steady researchers think it could help identify alien communication. But first, Granger Paloza the Game Show to celebrate the 20th anniversary of the podcast is in just two days, September 24th. And you want to join us for the

2:05live stream at 8 p.m. Eastern as six fabulous guests compete for Granger Glory because we will all have fun, but some of you will win books too. Check any of my social media pages for more information or just head over to my YouTube or Facebook page and RSVP for the event. The links are in the show notes and I hope to see you there. Maybe it's weird for a word person, but I love spreadsheets. I love data and charts and the other day Pat was teasing me about it over breakfast and asked what he thought was going to be a ridiculous question. Don't you want to know how many times you use the word the in different types of documents? And I was like, well, actually, there's this fascinating thing in linguistics. And I was off to the races and he just started cracking up. But there is a fascinating thing in linguistics about words like the and it's called Zips a Law. And now I have to tell you about it too. Here's the deal. In many, many human languages,

3:12the most common word appears about twice as often as the second most common word, three times as often as the third most common word and so on. Mathematicians call that being inversely proportional. Here are some real numbers so you can see how precise it is. In the brown corpus, that's this famous collection of American English texts. The word the appears about 70,000 times. That's about 7% of all the words. And the second most common word of appears about 36,000 times. That's almost exactly half as often as the and the third most common word and shows up about 29,000 times. The pattern kind of falls apart here because Anne isn't showing up as third as often as the, but the words do still fall off and a regular pattern as you go quite far down the word list. And people have come up with a more complicated mathematical formula that describes the pattern

4:16that they call the Zip Mandelbrot Law. Now that might not seem like such a big deal, except that it is not just for English. It shows up in a huge number of languages. So who discovered this pattern? Well, the law gets its name from a Harvard linguist named George Kingsley Zip, and here's my favorite part. He apparently wasn't a huge fan of math, but he wasn't the first person to notice the pattern though. A French stenographer named Jean-Baptiste Astoup had documented it by 1916, but Zip was the one who really looked at why it happens, which he included in his 1949 book with this great title, Human Behavior and the Principle of Least Effort. Yes, Zip said that speakers want to be lazy. He said, we'd love to use just one word for everything because that's easy to remember, but listeners are different. They want clarity, so they want different words

5:17for different meanings so they don't get confused. So his theory was that the distribution of words we see in language comes from the balance between those two opposing forces. Speakers trying to minimize effort, and listeners trying to maximize understanding. Zip died in 1950 when he was just 48 years old, but his law lives on, and it's gotten even more interesting since then. Those other languages I mentioned? Well, researchers looked at 50 languages in 2018, 50 from all over the world, Indo-European languages, Sino-Tibetan languages, Dravidian, Afro-Ijianic and more, and they all followed the same shape, with the middle of each word list closely following Zip's law. The rarest words dropped off faster than the law predicted, but it happened the same way in every language. Other researchers looked at other samples, including James Joyce's Ulysses,

6:19Servantes's Don Quixote in Spanish, massive French novels, finished texts with complex word forms, and even constructed languages like Esperanto, and the pattern still held. So we've got this pretty enticing universal-ish pattern, but scientists have been arguing about why it happens for 90 years. Zip's original idea about language gravitating toward the least effort it can get away with while preserving clarity is one possibility. Another theory said that it's the result of success, breathing success, technically called a preferential attachment process. The idea is that once you use a word in a document, you're actually more likely to use it again. And I won't go through them all, but another is that it's all just the result of random processes, that if you sit a monkey down at a typewriter, you can get text with a Zip fan distribution. One 2014 review looked at all the competing theories and concluded, and I'm quoting here,

7:24that quote, no prior account straightforwardly explains all the basic facts, unquote. And now here's where it gets really wild though. Humpback whale songs follow the Zip Mandelbrot law, and it's a little more controversial, but some people also think they find it in bottlenose dolphin clicks and whistles. Researchers at the SETI Institute, yes, the search for extraterrestrial intelligence, they have thought about using Zip's law as a test for complex communication. They tested it on a pulsar signal from space and found that those signals didn't follow the pattern. So they say following Zip's law is one thing they might look for to decide whether a new signal is alien communication or not. And here's another cool discovery. Remember how I said Zip's law works for words? Well, a 2015 study found that it works for phrases, too, and it actually continues much longer down the rank to list for phrases than it does for words.

8:29The authors speculated that maybe phrases rather than individual words are the fundamental units for understanding Zip's law and language. That might explain why no theory fully accounts for the word level data. Maybe we're like researchers trying to understand a building by studying individual bricks when we should be looking at whole walls. And here's a final tantalizing tidbit. The undecifered Voynich manuscript whose parchment dates from the 15th century and that people have been trying to translate for hundreds of years also follows Zip's law. Some people think that means it's a real language that we just haven't figured out yet. But other people remembering those typing monkeys and made up languages like Esperanto say it doesn't actually tell us much either way. So here's where things stand. Zip's law is one of the most robust patterns in linguistics. It appears in at least 50 languages. It works for high brow literature, newspapers, and even whale

9:30songs. It could even help us detect alien intelligence. But after about a century of research, we still don't completely understand why it exists. Maybe it's speakers being lazy. Maybe it's our tendency to be drawn to things that are already popular. Maybe it's just the way randomness shows up in language. Or maybe it's all three working together. But it's a fun thing to ponder over breakfast, especially for a day to lover like me. Your call has been forwarded to an automatic voice message system. A missed call doesn't feel like anything. No red flag, no notification screaming at you. It just disappears into your call log. And nine times out of ten, that was someone ready to hand you money. That's why today's episode is brought to you by Quo, spelled Q-U-O. The business phone system built so you never miss a call. Here's what I like about it. All your calls, texts, and voice mails live in one place. So anyone on

10:35your team can pick up a conversation, see the full history, and respond fast. No more, wait, who talked to this customer last? And if you're worried about after hours lead slipping through, Quo's optional built-in AI agent can answer questions and even book appointments while your team's offline. It's the number one rated business phone system on G2, and over 90,000 businesses already trust it to stay reachable. Money is on the line. Always say hello with Quo. Try Quo for free. Plus get 20% off your first six months when you go to Quo.com slash tech. That's q-u-o.com slash tech. 75% of women who seek menopause care don't receive it. The lying awake at 3am, wondering what's happening to your body, ends now. Midi is virtual care built for women in Perry, menopause, and menopause, with clinicians who specialize in women's health. Plus it's covered by most major insurances. Getting started is easy. You can book your first visit with a clinician right away. Midlife needs midi. Visit joinmiddi.com to book your first visit today. That's join

11:37m-i-d-i.com. Insurance coverage varies. Check with your plan for coverage. AT&T Business is built to work so you can spend less time troubleshooting and more time focusing on what actually matters. Powered by AT&T Business, built to work. Get today at business.at.com. Finally, I have a quick little family lecture from Stephanie. Good afternoon, Minion. My name is Stephanie Bryant and I'm in Kentucky. Our family lecture was but medicine for high-decoratorium cream. Kids cannot say high-decoratorium cream. Thanks, Stephanie. Hydra Cortisolm cream is definitely

12:42a tough one for kids to say. If you want to call with your family lecture story, a word or phrase you use with just your family or maybe a friend elect you use with just your friends, you can leave a message on the voicemail line or use your computer or phone to record it on speak pipe. And both of those are in the show notes. Grammar Girl is a quick and dirty tips podcast. Thanks to Nat Hoops, Morgan Christianson, Rebecca Sebastian, Steven Shepanski, Holly Hetchings, Dan Fireobind, and Maram Elnageeb who has written all 28 subway lines in New York City. And I'm Minion Fogarty, better known as Grammar Girl. That's all. Thanks for listening and don't forget to like and subscribe where you listen and join us Thursday for Grammar Paloosa The Game Show. Hey everyone, it's Adam, post of Ted's podcast, Rethinking with Adam Grant, and this episode is powered by AT&T Business. When you're building a business, momentum is

13:47everything. There's nothing worse than hitting your stride only to have a spinning loading icon bring everything to a halt. When navigating competing priorities, reliability isn't a luxury. It's a foundation. AT&T Business is built to work so you can spend less time troubleshooting and more time focusing on what actually matters. Powered by AT&T Business, built to work. Kevin Hart here in this Cashbacking season, I'm getting scary amounts of cash back on all my Halloween stuff. Do you like scary movies? Have you seen my movies? Yeah big fan. All right scary guy, I'll see you around. It's Cashbacking season with Chase Freedom. Cards issued by JP Morgan Chase Bank and a member FDIC. The GLP1 pill you've been waiting for is now on row. Yep, it's finally here with the same clinically proven ingredient, now in a pill, and now on row. It's the first FDA-approved GLP1 pill for weight loss at the lowest price available. That's one daily GLP1 pill for big results. Now on row, go to row.co-slash-listen to see if you

14:49qualify. RX only go to row.co-slash-safety for serious side effects and box warning associated with GLP1's. 500 orders a month was manageable. 5000 is madness. Embrace intelligent order fulfillment with shipstation. The only platform combining order management, warehouse workflows, inventory returns and analytics in one place. What used to take five separate tools, shipstation does in one. Automate fulfillment and rate shopping to ship up to 15 times more with up to 90% savings. Go to shipstation.com and use code audio to try shipstation free for 60 days.