Hello boys and girls ladies and germs. This is Tim Ferris. Welcome to another episode of the Tim Ferris show. My guest today is Luis Vaughan on you can find him on Twitter at Luis L.U.I.S.V.O.N.A.H.N. Luis is an entrepreneur and consulting professor at Carnegie Mellon University who is considered one of the pioneers of crowdsourcing.
He is known for conventing captures, being a MacArthur fellow and selling two companies to Google in his 20s. He is currently the co-founder and CEO of Duolingo, a language learning platform created to bring free language education to the world. With more than 500 million users, it is now the most popular language learning platform and the most downloaded education app in the world.
Luis has been named one of the brilliant 10 by popular science, one of the 50 best brains in science by discover, one of the innovators under 35 by MIT Technology Review and one of the 100 most creative people in business by fast company. Luis also won the 2018 Lemelson MIT Prize, the largest cast prize for invention in the United States.
Luis, it has been a while since we spoke. Last, it was 2016 when we had a conversation on this podcast. Welcome back. It's nice to see you. Thank you for having me, Tim.
And I thought we could start with just a few bits of trivia, a little bit of history. And why don't we start with why the Duolingo Owl is green? Would you mind just explaining that for people?
Yes, so hopefully most people have seen the Duolingo Owl. It's our mascot. First of all, it's an owl because in most Western countries, owls represent knowledge. Turns out this is just not true at all in Asia. And now that we've been expanding to Asia, people ask us why the hell are you using an owl? Owls are just kind of vicious animals. But okay, it's an owl. And it's green and it is a ridiculous reason why it's green.
When we were getting started, my co-founder Severin, who's an amazing guy, his last name is hacker, Severin hacker. That's name ever. I still think it's made up. He swears it's not. This is a computer science person whose last name is hacker. But he, you know, we were getting started and we had a design team that was working on our logo and everything. And we thought we wanted a mascot. We thought it was important to have a mascot because well, it would kind of help people learn and it would make our brand iconic, et cetera. And it that turned out to be true.
And we came up with the idea of an owl. And then, you know, we were just talking about things. And then Severin was not very involved in the design process at all. And he had some point the designers asked them, well, is there anything you care about in terms of design? He said, look, there's only one thing I care about. And it is that I hate the color green. And so we thought it would be funny if he was the co founder of a company where most of our colors were green. And that's he's had to live with that for, you know, 10 years.
All right. And we will try to not replicate too much of the first conversation, but I think a little connective tissue and context for folks would be helpful. So I'm going to try to do a quick recap of a few points. And I want you to correct anything that I get wrong and certainly add in any color that you think would be helpful.
So you were born and raised in in Guatemala. And you were raised by two parents, both medical doctors, you ended up. I think it started with receiving a Commodore 64 much to your disappointment instead of an Nintendo becoming evolved with taking things apart and learning computer science. And flash forward you in skipping a lot of steps here, but had developed a technology and a company recapture, which after about a year and a half sold to Google.
Those people have filled out a capture or recapture to prove they're not a robot or a bot. And a year and a half later, this Severin hacker, he've already mentioned, who is then a PhD student of yours was looking for a PhD project. And that led to a conversation, including the observation that number one language or I should say education was important to both you and that in your home country of Guatemala.
And education was in a sense, an obstacle to overcoming socioeconomic areas and to elevating your life because of the cost involved. So the rich were able to educate themselves and sort of perpetuate that cycle of wealth, whereas the poor were not. And there were many, many different facets of this, which led then to what would later become dualingo.
So that roughly accurate, that's exactly right. We can wrap up now we're done. All right. And I don't know when you and I can't recall offhand when we first connected. I want to say it was somewhere between 2010 and 2012. It must have been. It was around 2010 or 2011.
So right in that range. And early, early days with respect to dualingo. And the last time we spoke, you had. This was in 2016 last time we spoke on the podcast. Roughly 60 employees. So could you just give us a then and now comparison of dualingo then. Dualingo now. What the biggest differences are.
Yeah. I mean, so we've grown a lot as a company. Of course, we now have about 600 employees. So of course, 10x then employees. We used to be back then we were a private privately held company VC funded. We are now a publicly traded company. That was a big. A big milestone for us.
Another big thing that changed between 2016 and now is.
自2016年至今发生的另一个重大变化是……
We really figured out how to monetize dualingo. When we launched dualingo, the thing we cared the most about is that it was a free way to learn languages. And at first, we just weren't concerned too much about. Figure out how to make money. And the way we supported ourselves was through venture capital. We just kept on raising, you know, more and more venture capital.
And at the time, the prevailing wisdom was, hey, don't worry too much about making money. Just grow your users. And this is this is what we did. And the way we grew our users, by the way, is not through marketing. Because since we weren't making any money, we thought it was kind of a waste of our money to spend to acquire users. If we weren't making money on the other side.
So the way we grew our users was just by making the product better and better. And it grew through word of mouth. And I think what one thing that happened is around 2015, 2016. You know, we had raised a bunch of venture capital. I mean, probably around $100 million total at the time. And we still weren't monetizing at all, but we had grown to a pretty good amount of users, active users.
We raised around the funding from, at the time, was called Google Capital. And now it's called Capital G. And the partner that was on our board, Layla Sturdy, took me out to a bar. And she said, listen, you've been raising a lot of capital and your valuation keeps going up and up. At the time, I think we were valued at something like half a billion dollars or something. But right now you're just raising money from Google. And let me tell you, there is no bigger fool. Like you're not going to find another bigger fool to raise money at a higher valuation with no revenue. You got to figure out how to make money.
And so she gave me enough drinks that convinced me to promise her that next time we spoke in six months, I would have figured out how to make money. And so I went back and I like, all right, everybody, we got to figure out how to make money. And we did, but it was very important to us. We really cared that we, you know, the easiest way to make money when you're teaching somebody is just to charge them to learn.
That's kind of how education monetizes. It was very important for us that we didn't do that because the mission of our company was really, you know, related to giving access to education to everybody for free. So, and we were little constrained here in the end, we ended up finding a monetization model that worked out really well, which is this premium model that is similar to what the dating apps do or what Spotify does, which is, you don't have to pay, but you may have to see some ads at the end of a lesson.
We make money from the ads. And then if you don't like the ads, you can pay us to subscribe to a premium version of the wellingo. And then we turn off the ads and we give you a few extra perks. And that combo worked out really well. I mean, well enough that that is basically what we use to IPO, you know, it has grown to hundreds of millions of dollars of revenue.
So that worked out really well. And we're also very proud of that because of that dualingo, you know, it used to be that dualingo was the most downloaded app in the education category. It's still this. But now it's also the highest top grossing app in the education category to make more money than all other education apps. And we're very proud that you can still use dualing when entirely for free.
So even though, you know, most other education apps you have to pay to use them hours. In fact, something like 97% of our users or active users use dualingo for free. Still we make more money and all the education apps. So we're very proud of that. The monetization has was a big shift in our company.
Laila is one of my favorite people. Laila is awesome. Yeah. So we're going to come back to Laila and give her more credit.
莱拉是我最喜欢的人之一。莱拉很棒。是的,所以我们会回来赞扬莱拉,给她更多的肯定。
Do you remember what types of drinks she was feeding you? Do you have any idea or did the memory go out the window?
你记得她给你喂的是什么类型的饮料吗?你有任何想法,还是记忆已经消失了?
You know, I don't know that night. Honestly, I don't know. Yeah, no problem. It was at a bar near the dualingo office. It's about a block away from the dualingo office. Kelly's bar.
Kelly's bar. All right. It's the scene of the crime. So we're going to come back to that. I wanted to return to monetization. And I'll just mention to folks as foreshadowing, the reason I'm interested in talking about the Orchard is because, again, this will test my memory of the first conversation.
But as I recall, when you sold recapture to Google, you had maybe six, seven or eight employees, something like that. Yeah, around 10. Okay. Around 10 employees. And then for a while, I want to say maybe into the 30 to 40 employee range, you had a very flat organizational structure, so to speak. Yes, I'm doing. Effectively a lack of hierarchy at dualingo. And although you had a lot of experience with computer science, certainly, and product development and thinking about the technical side of things, building a company of 600 people was not one of them at that point. I had no, no previous knowledge of how to do this. Yeah, right. So that's why I want to come back to the Orchard.
But first, on the monetization side, what are some of the things that you tried that didn't quite work? And why did they not work just in terms of whether primary or ancillary models? Because that, that I recall as someone who is one of the pioneers of crowdsourcing, you certainly have come up with many different ideas for how you might approach this.
The first thing we tried, and this is before the Layla talk. And it was just very early on we tried this and it was a really clever idea. But ultimately, I just don't think it was a very good way to monetize. Very early on when we had started dualingo, what we said we would do is look, there were all these people that were learning a language in dualingo for free.
We thought rather than giving them ads or charging them, we would get them to do some work for us in exchange of the learning. And so the idea was going to, was the following. In fact, we built this and it worked. The idea was the following. Imagine you were a Spanish speaker who was learning English on dualingo. You know, you would learn for free, et cetera.
At the end of your lesson, we would say, hey, you just practiced food words. Do you want to practice with a real world news article about food that is in English, you're learning English, and to practice, can you help us translate that into your native language of Spanish? And, you know, we would give you like a paragraph of that or something. And then we would give many people the same paragraph and try to get them to translate it. And it turned out that actually got really good translations.
So what ended up happening is we signed a contract with CNN, where CNN would send us all their news in English. Then our users to practice their English would translate those articles into Spanish. And then, you know, we kind of combined all of the different translations into one translation. And we had all this kind of multi-step process to clean up the translation, et cetera. And then we would send back a translated version to CNN into Spanish that they could post in their Spanish site. And they would pay us for that.
So we signed a contract with CNN, we signed a contract with BuzzFeed, and this worked. And it sounds very clever. The problem is, as a monetization technique, first of all, translations of bad business to be in. Computers are getting better and better at it. And even without computers, it's just kind of a race to the bottom in terms of price. Because you can always just find somebody else to do it cheaper for you. So that was kind of one thing.
And then we just ended up realizing this was not an easy way to make too much money. The other thing is we started realizing, because that's where the money was coming from, we started realizing we were spending a lot of our thinking time and a lot of our development time when making sure the translations were accurate. Whereas to begin with what we wanted to do was make an education company. Right. And we wanted to spend most of our time teaching better rather than cleaning up translations. And so at some point, very early in the company, we weren't making very much, we were making, I mean, it's like thousands of dollars, not a lot of money.
We decided we took this hard decision. This must have been like 2013 or something. We took the hard decision to stop that. We're like, we're not going to do that, and we're not going to worry about making money. And we're just going to do venture capital for a while. Like don't worry about it. So that was one.
The other one is after the lay-la conversation, when we needed to really figure out how to make a lot of money. Now the thing is when you have a half a billion dollar valuation, you got to make a lot of money. Like when you're making zero, you can't make it so that you're only going to make a hundred thousand dollars a year. Like to justify half a billion dollar valuation, you have to making.
You can't turn it into a lifestyle business. No, no, you got to be making, you know, at least tens of millions, if not hundreds of millions of dollars in revenue, to have that. Yeah. By the way, when lay-la said you have to monetize, I went back to the company, and you know, at the time we had maybe 60, 70 employees, and I said, okay, well, we had to figure out how to make money.
And the first question everybody had was like, why? I was like, well, companies make money. That's how they support themselves. And they're like, well, we've been fine without that. You know, I didn't have to explain. You just cannot raise venture capital forever. Then it's like, okay, fine. We understand we have to make money. So how are we going to do it? And then I would say, well, you know, maybe we can put an ad at the end of our lessons, like, no, no, no, no, ads are evil. Don't put an ad.
Okay, how about we, you know, have a subscription? No, you can't charge people. You can't do that. And so it's like the whole company was basically against monetizing. And it took us about six months to finally convince everybody that monetization was not a bad thing. And that in reality, what it would allow is if we could make a lot of money, we could use that to invest it and to teach better. And people actually finally understood that, but it took a while.
And in that time, at some point, one of the first few things that changed was people were like, you know what, we could do ads. We could do ads. But they can't be the crappy ads that are all out there. Let's do a clever idea. And the clever idea was, and it's a great idea. We just never made it work. And I don't know if at some point I do want to go back to it. Can we show ads in the language that the people are learning? So if you're learning Spanish, could we show you some ads in Spanish? And I think that's a really good idea, because it's kind of like a win-win situation. Like the person who wants to actually hear the ad or see the ad, they practice. And then we make money off of it.
There's one big problem for that, which is that usually companies, that their budget doesn't work that way. So for example, in the US, and you're advertising two American consumers, you know, Coca-Cola, I'm sure has an ad in German, but they just do not want to show the German ad. That's just not what they want to show, because I know, no, no, no, no, in German, we have a very different narrative. That's just not what we want to do. We want to show the American ad. And I'm like, ah, can we show the German ad? And that just, we never got it to work. I thought it was a good idea. I never got it to work.
In the end, now we revert it to showing standard app ads that we show that are, you know, what you're supposed to get if you're in the US. You got the English ads. May I just because I'm over caffeinated, just throw something out there really quickly? Yeah. Which is, and I'm sure with your technical abilities, you could figure this out, or the advertiser could just supply the raw text, of course, for close captioning that they would use otherwise. But you could show the translated subtitles in the target language, say, of German.
That is a good idea. I probably should that at some point, there's one other complication with this, which is right now, the ads we show, the programmatic ads that come from Google and Facebook. Like we show Google and Facebook ads and Google and Facebook are the ones who serve us the ads. If we wanted to do something like this, we would have to have our own ad sales force. So we would have to go and contact companies and ask them to do that. That's doable. And probably you can make more money per ad impression if you do that.
Yeah, but it's the pain in the ass of having basically an ad sales team. Yeah. And so because our monetization, particularly subscription has done so well, we haven't felt the need to do that. I mean, at some point, maybe we'll do it, but we just haven't felt the need to do that. So I want to give you credit where credit is due on, I'm sure there are many things I could give you credit for. But one is a relentless focus and ability to refocus.
And I want to mention an article for folks. It's very short that they might find interesting. It's called the top idea in your mind. It's by Paul Graham, which relates back to the point you made of realizing that you're not in the business of tidying up the last 5% of translations that you have sourced from your company. And that's just from your users. And then in fact, you're an education company, first and foremost, right? So being able to say no to a good, even a profitable activity in order to remain focused just as you would not want to necessarily build your internal sales team, right? Because it would just sort of detract from the focus and resources.
I want to return with a question about the months it took to convince the team, the employees to get behind monetization of some form or another. How did you do that? And how much of it was like here is the mission because presumably they care about the mission of the company, but they also care about the value of their equity and want liquidity at some point. So you could just say, OK, guys, if we keep raising venture capital, like there might be secondary sales, but at some point, both the investors and employees will want to be able to cash in on at least some of the value of their equity. That's part of the reason you came to this company rather than going to work for someone else. What was the conversation or the process of persuasion?
You know, it's interesting. The first few employees that we hired at Dooling go, by the way, it's an interesting. I think a lot of us do with the fact that we have a really good company culture, but also that we're in Pittsburgh. Our employee churn is very low. So we employees stay for a very long time at Dooling go. So the majority of people that started at Dooling go, you know, 10 years ago, they're still around. And the thing about these people was when we hide the first few employees that we hired were people from Carnegie Mellon University, because I was a professor there. And I basically I just knew who the best students were and I'm like, you want to come work for us.
All these people at the time had offers for, you know, the usual suspects in your Google and Facebook and the usual suspects. And at the time, we had just raised our serious A, but we could not compete salary wise with Google or Facebook or anything like that. Now we can compete, you know, our salaries are comparable to Google or Facebook, but back then we just couldn't. So the people that took a job with Dooling go, we're taking a major pay cut sometimes like 50% pay cut like we're paying half the salary that they would have gotten on Google. They had offers at Google. They took the job and the reason they took the job was they were really, they really cared about the mission. So, you know, I like to tell Severin and I care about the mission. These first few employees were zealots. They were like, look, this is the main thing that matters to me.
And so these people for years have been kind of the guardians of the culture. And so it, it, Dooling was a very mission driven company and of course it has to do with Severin and I, but I think it also has to do with this first few employees who just really care and they're still around. And so when we were starting to monetize the discussion was really never about like, hey, you could make a lot of money off of, you know, your shares or whatever that. That just didn't resonate all that well. Yeah, for sure.
What ended up resonating the most was, hey, look, right now we have about 60 employees. We could hire way more employees if we can make a lot of money and the majority of them can be working on improving how well we teach. And that really resonated and it became true. I mean, today most of our employees are actually working on making our product better in terms of how well we teach, how engaging it is. And it is because we make hundreds of millions dollars per year. We use that to fuel the mission. So I think there was just a shift in thinking that really worked. And we believe it. I mean, I think the company now believes that I'm glad I asked these stories matter.
And I don't know the answers, by the way, if you're listening, I mean, Luis has been busy to put it mildly. We don't always catch up very frequently. So this is an excuse for us to catch up as well. Let me ask you a question that I would ask if we were just having drinks over dinner. I would not ask you to make any promises. I'm not monetization to me, but the broader question, since you mentioned computers getting better at translations, this doesn't directly relate to do a lingo. I don't think, but it might in some indirect way.
顺便说一下,我并不知道答案,如果你在听的话,我的意思是,路易斯一直很忙。我们并不经常联系。所以这是我们联系的一个借口。如果我们晚饭时在喝酒,我会问你一个问题。我不会要求你做出任何承诺。对我来说,这不是赚钱问题,但关键问题是,既然你提到了计算机在翻译方面的进步,这与Do a Lingo的直接关系似乎不大,但它可能以某种间接的方式存在联系。
And for people who don't know a bit of the history, if we look back and say recapture part of what fascinated me about recapsion is like, all right, if you, if you, and I think you did, run the kind of basic arithmetic on how many human hours were going into verifying that each individual is not in fact a bot. You began to wonder, is there a way to put, put these human hours to work for some benevolent purpose or a positive purpose? And at least one fast of that ended up being helping to digitize books. Right.
So using in human eyes and then just kind of the law of large numbers to ultimately get a good transcription of digital books that machines were having a hard time with.
因此,使用人眼对数字书籍进行校对,然后运用大数定律,最终得到一份机器难以处理的良好转录。
What are some of the most interesting things or interesting developments that you're seeing with respect to crowdsourcing versus just raw computing power or new technological pure kind of machine driven technologies?
Well, you know, computers are just getting a lot better things compared to 10 years ago, 20 years ago, AI is just a lot better. For example, at this point, it used to be that captures were mainly these distorted characters. At this point, computers are almost as good if not better than humans at reading these distorted characters. So showing some distorted characters is not a good test to distinguish humans from computers anymore.
If you've seen with recapture lately, what they show you is these kind of images of like find the traffic light or find the stop sign or find what they're doing there is their crowdsourcing things for like mapping software or self driving cars where self driving cars may take a lot of pictures for things and they may see 90% I don't know the exact fraction where they may see 90% of traffic lights and realize that it's traffic light. But for some of them, they just can't quite tell. And so what recapture is doing is it's taking some of the things where the computer is not not super sure about and is getting people to kind of tell them. So that's helping to improve the technology. So that's something I really like that application. I think it's worked out pretty well. So that's helping to improve technology.
Now in case of language translation, that's gotten pretty good. I mean, if you try it and it's kind of good enough to go and if you're going to go to Germany or something, it's kind of good enough to like if you don't know any German just kind of you know you speak to it and it speaks out something and you sort of can communicate. I know people have asked us about doing was like well, do are people still going to need to learn a language given that translation is getting better and better.
And it's one of those were like, I don't think this is going to affect us at all. There are, you know, there are these use cases like if you're going to go on a business trip for two days and you really have no interest in learning a language. Yeah, sure use Google translator use whatever whatever it is you need to use. But there's a couple of things.
First of all, no matter how perfect the translation is. There is this thing that happens that there's always going to be a delay. Even even if it's like super computing power, et cetera, because word order changes. So German is a good example in German usually the verb is at the end. So a German sentence, you know, the way you would say it kind of English is like I did something yesterday. I did it really fast. I did it with my friend and what I did was run.
And so you kind of have to wait until the verb comes out to start saying it in English because in English you would say I ran with my friend yesterday really fast. And so there's always this kind of like pause that makes conversations pretty awkward like you just this is just you can't really live your life that way. And the majority of language learning is either to learn English and actually, you know, say move to the US or actually learn English for for business reasons, et cetera, you actually want to learn English.
And then the other big chunk is hobby and you know people are still going to learn as a hobby because you know, it's the same reason why we learn, you know, all kinds of things that computers could do really well like music. So I just don't think that's going to affect us as a company terms of language learning. Yeah, I don't either. I mean, I have been incredibly impressed specifically with Google translate with respect to.
In my case, East Asian languages, because I do a still communicate a fair amount in Japanese and to a lesser extent Mandarin, but.
在我的情况下,我会说东亚语言,因为我依然经常使用日语进行交流,并且在较小程度上也会使用普通话。
The translation specifically in Japanese has improved tremendously. Yeah, it's good. Yeah, it's incredible. You want to talk about sentence construction and word order changing. Certainly in Japanese. That is that is true because it's just for people who are interested in this kind of thing, but don't know very much about it.
If you have say I eat the apple in English subject verb object in Japanese, generally speaking, it's going to be subject object verb. So I the apple E I to the school go that kind of thing. And it's it's improved a lot, but there is that delay. As you mentioned, and if if I look at the.
Let's see I'm looking at duelingo.com slash courses right now. And this may not be the best representation of use on the platform, but I'm curious to know. Who the primary users are in the pie chart. If you're just going to break down kind of the top few categories and what they're learning.
And I was surprised to look at courses for English speakers. It looks like Spanish is number one. Okay, not too surprised there. Then French. Okay, interesting. Very interesting because you know French is certainly spoken by the French diaspora and other places not just in France. It's a result of colonialism. Right. Exactly. So that's what I figured colonialism. But then you have Japanese Korean German Italian Hindi Chinese.
So Hindi Chinese. I think specifically Chinese I would have expected to be a little larger. And I'm astonished that Japanese and Korean, especially Korean because I could even chalk up Japanese to like nerdy tech folks who are Japan a files and love Japan. And so on I could I could chalk up some of that to that which may not have any accuracy to at all. It's 100% accurate. Okay, all right. Great. And then Korean though. I was like, why Korean? Why is Korean number four K pop K pop.
Okay, K pop is humongous. I mean K pop and also Korean TV shows, you know Netflix. There's just been Korean entertainment is is really gaining a lot. Right. See a squid game. That movie. Yeah, Squid games all the one that won the Oscar with a basement. I know the one you're talking about parasite parasite. So stuff like that Korean influence just keeps growing and growing. And so it is interesting more people and do a link or learning Japanese or Korean than Chinese Chinese is an interesting one Chinese is.
There's the size of the language in terms of native speakers Chinese is very large. And in this case we're talking about mandarin. Right. Mandarin Chinese. Yeah. You know, it's the largest language in terms of native speakers. And then there's a different number which is the number of people who want to learn it. We have a pretty good idea of the people who want to learn it because when we offered on the link we can see the people you know how many people actually learn it.
There's some that are flipped mean Chinese is flipped in that mandarin Chinese is flipped in that there's a lot of people who speak a natively not that many people are learning it. So, you know, I don't know exactly of the top of my head, but about 1% of our active users are learning Chinese.
Which I would have expected to be higher and in particular part of the reason why I would have expected it to be higher is because the circles that we move in I assume they're similar to circles you move in. It's kind of educated people tech people. They always have their kids learning Chinese. This is like a thing on my kids are learning Chinese etc. Yeah, this is a thing. This is a thing you go to Palo Alto. It's like every other parent has their kids and sort of bilingual nursery school or whatever.
Yeah. So you think the whole world's learning Chinese, but this is actually not the case. So anywhere in middle America, you know the school only offers Spanish like that's just to begin with right and I think there's this a number of reasons, but I think probably the biggest reason for Chinese that not as many people are learning it's just a very hard language to learn. Yeah, I think people know it and they're like it's hard.
Yeah. So, you know in the US are top languages are exactly as you said Spanish, which makes sense we're very close to Mexico and Latin America and it's just a very large language. So, French, which I think is, you know, beautiful language and everything, but I think a lot of it that one is the opposite of Chinese and that the number of people learning it versus the number of people who are native speakers. It's just a lot of interest for French given the number of native speakers. And then, you know, Chinese and Korean are pretty high up and Italian and German.
Now, we have, you know, we were asking about the types of users and doing well, I think we have two big buckets of users in my head, I classify them as kind of two big buckets, one big bucket, which is about call it 50 to 60% of our users is people learning English. These are people who are usually not in the US, there are, I mean, some of them in the US, but they're usually not in the US, they're usually, you know, in non-English speaking country and they're learning English. The reason they're learning English is they actually want to learn English, then the reason is because they want to get ahead in life. So, either they want to get into a better educational institution or they just want to get a better job.
The thing about English that is so magical is that anywhere you live, if you speak English just by speaking English, you can make more money. It's like direct. So, for example, if you used to be a waiter, now you can be a waiter at a hotel, makes more money. If you used to be, you know, I don't know, personal assistant, now you can be a personal assistant for somebody in a multinational corporation or something. So, it's just knowing English just gives you better opportunities in life and it's by far the most learned language in the world, certainly by far the most learned language on Duolingo. And those people are very committed.
And then the other big bucket of people are people who are English speakers already who are learning another language. So, think of this as a person in the US who's learning Spanish or French, etc. They're pretty different. You know, it's more of a hobby. Obviously, I'm generalizing here. There are some people obviously who need to move to Spain and they're going to learn Spanish for that. But for the majority, it's kind of more of a hobby that they're learning the Japanese or Spanish or whatever. The funny thing about these people is when you ask them, what would you do if Duolingo went away? The most common answer is I would spend more time on Instagram. They're not that committed. They're like, you know what? I like it. The reason I'm learning Spanish on Duolingo is because you see I used to play a lot of whatever. Class Royale or a lot of some game. And I was a complete waste of my time. Now I'm using Duolingo is pretty fun. And at the very least at the end of the day, I'm learning some Spanish. So they're an interesting group of people.
The other thing to say is in the US, 80% of our users were not learning a language before Duolingo. That's amazing. We're completely growing the market. That is not true in a country like Brazil where people are mainly learning English. These people were learning English before Duolingo. They'll continue learning English even if Duolingo were to go away. So it's kind of these are the two big groups of people I'd say.
Alright. So I would love to just get a fact check if you know here. I mean, maybe the math is so easy that it's almost self evident. But I recall someone saying to me, which kind of makes sense. Although I haven't done the math on it that there are more people who speak English or are learning English in China as Chinese nationals, then all of the native speakers of English in the world combined. That's probably close close to true. Yeah, I would imagine that that might be something like India as well. I mean, although you do have the British history. Yeah, it's probably close. You know, native speakers of English. Let's think about that. There's 300 million in the US. But then there's a UK to. Yeah. And you know, Australia, Canada, etc. They probably combine that another hundred millionish. Yeah. So very rough numbers here. We're talking about 400 millionish. Yeah, it seems like China's probably not quite there. Well, it could. But there's a big, big butt here. They may all claim claiming to be learning English and in reality, they may be learning English, but the learning outcomes may not be so good. Yeah. So it's like, yes, they're learning English, but you know, the maybe they know how to say hello and good morning and you know, where's the bathroom, but that's about it. So, you know, I just it's not clear that they're, you know, really good English. I do not believe that in China. There are 400 million really, really fluent native speakers. I don't believe that to be true. Yeah, I don't believe that either. And anyone who travels widely in China, I think would probably come to that conclusion. It was just not the case. Yeah.
So let's as I promised earlier, come back because I'd love to talk about some of your lessons learned key decisions in growing company of this size for the first time. And there are many ways we can tackle this. We'll talk about the personal. So you're just in terms of personal management, work life balance, if that exists lessons learned.
But let's let's also come back to the org chart and Leila, could you describe how that was done, what it ended up looking like. What were the implications and just just kind of walk us through that. And I remember in our first conversation, I told the story of, and maybe not the best example.
This since it is ultimately has been superseded by Netflix and superior technology, but blockbuster back in the day when blockbuster as a company was about to become a blockbuster brought in, believe someone from McDonald's and exec to specifically help them with org chart design. I have never designed an org chart. So could you just describe what happened and where you landed with it.
Yeah, I mean, we've gone through a bunch of different org charts. I mean, when we were from zero to call it 30 employees org chart was was very flat as in I was managing everybody. That was that and I actually think, you know, maybe I took it a little too far, but I actually think in the first for the first zero to N where N is around maybe 20 to 30 people. The best thing you can do as a CEO is be a micromanager.
I actually believe that when you are such a small company, usually you don't quite yet have product market fit. You haven't really figured everything out. You have one goal, get the product market fit. And I don't think you should be in the business of, you know, kind of like coaching people for this or that. No, just I think you just micromanaged people to like get to product market fit. I actually believe that. At some point, it really shifts even if you love micromanaging, which I love micromanaging, but I've learned not to do that anymore.
Even if you love it at some point, this is just becomes just can't do it as well. And it is in your best interest to start actually developing people. And you know, that shift should happen maybe at around 20. I don't know the exact number. That is when you should really start having kind of a couple of managers, I think splitting things up into teams.
What we did then is not like we had a pretty good idea of exactly what our org judge should look like, but what we did was we hired our first manager. That wasn't me. It's a woman that this was made, I don't know, seven years ago, something like that. It's a woman that's still with us. She's now the head of all of engineering. She used to be a director of engineering at Google. And I knew her because after selling recap, I spent some time at Google. I knew her. I really liked her. Her name is Natalie glance. She's amazing.
And she, you know, she took a much smaller job at Dooling, but she was managing, you know, I don't know. It's a larger job that she had and she took a smaller job at Dooling, I think she really believed in us and she helped us. Well, from her eyes where I learned how to manage people really much better. She actually knew how to manage people. And she really helped us, you know, kind of start having a structure of well, we have some managers. And that was. That really made a big difference starting to have teams.
And the thing that we did after that is we discovered this idea of metrics based teams, which to this day we use and I think has been a really, really good thing. I think this is not the common thing, although some companies do do it. It's not the common thing.
So at Dooling, you know, the standard thing that you would have in a company, for example, at Dooling, we have the app has a bunch of different features. One of the features could be like we have a leaderboard, for example, a leaderboard, that's a feature. In many companies, common thing to do is that you have a leaderboard team, a team that owns that big feature. So just kind of you split it up by feature. These are feature based teams.
We do not do feature based teams. We do metrics based teams. So we don't have a team that owns the leaderboard. Instead, we have a bunch of teams that own each a single metric. So for example, a metric that we have is time spent learning. What they're what they own is the number of minutes per day that the average user uses Dooling before. And it turns out that changes to the leaderboard increase or can increase or decrease time spent learning if you do the kind of the right or the wrong thing. So that team messes with the leaderboard a lot, but they don't own it. The only thing they do is they have this one metric and every quarter it has to increase. And so they just work on increasing this one metric and they run hundreds of AB tests to increase this metric.
So we discovered that soon after Natalie showed up, we discovered these metric based teams. And we started, you know, at first, our first metric based team was a retention team, which is look all you have to do is make sure that users come back every day. And that team has done all kinds of things really optimized the streak on Doolingo.
So this notion of a streak that people have. Yeah, if people have used it, what every day for seven years, I was looking at the streaks. The streak is crazy. We have well, the numbers are a little larger, but now that we're a public company, we can't reveal numbers as the latest number we've revealed is that we have one and a half million daily active users that have a streak longer than a year. Meaning they have not missed a single day in the last year or longer.
So that's a very powerful mechanic. The retention team has worked on that. So basically what we did is we discovered these metric based teams and what we do now at Dooling was when we care about a metric that we want to optimize. So a metric could be daily revenue. Or you know, whatever it is, we form a team
around that metric and that has worked up pretty well. That's kind of one big shift with us that happened. And then the other big shift that happiness. So that worked out pretty well.
At some point we had like 50 teams. I don't know, maybe a little less than we like 40 teams. And that was started to become, you know, we use the term goat rodeo. It started to become this craziness. And so what we did is we decided to pull the teams together that were similar to each other into this thing called areas.
And now for example, we have a monetization area inside the monetization area. We have a team that owns ad revenue per day. We have a team that owns subscription revenue per day. We have a team. So that. And then we have another area called the growth area where it's just growing our active users. We have a team that owns time spent learning. We have a team that owns retention. We have another team that owns new user retention, et cetera. And so we split up into areas. And that's how we're split up now.
And what has what has been really good is. And the shift for me has been, you know, it went from micromanager to kind of learning how to manage to now. What I do is I, you know, I'm a manager of managers. Well, I'm a manager of managers of managers, managers, managers. But at some point that kind of doesn't matter that much. Because you're just a manager of managers is kind of what matters. And I've learned how to start managing managers much better. I've gotten good at it.
All right. Last time we spoke, I have a number of follow ups here. Let me ask the easy one. Might not be the easy one to answer, but the easy.
好的,上次我们谈话之后,我有一些后续问题。先问一个简单的问题。可能不是容易回答的问题中的简单问题。
Simple question first, which is. Why a daily metric as opposed to a different time interval, like a weekly. For instance, in terms of revenue or something else, there's two reasons for daily stuff. One is we really big believers that.
You should be using dualing everyday. And I just basically, you don't have to use the link every day to learn a language, but we really want to build a habit. And the best habits are daily, I think. Because it's kind of hard to have these habits that are like, well, every three days I do something like this. It's just kind of hard. It's much much better to have a habit like brush my teeth just to it every day.
So we try to build habits every day. And so for us. This is one of the reasons why we look daily the other reason why we look daily is that. We run a lot of AB tests and looking at the daily timeframe allows us to move faster. So because we just don't have to imagine that our time frame was a month. We said, look, how many people do X in a month.
We have to wait a whole month to figure out what that number is. Right. Is that because you're just controlling for confounding factors if you're running. If you're changing multiple variables at the same time, it just becomes too difficult to attribute causations. So the lag time just becomes long. That's exactly right.
So whereas if you look at daily stuff, it's much faster. So for example, for revenue, we really, we look at daily, well, daily bookings to be precise is basically the amount of money that comes into dualingo. We look at that a lot and we optimize that and it really is nice because you can see, for example, if we make a change, you can see tomorrow. What happened in the change day, you can already see, yes, daily bookings increased by 10%. You can see it in one day. So those are the two reasons why we look at the daily.
I mean, we've talked about starting to do hourly. We haven't quite done that. I know one of the difficulties with our leads. There's a pretty big differences in terms of times. For example, 9 p.m. Around 9 p.m. is the most popular time to learn a language. But there are times, for example, 4 a.m. There's just not very many users. So looking at hourly makes it a little harder. But we do daily.
So before we move on, I'd love to just clarify one thing that you said and that is related to the micromanaging and whether that point is at 10 people 20 30, but it was related to being pre product market fit. Would it be safe to say that if you have product market fit, let's just say you've created a business model in a business that is generating revenue that is found its match that it would potentially make sense to develop people sooner than the 10 people 20 people or whatever the head count might be. I think that makes sense. It just hopefully you're not in a period where you're doing this kind of zero zero to 20 employees for for eight years or something. Hopefully that period only lasts one or two years. Yeah. And in that time, you probably just need to be trying to figure out product market fit. If you're going to be in that for several years, I do think it starts making sense developing people.
All right. So to let's say just hop back to our first conversation, there were two books that you mentioned as being particularly helpful or that you would recommend maybe both one was zero to one by Peter teal. And the other was the hard thing about hard things if I'm getting the title right I can never remember this title by Ben Horowitz. Yes. And I'm wondering if now flashing forward from 2016 to present you've gone from 60 to 600 or so people. A lot has changed. You've gone from private to public. Are there any other books or resources or thinkers who you have found particularly helpful or helpful at all in the last handful of years. One book that I really liked. And we make all managers read it at the Lingo is it's an oldie. It's an oldie but but a really good one is high output management by Andy Grove.
It's just it's just like a how to manual like he doesn't bother with like you know his life story or whatever. I mean it's just a little bit of that but it's really it really is just very practical. It's like look, I have found that when you're doing performance reviews, you can talk to the people before doing before giving them the review or after you should always just do it after like it's just stuff like that. It's just these little things that are just like just I have found that this is better than that. There's just a lot of you know here another one that I it's just kind of off the top of my head is like when somebody tells you that they want to quit. Drop everything you're doing. And if you don't want them to quit if you want them to quit, yeah, let them quit. But otherwise drop everything you're doing and make this your highest priority because one of the main reasons people quit is because they don't think that you think they're important. That's just very, very valuable advice like that that is just super applicable immediately to everything you're doing. So I really think I know that will cut me a lot to become a better manager.
So we are going to discuss number of facets of dualingo and new developments and so on before we get to that. I asked a number of questions last time that I think you may have new answers to so I'm going to just ask those think there are four at least in front of me and then we'll we'll zig and zag back into doing a related things. So the first is what is one of the best or most worthwhile investments you've ever made that could be an investment of money, time, energy. Pick your currency. Well, I mean, you know, the easy answer for that is is dualing although I mean, I've been at this for so long that it was not an easy investment. Yeah, I mean, obviously it has has been very good. You know, I'll give a counter intuitive one that has really helped me. I'll say it's a weird one.
I have an aging mother and I think a lot of people my age have aging parents. I used to worry a lot about her because she used to live in Guatemala and you know, Guatemala is a dangerous place and I'm like, she she's pretty old. I mean, she had me at an old age. She had she had me age 42. So she's she's 85 and she used to drive still in Guatemala and it was like this whole thing. A lot of my mental energy was I said that's not 80% of my mental energy, but it was a significant fraction of my mental energy was just worrying that she was okay. And the investment that I did is she moved in with me. And it's a weird thing. I mean, my house is large enough that this is fine. We get along well and everything. But I feel so much better about it.
I mean, I'm just like, now I'm not worried about her. She's she's there. She's kind of she's she's thriving and she's actually keeps to herself. It's not like she bothers me all that much, but I just I'm just not worried about it. Yeah. And that this has made me why I freed up. I don't know. Call it 4% of my mental energy that I was spending on that. I it's freed up and I just have more time to worry about other things.
All right. The next question is in the last five years what new belief behavior habit has most improved your life. And at the time first conversation we had on the podcast. It was starting Duolingo in Pittsburgh. So kind of a slight oblique answer to the question. But I'm wondering if you have a different answer to that or an additional answer to that now.
I think a personal one if we go on the personal one kind of a habit that has really changed my life in the last few years is strength training. I used to always kind of just do cardio to work out and that was the main thing. But one thing that happened is I was. I don't know the year maybe 2017 in January.
First of all, I don't love the winter in the northeastern United States. I am from Guatemala. I'm not used to this. This makes no sense. How could you have a place where people just die outside if they don't do anything. It's kind of ridiculous. But anyways, I don't love the winter. It was January. I was not having a kind of a great January. I also got sick. I had some sort of like a flu and it was like.
I was not doing super well and I went to the office finally after all this and I guess I'd looked I didn't look so good. I guess and. Somebody that worked at the office who's worked with us for a very long time. Dennis looked at me and he's like. At least you need Gary. I'm like, I know what you're talking about. He's like. What are you doing tomorrow at 6 30 a.m.? I'm like nothing. Gary will show up. Anyways, turns out Gary shows up.
Gary is. He was his personal trainer. Dennis's personal trainer. Gary is kind of a. He's quite a personality. He used to be the strength trainer for the Pittsburgh Steelers and then the strength trainer for University of Pittsburgh basketball team. And now he just has this roster of clients of like all these rich people from Pittsburgh who are not athletes. Certainly I was not an athlete and Gary shows up and he's like, all right. I'm going to you know I'm going to make you stronger and he gives you some weights and and then he laughs at how weak you are. He'd like laughs.
I'm like, what are you supposed to make me feel good? He's like, oh my god. You can't even do like whatever 40 pound dumbbells for bench press. What are you doing? Yeah. And so he laughs at you and I've been with him since like 2017. It has really changed my life. I used to have all kinds of aches and pains no longer.
It turns out, you know, if you start, I you know, you know a lot more about this than me, but I think if you start working out doing strength training after age, call it 35 or so it's pretty hard to build big muscle. So it's not like I've built humongous muscle, but but I just I'm just a lot healthier and certainly a lot stronger. And so this has really changed my life. So Gary, Gary saw some big fan. Wow. I guess Dennis Dennis deserves some significant thanks to Dennis is also awesome just taking the initiative. No, he really he just looked at me and I you I must have looked like hell. He just looked at me like you need Gary. And I said not going to allow right. Yeah.
Oh, how many times per week do you do resistance training now? Three times a week. Three times a week. All right. We'll give this a shot. We'll see if this question goes anywhere and then we're going to kind of zoom back to do a link and related topics.
What purchase of $100 or less has most positively impacted your life in recent memory? Easy. And of course listeners love specifics. Easy. Yeah. Easy. An electric toothbrush. I got the oral BIO. This changes your life. I mean, it just you can't go back. Yeah, you can go back and I'll tell you when I started dating my current significant other.
One of the first few things that I did was I I saw that she was kind of slomming it with like a regular toothbrush. So one of the first I just I just on Amazon ordered her my first gift to her was an electric toothbrush. And she said to this day we've been together for longer. I've given her significantly more expensive gifts to this day best gift I've ever given her. It really just changes your life. Yeah. See if you're not brushing with an electric toothbrush, you're doing it wrong.
You know, I didn't expect this to come up in this conversation, but I'm going to I'm going to plus one that and mentioned that I recently went to a new dentist. I just very highly recommended by a separate doctor I work with apparently top of the top as far as dentists go I was nervous to go because I've not been to dentists since COVID. Now this is probably flashing back.
A common thing. Yeah, common. So I was terrified that it was just going to be a complete, you know, warfield of issues, but it was mostly fine. But the point he made to me was with his recommendations is that number one, you need to use an electric toothbrush of some type because it will prevent you from applying too much pressure. And there are other benefits to it, but he's but he I apparently brush my teeth like I'm trying to take.
You know old grout off tile or something and I just destroy my gums and it's not that I'm trying to push really strongly, but I do nonetheless and I end up with bleeding gums and all of this. And since using an electric toothbrush as far as that problem goes problem solved. So is one of those where you're like just don't realize because people have told me this before but you just don't realize how much it changes life another similar one is not $100. But it is it is really life changing is that Japanese toilet.
Okay, did you get the full Japanese toilet. I did. Okay, I got I got the full total Japanese toilet this again more than a hundred bucks, but life changing. Now it does cause a problem, which is I now really it's not like can't really don't like going to the bathroom anywhere but my house. It's just a significantly better experience. All right, for people who are who may be curious. I know I have a couple of friends who are obsessed with with to toilets.
Do you have a specific model? Can you recall which which you decided on? Yes, it's called near rest, near rest, total, neo rest and it is hold on it is a specific one, total nearest NX one. Let me tell you obviously the main feature is that it uses kind of like the bidet feature where it cleans you with water, which after you start thinking about it, it is barbaric that people in the United States and most western countries do not use water to clean themselves.
It really is. I like that like the adjective of a fan. Just think about this. I mean, if you were to touch with your finger, some cow poop. Yeah. Would you not wash your hands with water? Like would you just use a towel to like no, you wouldn't because you would feel like it's still dirty. Think about that. Anyways, so it obviously has that bidet feature, but it has all kinds of other stuff, including like white noise so that you know, other people can't hear you like it just it has features. It is good.
All right. So now let me give an alternative here because I'm guessing that the the neo this particular model is I mean, it's probably at least a few thousand dollars. Yeah, it's not cheap. I will give people an alternative if they want somewhere between the Bugatti of toilets and the barbaric practice that you heard the outline. You can get an actually I was given as a gift. A washlet, total seat.
So it effectively replaces your toilet seat on your pre existing toilet and it's a few hundred dollars. You plug it into a wall and it has a lot of the same features. So I'll just mention. It does. It is pleasant to use and you know, I never thought I would have a Japanese toilet or I expose a hybrid Japanese toilet, which I have now. But when I was in Japan, when I was 15, there's some basic features.
I mean, when it is freezing and it gets gets cold in certain parts of Japan, warmer. Yeah, when it's freezing and you go in and you sit down and it's warm. Yes, that has a material impact on your day to day experience. But see, this is the thing about toilet and the toothbrush. I mean, the thing is these are things that you do every single day. It's like also spending money on a good mattress like it matters.
Like these are things that I just you just do it every day. And you know, I understand some of these things are can can be pricey, but I would not skimp too much on things that you do every day. You mentioned the mattress. I will also plus one that we can get out of the domestic objects category.
But when I was I spent a little bit of time in Panama in 2005 and got to stay with this lovely couple and the the matriarch of the family is this is mother I got I wish I can remember her name she was really an incredible woman very smart and she and she said to me it's still stuck with me she said there are two things you should invest in. Make sure you invest to the extent that you can she didn't caveat that but she said make sure you invest in your bed and your shoes because if you're not in one you're going to be in the other and I was like, Ha, that's good advice.
A lot of sense. That's really good. A lot of sense. Yeah. And you know mattresses can be expensive, but you don't have to go super nuts. Yeah. Just don't buy the $50 mattress like that. Probably you shouldn't do that. It makes a difference. And if your bed is I'll offer another hybrid option. I didn't realize that so many opinions on all these.
Yes, yes, if you're if I should say you can't afford to get a new mattress and the mattress is if he pillow top really helps get a good pillow top and it will make it will make a difference makes a huge difference. And if you can't afford that get a better pillow. This is another thing. Oh, yeah. I'll put a few I'll put a few pillow options in the in the show notes because I do have opinions on pillows.
I don't test. There's some people who are like I've tested 10 male order mattresses this year. I'm like, I don't have the time for that. No, but I will test pillows. Much easier. You know, the other thing I don't know much about I was recently reading about the history of beds. And you know, this is a few hundred years ago, not that long ago. Most people didn't have a bed. Yeah, like people slept on like bunch of clothes basically like they would just kind of pile up some clothes. And that's where they slept. That's like beds were just piles of clothes. Well, how far we've come when people read a little history, you realize a lot of improvements have been made.
All right, let's let's go from indoor to outdoor. And I have your last answer here. If you could have a gigantic billboard anywhere with anything on it. This is metaphorically speaking. Getting a message out, assuming people would understand it. Right. Let's just take that as assumed. What would it say and why it could be if you words paragraph could be an image could be anything your answer last time was it would be a billboard right across the street from Google.
Google's office here in Pittsburgh that would say we do a linger hiring and yes, we can match your salary. Do you have any new possible answers to that question? Yeah, I think the word has gotten out a lot more about doing a hiring and we get a lot of inbound applicants or I think I don't think that billboard is as necessary anymore.
By the way, we did have a billboard. This is not the one that we do, but we did have a billboard that was pretty successful in San Francisco that said that was for dual angle that said. Work in tech, owner house, move to Pittsburgh. Or something like that. That's clever. And it got us a good number of employees. It really actually worked with people are like, and it turns out if you move to Pittsburgh with a tech salary, which basically our salaries are the same as what you would have in, you know, Silicon Valley. You can own a pretty nice house. Work in tech, owner house. What was it? What was the last move to Pittsburgh? That's great.
You know what that reminds me of is that that's it really covers a lot and tells a whole story in very short form. I don't even remember the story in six words by Hemingway, four sale baby shoes never worn. It's kind of fixed me. Yeah, your billboard. Yeah, incredible. Yeah, yeah, yeah, yeah. I don't know what I would do now. I mean, probably something silly.
I mean, I don't know if you've seen all the dual angle memes with the owl trying to kidnap your family and stuff like that. I have not. Well, they're pretty popular. They're pretty popular on TikTok, basically saying, you know, the idea is that our, our, in our notifications can be pretty passive aggressive. So for example, we have a notification on the app that if you, we send you a notification every day to remind you to practice.
But if you don't practice for five days, we stopped sending notifications. At some point, it occurred to us that we could have a notification that said when we stop, we actually should tell you to stop that we're stopping as in we should be nice. And tell you, hey, we're stopping. So we, the last notification we send before we stop is, hey, these notifications don't seem to be working. I'm working to stop sending them for now.
And I come from the out and a lot of people consider this very passive aggressive. By the way, this notification gets people to come back a lot because they feel bad. They're like, oh, my God, I'm not giving up on my Spanish or my friends or whatever. So it gets people to come back quite a bit, but that notification has really gotten all kinds of people to talk more and more about our notifications.
And there's a whole persona that has evolved, not by us. It's like the internet has evolved that whole persona for our owl that it kidnaps your family if you're not learning Spanish. And there's a number of kind of taglines that they have that without the owl would say. So for example, one is Spanish or vanish or French or the trench Japanese or your niece. Like they just come up with a lot of these.
So I just have a fun billboard that just said Spanish or vanish. Sometimes it's the little things that get people talking. I'll just we won't get into it right now, but people should Google Derek Sivers SIVRS CD baby that was his company at the time. Email. It's a confirmation email that they would send out when orders came in as an automated email. So people can just search Derek Sivers CD baby email will put in the show notes as well.
我只是有一个有趣的广告牌,上面写着“说西班牙语或消失”。有时候小事情能够引起人们的讨论。我不想现在详细说,但人们可以通过谷歌搜索 Derek Sivers 或 CD baby(他当时的公司名字),可以找到一封确认订单的自动邮件。人们可以搜索Derek Sivers CD baby email,并且我们也会在节目注解中提供相关信息。
All right, coming back to present day. I have notes in front of me, one of which is related to mission and mission steering business decisions. When I think many people, including myself, sometimes hear mission, we're like, oh boy, here we go. This is going to be like a some type of thing on a wall and office space or some type of kind of lead with integrity type thing from a GE type of company. What does that mean? And does that really have an impact on people on a day to day basis? What do you mean when you talk about mission?
Yeah, so I feel similarly to you. I'm distrustful when people say that it is interesting that a dual angle. I think it really has to do with this first group of employees that we talked about before that are were like pretty big zealots on our mission that we really most every decision that we make a dual is is very mission oriented on our mission. And so that's the statement of the mission that sounds good, but and it's I'll say the statement, but I can explain it more.
So this statement is to develop the best education in the world and make it universally available. The real driving forces we want to make sure that anybody who wants to learn can do so. Now, we're not out there giving people phones or anything like that. We do assume that you have some sort of smart device. And for that assumption is that's just the world's going that way on these days you can get an Android device for 30 bucks. So it is kind of going that way we make sure that our app works on very low and Android devices, for example, but we just we really want to make sure that anybody who wants to learn a language. Well, not language is what we're expanding to teaching other things too, but anybody who wants to learn a language can do so without having to pay us.
And this matters to us. And most of the decisions we make are related to this, but one of the things that I think has really affected my thinking and certainly the company behavior is we are all really big believers that the mission we like the mission and it's great. And the big thing is that we are really big believers that that mission is actually well aligned with the long term success of our company as a business entity. So as a actual money making publicly traded company, it is aligned with a long term success. And let me kind of explain, you know, for example, we could say tomorrow, you know what, screw it, you have to pay to use tooling.
My guess is we would not know my guess I'm pretty sure we would make a lot more money if we did that. Our number of users would go down, but good number of them would also pay us and we would make a lot more money and next quarter would be an amazing quarter for us like we would. It would be an amazing quarter for us. I don't know the off the top of my head, but our revenue probably would 10x some crazy number in like one quarter would just go nuts.
However, what that would do is it would completely stifle our growth because the way Doolingo has grown is through word of mouth. We have all these people who use us for free, they don't pay us. We make some money off them from the ads, but you know, most of our money comes from our description. But what's really useful about all these people that use us is they use us, they have a good experience and they tell their friends. And the vast majority of our users, I mean, really overwhelming majority of our users when you ask them how you know how do you find that about Doolingo.
It's like I have friend told me or like my significant other told me or whatever it's basically word of mouth. And I really do believe that having a really great free user experience, it's what's best for the business in the long run. Again, not next quarter, but if you take the long view, which we are at Doolingo, I mean, we've been at this for 10 years, I'm going to stick around with Doolingo probably forever.
I mean, I really love working here, et cetera. So my view now is a long term view. I mean, I'm here thinking about the next 10 years, the next 20 years, et cetera.
And I really think that if we do the right thing for our users in terms of like actually being able to learn a language really well and without having to pay us. That is actually what's going to optimize our revenue in the long term. So I believe that I think most everybody in the company believes that.
And I think that that alignment has made it so that a lot of decisions are a lot easier because we do hire a bunch of people who are like. What the hell are you talking about mission? Like, you know, this is people who went to like whatever business school and everything's like, well, I'm just here to make money.
So we do hire some such people, but I think once they understand that we really it's not like we don't want to make money. It's not like we, you know, we really are big believers that there's real long term alignment here.
I think that really that aligns everybody. What other ways does the long view impact decision making? So maybe you could give another example or two as to how that informs not just the culture, but, but how you make decisions.
And you already mentioned one, which would be the decision to keep it free for the majority of users versus force a subscription upon everyone and then have a short term game for all of these long term sacrifices.
Can you think of any other ways in which taking the longer view affects decision making? I'll tell you another one that has worked out really well for us, but it has to do.
You know, one of the things about taking advice from certain companies that I've learned is taking individual pieces of advice sometimes doesn't work because they only works for that company in the context of that company. Right.
Totally. We have a very specific context, which is because again, because our culture is good because Pittsburgh is not quite Silicon Valley, et cetera. We have low employee churn.
If you see a Silicon Valley company a lot of times, I mean, by the way, Silicon Valley is amazing. These companies operate really well. But one of the things that is common in Silicon Valley is people don't stay at jobs for very long. And companies have become optimized for that.
They are optimized for that. That's just that they're very good at operating excellent companies when the assumption is that people are going to stay there for two, three years. And they get really good at that at dueling.
People stay for much longer than that people stay for pretty long time. And where we take the long view in doing things is in hiring. We are really picky with hiring.
人们逗留的时间比人们逗留的时间长得多。我们注重长远发展的地方在于招聘。我们在招聘方面非常挑剔。
And it's because people stick around for a long time that it really matters. So it matters that you know, if they're going to be here for eight years, it matters. It's okay to wait a couple of months to find somebody that is probably going to be 20% better for this job, but not because overall, yeah, totally.
However, if you are a company where you know, you have pretty high churn. Probably this is not as worthwhile. So it's just for us, this has been a pretty good thing for you know, taking the long view in hiring has been good for us.
So I'd like to ask more about that specifically. So I am trying to get better at hiring myself. And had some maybe beginners luck where I did really well in the beginning had people stick around for a long time, but have had a couple of misses in the last few years. It's been hard.
Yeah, it does happen. And it's been it's been hard for me to really deconstruct the reasons for that in a in a postgame analysis, although I do think that there were probably.
是的,这种情况确实发生了。但是对我来说很难在赛后分析中真正揭示其原因,尽管我认为可能有些原因存在。
There was some unnecessary rushing to the hiring process. So we talked in our last conversation about hiring and you mentioned that you've learned how expensive hiring mistakes can be no doubt you've learned that.
Yeah, more than once. So you guys are very strict in your interviews and you described the process. I'd love to see if this is changed. And if it is different for non technical roles.
In 2016, it was number one resume screen, good GPA internships and so on. Number two, two phone screens, which includes problem solving. Number three, bringing remaining candidates on site where they have four different interviews, some of which involve more problem solving as well as paired programming during which they fix a bug.
So this seems to be specific for technical candidates. I'm wondering if you have anything to add to this. If you could maybe elaborate on on what taking your time and looking for that extra 20% looks like. And I suppose I'm most interested in in non technical roles since I am not hiring for for real technical capabilities, but anything you have to say on any of those points.
Yeah, that interview is pretty similar to what we do now. I think, you know, some the just the actual questions may have changed a little bit what we look for in resumes may have changed a little bit, but it is the overall is pretty similar and that is for technical technical hires.
It really depends, you know, that's another thing about hiring. I mean, it really depends on what your what function you're hiring for. So for example, designers. What matters a lot is their portfolio. And the portfolio is this magical thing. Portfolios are amazing because within 30 seconds by looking at a portfolio, you can tell a lot. It's just you don't even have to do all that much interview or anything. They just look at a portfolio.
Yeah, it's good. And the other beautiful thing about portfolios, by the way, is that it's so nice because they're really good for diversity in that you don't have to know this person's name. You don't have to know what they look like. You don't have to know anything. You don't it's just the workspicks for itself. Yeah. So this is a beautiful thing about design, but it's very rare. Usually you don't most fields don't have a thing like the portfolio where you just look at it and you're like this is it.
So, you know, I think outside of these you have to get really good at interviewing and you got to figure out there's a few things that really work out. One is work product. So whatever it is that they're supposed to do you have them do some of that it could be that they do it offline as in like they don't have to do it in front of you. It could be like look in order to apply you have to you have to write this thing or whatever it is you can take your time just send it to us as part of the application work product is really good.
I think that really tells you whether you and it has to be pretty close to what they're supposed to do when when you hire them. So that's that's really good. I think there's another one which is just kind of general intelligence stuff which you may or may not want to do. But there are some pretty simple kind of there's this like three question IQ test that you can Google for.
Which is just you know it's not going to give you an exact precise you know with just three questions you're not going to be able to get an exact IQ. But basically you know people can solve this all three of these questions they're pretty good with cognitive abilities. You may want to do that. And then the other one is just kind of more of these behavioral interviews that you ask these questions that you really do it does make a difference for non technical stuff.
I mean things like talk about a failure. Tell me about a time you failed and the types of things you're looking for say do they recognize that they have failed like the wrong answer the horrible ones is this like failure not failure. Like where they tell you like I just care too much. Yeah or like it starts looking like a failure but actually it was a home run.
You're like no no no no no like a real failure. And so you want to understand that they understand when they failed. The other thing you want to hear there is. Did they blame others or themselves. It's very telling when they tell you about a failure it's a real failure but they like but you know what I knew what to do the whole time.
It's just my manager was dumb or whatever like that that's a very bad sign that because look everybody's failed and there's at least one instance and usually many where it really was your fault. I want to hear about that not about how you always knew all along and you always write and it's just your manager sucked or somebody else because when you do end up hiring them you know these are the type of people that would probably blame everybody else for their failures and you don't want that.
So there are these kind of behavioral interviews that I think really work out pretty well. So we do we do apply all of those but you know one of the things that I have learned a lot is and we we employ it at Dooling go. When you are the hiring manager so the person that's actually going to work with them. And usually it's because you you're hiring somebody because you'd have a need you're like well I need a designer on my team or I need whatever it is I need this on my team and I have the need.
You're extremely biased towards hiring yes you're like look I just need to get this done I just want to get it so one of the things we do is ultimately a Dooling go the decisions are not by the hiring manager. I'm a manager can have a lot of input but there's somebody else that is like and a lot of times that's me who's like I understand you're in deep pain right now and you need more you know another person but this person does not pass our bar. And it's not that I'm that good at the bar it's just that I'm a different person so I made the same mistake when I'm hiring from you know an executive or something I have a bias towards hiring because I'm like man I don't I don't want to spend more time looking for you know this whatever head of whatever by the way we did this for a CFO or a chief financial officer we took a long time and at some point I was gave up I'm like man I'll just take whoever like it doesn't matter.
We didn't do that I was lucky I was lucky that we ended up I mean I had spent about a year looking for our CFO yeah it's a long time. Thank God we waited I mean we are CFO mask group is amazing thank God we waited but you know in the middle thereafter of I don't know nine months or so I started faltering I started being like just take whoever. So it's not that I'm that good at holding our bar but I'm very good at holding our bar when it's not me yeah that makes sense to me that's a big big piece of advice have somebody else hold the bar.
Okay so this is actually a very opportune time to segway so the two things I'm going to cover well I'm going to cover that's not true the royal I that you are going to cover I'd love to explore our number one. How did you decide to go public when did you decide it was the right time to pursue that and the CFO may. They may somehow be involved with that but the question after that is going to be what's next right new products you alluded to something beyond language learning so certainly I would love to hear more about that but let's begin with the going public question and just how you. Thought about that made decisions around it and then from there into new products new revenue streams things like that.
Okay so first of all the idea of going public for example last time we talked you know I don't know exactly what was going on six years ago in my head but the idea of public going public I think around then just seems so foreign so like that's never going to happen. I wasn't just not thinking about that once we figured out a revenue and we figured out that we could make a lot of revenue the first time this really was discussed was we were having a meeting with one of our board members his name is being Gordon and his being is like a legendary guy in Silicon Valley very famous very famous says a lot of crazy stuff a lot of really good crazy stuff but also sometimes it's like just really crazy stuff.
And you know he was just sitting there and we're talking about a revenue and he said look you're going to make this much revenue next year you're going to make that much revenue that you're after that and then you're going to go public and we were like what. I could just completely really you can do that you could just go public that's like a thing and he's like yeah why not. And that's kind of where it clicked and that's we know we're like okay maybe and we started thinking about it and then we when we got to the point we were basically breaking even which happened you know being talked about that maybe the year after that we were already breaking where we're roughly breaking even I mean it was like it was not exactly breaking pretty close to breaking even we're like maybe we should maybe we should consider going public and the first step if you're going to go public the first step you need is you need a CFO like this is can.
Maybe you can go public without a CFO but seems like a really bad idea don't do that. Yeah, generally ill advised. Yes so step one is you need a CFO that actually knows how to take you public. A lot of people give you advice that you should hire a CFO that has taken a company public. That is hard for many reasons for one there's just not very many such people. Secondly, CFOs turn out to be very highly paid individuals and when you go public you get paid a lot as a CFO so what ends up happening is a lot of these people who are CFOs who've taken a company public are just very rich. And they don't want to work for you anymore they're just like whatever I have my house in the Hamptons and that's where I am and I'm just not going to go work with you.
So, it's just a very limited pool of people who have taken a company public once and so there's that and then we did interview some some people and you know they have good experience what we ended up doing for CFOs not take somebody who had taken a company public but you know obviously you finance et cetera you came from Goldman Sachs but we maximize for kind of intelligence and grip. And that was a really good move for us and so that worked out for us quick question so intelligence I can see how you test for intelligence how do you assess grip.
It's just you know I think a lot of it has to do from references I mean this one map that we hired we knew a few people that knew him. There's a reference we're just and they were true the thing is they were truly they'll tell you like oh Matt's group is like the hardest working most I know so most honest guy I've ever met and I'm like to me the words honest and CFO just kind of don't go together that much I don't I don't know if I just had I was biased against it but then we hired a guy that is like the most honest guy like we call him the most sometimes some of us call him the Boy Scout because it's just like guys just kind of a good guy and so we hired a mat.
And you know by the way the other thing with Matt is Matt was going to be based in New York right before COVID we hired Matt. I don't know his first day was maybe a month before COVID before like it's probably like February of 2020 or something like that he was going to be based in New York but we're headquartered in Pittsburgh we have an office in New York with about a hundred people he was going to be in that office but he knew he had to travel out to Pittsburgh. On his first day he shows up with him I don't know he'll have like ten slides a deck with his travel plans for the next two years he had pie charts of fraction of time he was going to spend like in Pittsburgh in New York on the plane like it's all this stuff that he had done like a massive analysis like what a consultant would do for some sort of analysis he's like this is my travel schedule for the next two years and I'm like wow I would have never done anything like that. I've never done anything like this but I see we hired the right person he must have spent hours and hours on this and then poor guy.
COVID came and then that travel plan went down the drain. I just didn't have to travel. But anyways you hire a CFO that's kind of the first thing and then and then. For me I think what really mattered was is our revenue and this seems to be like what matters the most is our revenue predictable is our our finances predictable if your finances are not predictable you should not go public you're going to have a real bad time. So you know you want to know that your finances are predictable and by predictable I mean I've learned you need to be able to predict your finances with you know a year out and be like 3% away.
So this is pretty good prediction that you have to pretty good forecasting so you have to do that and you have to have a path to real profitability that's the other thing. If you don't have either of those I would say I would not go public your readers have a real bad time. The other thing to say about having gone public I'm very happy we did.
The company operates better I think of it as like you know when nobody visits you in your house you can't have to clean all that much. It's not a very clean house if nobody ever visits you you're like you can't live like a slob it's like but if there's going to be a visit coming you're like oh shit I got a clean up once a quarter. Well it's not once a quarter because now we're like constant scrutiny but but but basically I think what happened is once the process of going public forced us to do all these things much better. You know these processes that were like they were just not well set up as a process or like we may not even have had a process and it's like no we're going to go public now now we have to have a process now so at this point our house is pretty clean because we get visits all the time.
And I think that the thing about living in a clean house is you feel good about it yeah you know just kind of better and so I'm very happy that we weren't public. Yeah you know I just want to say also to folks who may be listening and have a small business and they're like I'm never going public this isn't relevant to me what I would say is. And there are there books that you can read that I think are.
Are interesting for as a thought exercise at the very least there's a book called built to sell I think the author's name is John Warlow. And if someone wants to improve their own business let's just say it's a smaller business with five employees. Pretend like you're going to sell it right pretend like you're going to sell it in 12 months and you will go through you renovate the bathroom is what you're going to renovate the bathroom you're going to realize like oh yeah the duct tape on that part of the plumbing no no one's going to buy that that's not going to pass the inspection so we're going to have to fix that. And you can go through a very similar house cleaning process where you build systems you remove yourself as a bottleneck for certain decisions and so on that can be hugely hugely beneficial so I will just mention the parallels there.
What have been the biggest challenges of being public. Yeah or being a publicly I don't want to make it too abstract I mean there are the challenges of being a publicly traded company I'm just wondering being the CEO of a publicly traded company. A lot of people assume that my job was going to change a lot it hasn't in the other things we worked really hard to make sure that our culture didn't change a lot if you ask a random which I have if you ask a random doing employee how is it feel before and after they'll tell you very little changed. There's a couple of things you know you can't really give forward looking statements as much read where you can say like well next year we're going to make this much revenue or whatever just don't do that. Stuff like that but it's minor I would say and I think my job in particular the only real change is now I have to spend what I have a deal with our head of investment relations I only I have to spend 12 hours a quarter on talking to investors are analysts and so that's that the reason I have this deal is you can spend 100% of your time doing that and for sure.
Love talking to investors and analysts love talking to them but it just you can't run a company when you spend 100% of your time talking to them you just actually can't do that. So I give her 12 hours a quarter how did you settle on 12 hours I'm just so curious is that one hour a week no we negotiate it we negotiate it I started with five she's like you can't do that and she's like how about 20 I'm like in the end we just settled on on 12. That's about the biggest change I don't think there's there's much else I guess I have to write a you don't have to do this by the way I just learned I was very clueless before going public about what it really meant to be a publicly traded company one of the things I recently learned is you don't have to have earnings calls or shareholder letters you don't have to do this.
I thought you did and we do them we do them every quarter turns out you don't this is not a legal requirement of any kind but it's probably a good idea to do it so anyways the other thing I have to write a shareholder letter we do shareholder letters and I have to do that every quarter but other than that it's been it's been good I mean we're fortunate that you know stock market has been pretty choppy we're fortunate that our stock has held up pretty well there are some companies for whom you know they're like down 90% from when they went public or something. That must be very tough. We're fortunate that that's just not the case.
Yeah that's when the activists start circling in the waters. Yeah yeah you know it's fortunate but I also think the thing we make sure of is we both had a highly forecastable business and a good path to good profitability. I think if you have those two you're going to do okay I think it's you know last year in particular you saw a number of companies that were. You know either not very forecastable or just kind of did not care at all about profitability and I think in today's climate that's just not so great. Yeah yeah that's sort of tying an arm behind your back when you get into the boxing ring you might win you might but it does make it a little more challenging.
And you know what else does not help I don't know if you've watched this show we crashed which by the way say next line show yeah the story of we work yeah I saw episode one actually it's out by Southwest. I love that show but you know one of the things that does not help is investment bankers will tell you what you want to hear. You're they're supposed to be like you know there to coach you but they make money if you go public. And they make a ton of money if you go public so it is common that they'll tell you yeah of course you're ready to go public course yeah and I just don't think they are as brutally honest as they should be. In that respect yeah well you know it's like the old buffed is done ask a barb if you need a haircut.
Being aware of the incentives involved is important. Yeah which is also you know it can be true for reward it can be true for punishment or pain as you're mentioning earlier with hiring right if you are in the hot seat and you are facing the consequences of a lack of a hot. You are going to be strongly incentivized and you might use motivated reasoning to push for a faster higher than you should. Yes and this is the crazy thing about that is that even if you're not you may not even be malicious like I'm not saying that it's just it's just this incentives work that way like. I'm not saying that investment bankers are even malicious or anything they're just like in their way of viewing the world they're like yeah it's good for you to go public. Yeah it's just human nature yeah all right
Looking towards the future new products revenue streams. What's coming yeah we're very excited by this so historically doing has been language learning. You know that's the thing we teach languages by now we're we're the largest way to learn languages in the world we're you know with most popular than. And any other way to learn languages including public school systems so that's awesome but you know our mission has always been more general just kind of we really like all of education we started with language learning so we're starting to expand to other things.
We've launched already a literacy app that's different from language learning in that it's your native language you just need to learn how to read. So this is an app for kids right now it's only in English eyes doling go ABC that teaches kids. When we were working on it we thought kids three to six turns out kids get there's like a downward pressure for learning how to read earlier and earlier so some of our top users for the app are actually do. So there's these kids that are age to that are learning how to read on our app which I'm like I don't even think I knew I walk when I was to I don't know but somehow these kids are learning how to read at age to with with the app so we started with that we're working on a math app.
That is going to come out this year where the ideas you're going to be able to learn some math we're starting with elementary school math. So like I'm not first grade so not counting but we're starting with kind of basic arithmetic and fractions and stuff like that one of the interesting things about that is that. Well the reason we're starting with elementary school math is because that's kind of where you start seeing a divergence in people who are quote unquote good math versus not. Right around when you have to learn kind of multiplication and division stuff like that so we want to get to kids then that's one thing but there's another thing and I think it's going to be for the growth of the app. It's actually pretty fun for adults in that we've made it into it feels a lot like a game like like tooling I mean I have the app on my phone it feels a lot like a game and this you know third to fifth grade math is right at the level where adults find it challenging.
But doable so if we were to give adults like most adults like ninth grade math did I can't do it like most of the like like to hard but if you're giving like adults you know some. Fraction problems or. Multiplication or stuff like that it's kind of like brain training so we actually think this app is going to grow both for students who are elementary school students but also. The app is going to kind of cater to both and we're very excited by that and again it's you know I have a PhD in computer science you would think that I am good math but I mean I'm loving kind of basic multiplication here where I'm not learning it I'm just just kind of every day kind of brain training so we're very very excited by that project.
Okay so the math is going to launch in 2022 I I am very much looking forward to that and it's timely for me it's been on my brain and if you can let. Just allow me to be a little self indulgent talk for a second. It's a bit a bit later in my life trajectory but I in ninth grade I had a I was very good at math and I had a math teacher who just brutalized. The male students in the class I don't know why she did this but she she just really had an extra grind basically all of. The male cohort in that class stop taking math. Myself included and my brother had the opposite experience and ended up with PhD and statistics and is. About as quiet heavy as you can imagine. He had a very very good teacher and I've always had on some level I consider myself numerous but in a very basic way and I've wanted to.
Rekindle some of those capabilities or at least the interest in math and concurrent with that one of my close friends closest friends. Is about to go to grad school and will need to take an intro to stats class but this friend also stopped math quite early didn't have the support structure that I did so for her it was probably. In the earlier elementary school years so I've been thinking a lot about this and you know how do you make it not just. Learnable but hopefully on some level interesting and enjoyable and rewarding so looking at the others I can't I can't remember his name there's a professor or was it Harvey mud who developed a book called math magic.
I've heard of that one yeah yeah I feel like you know some of those techniques might be considered like cheap tricks but I do find that it is very rewarding if you teach someone how to say multiply. Double digit numbers by 11 and it's a trick yeah doesn't apply to 12 but but it shows people that they can do something they thought they couldn't do so I'm very excited to see the math program. The other thing that you know we've learned that dual English what we did with the original dual angle app. The language learning happens we got really good at getting you to repeat stuff over and over again and you know the way you learn a language is by a lot of repetition. And I really believe this to be true also of kind of the type of elementary school math and so one of the things that's awesome about this product is that it really is fun and you.
We've made it so that repeating these multiplications or whatever is actually fun. Because you know you get all these kind of this progress bars and points and stuff like that and it just kind of feels fun and in you're doing exactly what you're supposed to be doing the way you get much better at this math is by. Repeating at 10,000 times this just that's just how you get better at it. So is there anything else that that you have coming that you'd like to talk about or anything else that you're working on that you're particularly excited about. The thing that I'm really excited about in addition to all these things is a project that we're not.
People in the US don't know so much for this project but it's a pretty successful and amazing project it's called the dual angle English test. What it is is a standardized English proficiency exam so basically you go there you take a test for call it 30 minutes 30 to 45 minutes and at the end you get a score from 0 to 160 about how good your English is that's it. That's the product it turns out that doing this is pretty necessary in most countries in the world again not that necessary in the US because everybody speaks English but. If you want to get a work visa in the UK you have to take a standardized English exam if you want to come to college in the US from an English speaking country you have to take a standardized English exam. If you want to get promoted in Japan and most jobs you have to take a standardized English exam so there's all these reasons for taking this and.
We made our own test the reason we made our own test is because before our test you had to these all these standardized English test you have to go to a physical testing center to take them. Yeah, like you know like actually go somewhere and take a paper test etc. We thought that was ridiculous so we made our own that you can take it online and it's been really really successful in a really rewarding project because what this has done so by now our test is is now accepted for example by 4200 academic programs in the world so if you want to apply to most universities in the US like if you want to apply to MIT or Stanford they accept our test as proof of English proficiency.
So this is great but the reason this has been so rewarding is that we're getting all these people to take this test that simply could not take a standardized English proficiency exam before. People who live far away from cities they just couldn't or you know it turned out that in the democratic republic of Congo there was just no way to take a standardized English test. So if you were there and you wanted to apply to a place like you know a US university you have to fly somewhere to actually take the test which is kind of ridiculous so it's been very rewarding to do this and I'm very excited by this project because first of all we keep getting more and more acceptance on it. And so the hope is that this will become the standard for English proficiency so what we're hoping to do and not just for English we're hoping to do it for every language out there at least the big languages.
We're hoping to change so that when somebody asks you how much French do you know how much English do you know we want the answer to be oh I am a dual English 65 or I'm a dual in 90 that's what we want the answer to be that's not true today but that's what we want. And it would be a lot better because for example the most common answers for how much French you know are one I took four years of high school French which doesn't answer anything.
The other one is I'm intermediate which is like also like what the hell does that even mean so we really want people to say I'm a dual 65 in French and our approach for this is going to be the test is one side which gives it our score legitimacy look that score is good enough for you to get into you know MIT or Duke or whatever.
That's good. And then we don't have that yet but we're going to start putting that score in the app when I start putting an estimated score of the dual in going English test where the dual in go Spanish test once we have a Spanish test we don't have a Spanish test. We're going to start putting in the app that gives it scale so pretty soon literally hundreds of millions of people are going to have an estimated score.
And the hope is that people start putting in the resume etc and I think if we're able to do that this will be not only good for the world but also for the company that will be a very defensible position because we kind of own the scale so I'm very excited by that.
I want to share a little bit of backstory and then I want to ask a question about how you actually do this testing so the backstory that I wanted to share briefly is that you're very good at catching cheaters do a history of catching cheaters so I don't want to spend too much time on this but I'll just give a quick recap so you.
You have taught a Carnegie Mellon and my recollections is this is short hand here based on notes I have but you tell students they were not to Google certain answers for specific homework assignments then you would create honey pot websites with the answers to homework questions the websites would record IP addresses then you would call out the cheaters and tell the class if they confess they would simply get a zero but if.
If he meaning you had to do the work of figuring out who it was you would report them and they got so scared of your tricks that they eventually all stopped you so you have a history of catching people who are trying to game the system in the case of remote aptitude testing for English are you still using the the front facing camera and microphone on a phone or how do you set it up such that it is hard to game the system yeah we've.
Spent a lot of effort to make sure that is one of the hardest things you have to one of the hardest problems you have to solve if you are testing. Some like an English proficiency for high stakes test. And this by the way that's the reason why they forced people to go to testing centers because they wanted to make sure they were cheating.
It turns out we were able to solve this and we just use a number of tricks I mean obviously we turn on the front facing camera of the device right now the test only works on computers we did not do it for phones. So you have to take it on a computer front facing camera the device we have a real human watch you to the test. We also have artificial intelligence that kind of looks at where you're looking and that turns out to be really powerful when you are taking a test it is very predictable where you're going to be looking I mean you basically on the screen there is the area that has the question we need to see that you actually went and read that.
And you know your eyes scan that then you're probably going to be looking where you're typing et cetera so you can pretty well tell that that is actually the person taking the test as opposed to being taken by somebody else. We also do all kinds of other things I mean we record all the ambient noise we also lock down your computer as in like you have to download an app that that kind of turns off everything in your computer and also tells us whether your computer is connected to external devices whether you have all kinds of you know like for example some.
Some headphones that are telling you the answer or whatever so a lot of that we do a lot of that and the way we know we catch cheaters is that we actually have a program where we pay people to try to cheat and we see whether they're successful or not that's great yeah you read team your own system we do and that's been really good and you know every now and then they're able to find something that kind of passes through and then we patch it.
And I'll tell you even though this is kind of the new technology and everything and people would say like but it's still computers maybe maybe they're doing something weird it's a hell of a lot easier to cheat in offline kind of testing centers because the way people cheat in these offline testing centers is so easy and it's as follows most of these testing centers aren't developing countries.
The person who is the attendant the person who works at the testing center is not a highly paid employee in a developing country turns out non highly paid employees in developing countries will take a hundred bucks bribe for well almost anything so this is this is how you cheat there and it's actually very this is low low fidelity all school cheating which is like here's on a box. Just say that I took the test but actually here's my friend who actually speaks English let them take the test thanks bye so that is how people cheat.
People actually cheat that way you cannot cheat that way with our test because you cannot see who's proctoring you you just you can't you can't communicate with them et cetera so you feel pretty good about the anti testing mechanics here. Yeah I'm glad I asked that that covered a lot more than than I expected we would cover do you still eat Greek yogurt for breakfast at least last time we spoke you to the same breakfast for two to three years as well as the as an example of one decision that removes a thousand decisions or have you have you decided to swap in a new breakfast for yourself.
I swapped in I'm very similar but I just do now it's just a little jug of muscle milk because my trainer Gary on his first date Gary's a funny guy he only cares about strength he said let's have a drink. He said let's have the nutrition conversation I'm like okay and he said drink muscle milk and then I'm like anything else he's like nope just drink muscle milk thanks that was his nutrition conversation good talk good talk. I was hoping he'd be like well you know eat this type of whatever vegetable and this after that nothing he's just like just drink some muscle milk so it's what I do for breakfast.
It's simple daily have it yeah it's like brushing your teeth yeah all right two more questions maybe three and then we can bring this to a close and land the plant all right so next question is more generalized what advice would you give to a smart driven say college student could you could pick a different age. About to enter the real world and secondarily if you want to take a stab at this you know what advice should they ignore.
You know for me the biggest advice is. I mean it depends on what you want you want to get a job you want to work you want to work or do you want to just play video games if you want to play video games go play games whatever that's fine. But if you actually want to get a job I would say the most important thing is find a way to become useful.
You know I am getting older and I don't know if it's just always people talking about the younger generations this way but a lot of times a lot of kind of the you know the gen Z's used to be even millennials but the Gen Z's. They show up and they're like what are you going to do for me and it's like well can you do this no no no I don't like that that's not a look at learning opportunity for me.
Or like you hired me to be a software engineer I'm not going to do that's not my job I'm not going to do that that's not my job. The attitude of just finding a way to become useful gets you a lot farther yeah certainly a dueling but I think in most places in life where it's just like oh you need me to do this this is what's actually needed yeah I'll do it I'm pretty smart I'll figure it out I'll do it.
And now obviously you know if you're hard to be a software engineer you should not go and like build a building because you don't even know how to do that but just kind of just find ways to become useful and crucial for the company will get you a lot more perks and promotions than having this attitude of like you know what are you going to do for me. Yeah that's just my sense and I see that a lot of the what are you going to do for me attitude yeah you know just to add to that the opposite would be something like and I think I'm getting this right so friend of mine Kelly star at very well known performance coach PT has written many books works with a lot of professional athletes the top tier of of elite military and he also happened to be.
I would I would say he might disagree but Olympic level whitewater kayaker for a period of time and I went on the I went on a river trip with him and he mentioned to everybody as this priming talk as part of this orientation he said you know that there are really four words that will make the biggest difference in this trip and it's how can I help he's just like everybody should ask that and if you just I mean it's it seems so rudimentary but I'm not sure if you can do it.
So rudimentary but I would agree with you and man if you just take that attitude like you're on a river trip how can I help if you've got some extra time or slack. It really will distinguish you it really does it is amazing I mean I see it we do you know with course we hire a lot of people and everybody who hire is pretty awesome but the ones that have more of this attitude just get farther in life. Yeah I just because they're just there like how can I help I just I just need to help we have a I have a story of it's kind of two separate employees. This is kind of early dual angle where we didn't have a blog of course now have a blog for stuff one just kept complaining we didn't have a blog he's like I can't believe we don't have a blog why don't we have a blog. It was ridiculous we don't have a blog but never did anything another one started and they just said I noticed we don't have a blog so I started one. And here it's linked to the website that is the attitude that helps a lot more than just being like I can't believe we don't have a blog how about you do something about it.
Okay last question for me and then there's just a closing question of if there's anything else you'd like to add say or direct the listeners to anyway in the last handful of years what have you become better at saying no to could be distractions invitations anything are there any new realizations or approaches that have have helped you to say no to more maybe that's not even problem you have but I figured out asking. No I have a problem saying no and I definitely become better at saying no you know in some cases I just don't have the time but it is a problem I mean the promise I'm a people pleaser like I want to please the people in front of me whoever they are sometimes even if they are random stranger emailing me some stuff I still am a people pleaser so this is not that new but it is a big trick that I learn a little while ago specifically for invitations and stuff like that which is. I used to say yes to an invitation to like give a talk next year just go give a talk next year and I was to say yes because I thought next year I won't be that busy or whatever like and just if you change your framing whenever somebody asks you to do something a long time from now change the framing and say what if it was tomorrow.
Would you want to do this or not. Almost invariably the answer is not I definitely don't want to do this tomorrow and then you should answer based on that you know and I think that has helped me say no to a lot of things where you just say like okay because it's not necessarily a trick that people use although sometimes maybe it is a trick but you know when you I just get asked to do certain things months from now and I never think about. Man I'm actually going to have to do it and it's going to suck I just never think about that so I'm just like yeah months from now I doesn't matter doesn't matter you really you really should get a lot better at that and just think about. What if it was tomorrow whether you do it tomorrow good advice good advice Luis is there anything else that you would like to.
Add any requests of the audience. Of course people can find you on Twitter at Luis on on they can find do a lingo do a lingo dot com or anywhere they find their apps most certainly they can find it anything else that you would like to add or mention draw attention to or closing comments.
I do have one thing other you know you're welcome to cut it if you don't think it's going to be all that interesting but I would be remiss if I didn't say this and it is it is about my country Guatemala. I would like to call attention particularly here in the United States about the level of corruption there and you're like okay who cares Guatemala's corrupt. First of all it's extremely corrupt and there are real consequences for the United States and in particular one of them is probably illegal immigration there's you know Guatemala's if it's I believe the second country with largest illegal immigration into the United States there's a lot of there's literally millions of illegal immigrants from Guatemala here in the United States.
And you can be very concerned about that and not only can you be very concerned about the treatment of them in the border which I think is horrendous but you know once they're here like you can be very concerned about that.
And the sole reason why they're leaving so much is because the country is extremely corrupt and just cannot you know handle itself basically. And the reason I like to call attention to that is because I do believe the United States and their foreign policy can actually help with this in many ways and unfortunately that is just not something that's happening too much but I do believe that as long as Guatemala continues to be this corrupt there will continue being a lot of illegal immigration and I also think there's so kinds of stuff I mean it the thing is so close to the US that it has become a hub for you know basically international crime.
And you know when they're close to the US that means some of that crime spills over to here you know so far it hasn't happened where we haven't seen like we haven't seen a terrorist attack where the terrorists came to Guatemala but I wouldn't be surprised if that happens where you know it's just extremely corrupt place and they just kind of funnel stuff from there so.
Given that it's so close to this country I think this is of importance and of course I have a self serving reason for saying this I mean I would like my country to do better but at this point the level of corruption there is is unbelievable. So that's the one thing that I that I would like to call attention to that's why I asked I think this is perfectly fine place to talk about this and let me ask you follow up for those policy makers.
People listening who may actually be able to look at this work closely and have an impact are do you have any specific recommendations or requests of them I do a number of them probably one of the biggest ones is there's a free trade agreement with Guatemala the same one is NAFTA I mean there's a free trade agreement with kind of you know Mexico and Guatemala etc there are classes there like anti corruption classes.
They are being overlooked it is known that there's a lot of corruption in a lot of these countries and I think that the US can say you know what it's clear you're that corrupt we're going to we're going to clamp down on NAFTA unless you fix your problem and this would get these countries to fix their problems because they care about something like NAFTA because they make a lot of money off of it money moves things so that's that's one thing another one is considering I mean there are sanctions that they put to individuals.
The US has done these things where certain individuals they just consider them so corrupt that it like sorry you cannot enter this country anymore we don't even want you to come visit you know the shopping trip to Miami or whatever we don't want that. Crazy story some of the highest government officials in Guatemala including our attorney general is banned from entering the United States because they're considered so corrupt applying more of that that actually has real teeth and in particular applying that to corrupt business people because they are not going to be a real deal.
Because they all love their nice mansion in Miami or you know I bring up Miami because Miami Latin Americans love Miami they love having mansions there they all have their nice mansion in Miami etc if suddenly they cannot come to the US that's bad and it looks bad for their kids because their kids they love to come to Miami and their kids friends lived so this is the type of stuff that I think has teeth that the US can do.
Well thank you Luis I did I did not see that coming and I'm glad you brought it up. I thought about this problem a lot I thought about about this I mean I poured millions of dollars to try to improve I own a good fraction of the oldest running newspaper in Guatemala and we we do a lot of stuff against corruption so I poured a lot of money onto this.
Yeah but unfortunately I've kind of calculated that it's a lot I think the corruption problem in Guatemala could be solved with some money but I think it's like billions of dollars unfortunately I have not dropped that amount but I put in millions of dollars to try to fix this.
All right what I might do is ask you after we stopped this recording well to get maybe a few links for policy makers who may be interested in in learning more about this and the different resources they might want to educate themselves on whether it's trade agreements or otherwise so we'll get some links and put those in the show notes as well and Luis thank you for making the time it's nice to see you man it's been a time to be a little bit more interesting.
It's been a little while thank you Tim I mean and for those who can't see the video I appreciate the matrix like background that you have as well it's very compelling. Thank you Tim I mean I'm a great studio here near the Oolong offices and hope to see you in person not too long from now so maybe maybe we can next time you're in New York City we can figure out a way to break some bread or have a drink together. I would love that if you're in New York City I'm there very often I would love that.
All right wonderful well Luis thank you so much and I always enjoy our conversations as I mentioned before people can find you at Luis on on Twitter dolingo.com and we'll link to everything in the show notes at Tim. blog slash podcast and to everyone listening be just a little kinder than necessary today and. Thank you so much for tuning in and until next time take care.