How machine learning empowers knowledge sharing? Lei Yang, VP & Head of Engineering @ Quora

This episode features Dr. Lei Yang, VP and Head of Engineering at Quora, the biggest Q&A website in the US. Dr. Yang discussed her career journey from Google to Quora, current machine learning applications at Quora, and leadership insights on managing and scaling teams in the interview. AI Talk with Lei Yang, VP & Head of Engineering @ Quora

Prior to Quora, she worked at Google, where she actively engaged with the Google Research Award program and was the Google liaison for many university research programs. She holds a Ph.D. degree from Northwestern University in Computer Engineering. Dr. Yang has over 10 years of hands-on experience on machine learning and has built and led teams in the areas of ranking, personalization, content recommendation, and more.

Highlight 1: How is content on Quora different from other platforms?

Highlight 2: Challenges in applying machine learning technologies?

Full interview:

Listen to podcast on: Apple Podcast | Spotify | Google Play Music is a content platform dedicated to helping engineers and researchers develop leadership, entrepreneurship, and AI insights to scale their impacts in the new tech era.

Subscribe to our newsletter to stay updated for more inspiring talks:

Full Transcript

Alex: Hello everyone, welcome to Leadership Talk. Today we welcome Dr. Lei Yang to join our program.

Lei Yang:

Thank you, Alex. It's an honor to be here.

Alex: It's interesting that you have more than seven years’ experience at Google, from 2008 to 2015. What’s your major work over there?

Lei Yang:

Sure, I'd be happy to share that. I joined Google as my first job after graduate school, and I've worked at different organizations and teams at Google. The first team I joined was Ads. That's where I started my journey on machine learning and analytics. I worked on the spam and fraud detection department within Google Ads function. Then after a few years, I moved on to join Google Plus, and I spent about three years at Google Plus. Over there, I continued to use machine learning but then shifted the focus more on ranking, personalization, and content recommendation. Towards the last year at Google, there were some changes that motivated me to look outside. So I left Google after seven years altogether.

Alex: Were you a manager back then, when you worked for Google Plus Program?

Lei Yang:

I became a manager when I worked in the Ads team in the later years. When I joined Google Plus, I went back to being an individual contributor/ engineer. After a year, I became a manager again at Google Plus and then started building a team there.

Alex: Quora right now is one of the biggest Q&A platform. But back in 2015, I believe the team was relatively small. What actually attracted you to join the company?

Lei Yang:

I think the number one reason is the mission. At the time when I was looking, I wanted to work on a product that I genuinely believe is generating value to human beings and to the society, and I was evaluating a number of options. There were many rocketships and successful products out there, but Quora is really unique in the sense that it is really using technology to change human history, by affecting the way that we share knowledge. Since the company has been founded, that mission has never been changed; we've been marching steadily towards that goal. I felt really good about contributing something positive to the world. That's the first reason.

The second reason is related to the culture and size of the company and pace. When I worked at Google Plus, I was really enjoying it. Google Plus today as a product no longer exists, and there are a lot of views from the world about how (un)successful the product is. Back then when I joined Google Plus, it was a top priority for the company, as Google was very keen on using Google Plus to compete against Facebook in the social network domain. As a result, there were a lot of resources poured into it. More importantly, I really enjoyed the culture there -- it feels like a startup within a big company. But then, after a few years, we realized that the product is not really finding the right product market fit and not really getting user traction; it’s essentially failing. So we started to have a lot of churns, strategy pivoting, and organizational changes.

Then my team got re-organized into the search organization. Google Search is a very well-established product. It's hugely successful and used by so many people in the world. But to me, that team transition was a huge cultural clash. Working at Google Plus feels like driving a speedboat. It's small and fast. You might make really rapid shifts from time to time and there's a risk of falling. But people are generally working together to make the boat go smoothly. When I went to search, it feels like it is a huge aircraft carrier. It's really big and stable. It's never going to sink. People can fight on top of that and nothing's going to happen. That's when I realized that people spend time and energy to optimize for their own interests instead of doing the right things for the business. At least that's my personal opinion. That's when I felt like I was wasting my time. So that motivated me to leave Google.

One of the reasons why I found Quora to be really attractive is the culture, the people, and the general tone the organization has set to help people work together. Generally, people are really collaborative and are here for the mission, trying to make the product successful. That is a really enjoyable working environment. That's another top reason.

The last reason is I wanted to continue to work in the domain of machine learning. Ranking, recommending, and organizing content to give users a better experience is really, really interesting to me. So I want to continue to work on this. Joining Quora, I started working on exactly the same problem set, and it was really rewarding. So that's the answer to your question on why I joined Quora.

Alex: Seems like your passion lies in applying machine learning to solve product problems. How did you apply machine learning on Quora?

Lei Yang:

I wouldn't say I applied machine learning on Quora. Being an engineering leader, there's a big team behind me.

Alex: You’re already leading the machine learning team, right?

Lei Yang:

Yes. Generally speaking, if you think about Quora as a content platform, we can use an analogy, like it's similar to a newspaper. When you think about the process of getting a newspaper published, every single step in that process can be optimized using machine learning. At Quora, we use machine learning to rank the home feed for users, personalizing what topics, what answers, and what questions are the most relevant to you at this moment. We also use machine learning for internal search, recognizing topics, and recommending the right users for you to follow as well as the right spaces to join, and also ranking all the answers for one particular question depending on the quality and how users receive them. So there are many, many different and interesting applications of machine learning at Quora. We've been using various models on these different applications too.

Alex: Compared to applying machine learning, ranking and recommendation algorithms to Facebook news feeds, what’s different between Quora and Facebook?

Lei Yang:

That's a great question. I think there are a lot of common problems, there are also some unique problems. For example, one thing I mentioned was, at Quora, we have this challenge of organizing content. Because if you think about the mission, the mission is to share and grow the world's knowledge. So we have this pool of knowledge, and if they're not organized well, they cannot be distributed well to the people. When a question is asked, what is this question about? How do we know who are the experts to answer those questions? We have something like the knowledge graph to organize topics for all the content. We also have to match those topics with experts. I don't think Facebook has exactly the same problem. They might have similar challenges as well. But at Quora, we tend to think we have a pretty unique set of problems that we need to solve.

Alex: You have user profile as well, like Facebook. Then you have some attributes for each person, so you learn their behaviors.

Lei Yang:

There is a social graph on Quora as well. People follow each other. People upvote and downvote others’ answers and questions. For someone that I follow, I should be notified when they write something, and I should see the content they write on my feed. So I think there is a similar concept compared to Facebook.

Alex: Do you think this content is time-sensitive? Like Facebook or other platforms, they push this kind of news and maybe update to their friends. How about Quora? I follow someone who publishes the answer to some good questions, is that time-sensitive?

Lei Yang:

I think that's a tricky question, and I tend to think that you have to find the right balance. Timely content is important. For instance, if I follow you on Quora and if you write something, I would want to know immediately rather than a month later. Because what you're talking about today may be less relevant a month later. So there is that timeliness that we have to care about.

On the other hand, Quora is generally a knowledge platform and not a news platform. A lot of knowledge is evergreen, right? I follow topics like machine learning that might be more timely. I also follow topics like parenting, and there are timeless advice and knowledge that people can share on parenting. So you want to make sure the platform supports both. You want to make sure that all the timely content can get distributed immediately, and all the evergreen content can still have a healthy circulation along the course of history and not be penalized.

Alex: Lots of people answer many questions on Quora. What kinds of motivations do you see? Why do they spend so much time on Quora?

Lei Yang:

Again, a great question. People might have different motivations. To me, I think one of the biggest motivation is just the authentic desire to share knowledge. There is far more knowledge in people's heads than on the Internet and in books, and a lot of people like sharing them. But the problem is, everybody has a limited time. If I have some knowledge to share, I want to know what knowledge is the most valuable to others, who want to learn from me, and who is my audience. This is why I mentioned earlier that I think Quora is a product that's truly using knowledge to push the world forward because we're changing the way people share knowledge. It used to be that only very rich people can read books because the printing press was not even invented. Then there's newspaper, there's book, and then there's internet. Content gets flooded on the internet, but there's a lack of organization, not knowing who should I ask, what kind of knowledge I should be sharing, and what kind of knowledge I should be learning. Quora can provide such a platform.

Alex: What will be technological challenges for you to build such a big platform and apply machine learning to serve so many users? What is your biggest challenge?

Lei Yang:

There are many and I can share some. Specific to machine learning, I think one of the biggest challenges not only to Quora but also to a lot of companies who are using this technology is developer velocity. Basically, how fast can you iterate? There are many new advancements in machine learning. People are writing research papers all the time, and new things are getting out of the door all the time. But how can you quickly pick up a new technology / new model, and experiment it on your product from conception to production? That is a huge bottleneck.

A big challenge and also an advantage for us is that we have to build the right platform and infrastructure to support very fast iteration so that we can prototype and experiment with different machine learning technologies in order to understand what works best for us. We've invested a lot to build such a platform, to the extent that we believe compared to companies of similar size and scale, we’re one of the few where developers can really feel like it's easy to do machine learning, and it’s easy to experiment and prototype. That's something that we're going to continue to invest in going forward.

Alex: In my opinion, Quora is mostly text-based. Do you foresee in the future, you will have images or videos in Quora?

Lei Yang:

I think we're very open to that. Again, tying back to our mission of sharing and growing the world’s knowledge, knowledge is not only in the format of text and the way knowledge is shared may very well change in the future. A few years ago, there was no Amazon Alexa or Google Home; voice assistant is not a thing. But today, they're more and more applications like that. I can totally imagine in a few years, a lot of knowledge can be shared using video or audio. The challenge is how -- text is very easy to maintain, organize, and discover, but video and audio can be more difficult in that regard. So I think technology can really help there. As a platform, we're very open to trying new things as people have new demands, or there are new formats in which knowledge can be shared.

Alex: Let’s move to some topics about leadership. You mentioned you became a manager when you worked for Google Ads. Then you became a manager again when you worked for Quora. How did you learn to be a manager from an engineer background? How did you transit from an engineer to a manager?

Lei Yang:

I tend to be more cautious about giving advice to other people on how to be a good manager because I'm a big believer in situational leadership. I think it really depends on the people and the situation. I would say that personally, my style is learning by doing the job. One thing that I feel is general enough and safe enough to share is if there are opportunities for you to tackle problems that you don't know if you're capable of, say yes. Often times, I make mistakes along the way. But I got some of my biggest learnings when I'm failing and trying to get better at it.

As you continue on the leadership trajectory, at a point, you'll find that it is necessary to build some principles and frameworks and create structured ways to articulate them. This is really helpful when you need to manage managers. Many new managers make decisions based on their intuition and judgment, and later on they start to do pattern recognition, like hey, this is how I solved similar problems in the past, and I'm going to do the same this time. Now, when you have leaders under you, you cannot help them tackle every individual problem by doing this pattern recognition. You have to be able to teach them what are the principles that lead you to make decisions like this so that they can solve problems on their own. You'll find that along the process because you have to crystallize your intuitions and gut feelings into frameworks and abstractions, you now understand the underlying issues better. That's also a great way of learning.

There are a lot of resources with different management frameworks and principles that people can learn from. I tend to think it's always the best when you could develop frameworks based on your own experience. When I really get to understand something, I’d look back and think about the books I've read in the past and feel like, oh well, that makes sense now. But at the time when I read the book, it may not necessarily resonate that much.

Alex: I remember, you have about 100 people under you in your team. Do you have to spend time one-on-one with each of them, or what is your way to manage the team?

Lei Yang:

I can't have one-on-ones with every single engineer in the organization, although I will try to spend time in some way with everyone if I can. I do maintain regular one-on-ones with my direct reports, and I think that touchpoint is extremely important. I try to spend one hour every week with my directs. I think having transparency in your communication, sharing common knowledge and context, and even just getting to know each other’s style and understand how to work with each other is really critical. We also run different types of staff meetings, like weekly group sync with all my direct reports and monthly sync with all engineering managers. There is also a technical forum where all tech leads in the engineering team are getting together to discuss our technical vision, strategy, and planning of technical projects.

I think a lot of times, your leadership style is reflected in how you handle relationships with your people and how you run these meetings. Before you go into the meeting room, It’s really important to understand clearly what the purpose of the meeting is, and if possible at all, prepare the agenda ahead of the time. You don't always have to do it yourself and can delegate to other people. But in summary, putting a lot of thought into how you organize the face time with your people is really important.

Alex: In one of your previous interviews, you mentioned you truly love two books. One is called How to Win Friends and Influence People, and another one is Radical Candor from Kim Scott. I also remember you mentioned about how to build trust with your team. Can you give us a summary of what’s your takeaways from those two books?

Lei Yang:

Again, I think there are many great leadership and management books out there. These two are the ones that I liked enough to read multiple times. First, on Radical Candor, I think one of the biggest takeaways is how you can have that direct and transparent conversation and communication with not only your reports, but also your manager, your partners, and your peers. The key is if there are some issues or misalignment or disagreement, it's much, much easier to solve that in real-time and early on, rather than wait until it rolls and gets bigger down the road. Radical Candor provides a really good framework on how you can actually do this because it's not trivial for people to be direct and transparent with each other and give critical feedback.

Alex: Do you give instant feedback to your people?

Lei Yang:

Yes, I try to, as much as I can.

Alex: Do you think people can accept it because they think they made a mistake?

Lei Yang:

I think it's important that you tell them your intention. Sometimes, for example, if I feel like one of my people did not communicate his or her ideas well in a meeting, I would follow up right after the meeting and give them the feedback like: Hey, I feel like that could have been communicated better.

One of the tips that book gives is, in order to be able to give radical feedback to people, you need to have their trust first. For two people who have not worked enough with each other, critical feedback is almost never expected to be well received. Because people have to assume you have good intentions, and that takes time to build up. How can you build that trust first as a foundation, and how can you align your goals with your people and deliver such feedback in a very reasonable and respectful way that's going to be well-received? There are many useful tips from that book that I highly recommend leaders and managers to read about.

Alex: Another thing is, I remember when you join Quora, your team was quite small. I remember you were the manager, and then you scaled the team with over 100 engineers. What is your learning about how to scale the team and retain those employees?

Lei Yang:

I did experience a lot of growth for the engineering team since I joined Quora. I'm happy to share some of my learnings in scaling teams. This is a big topic and it's really hard to summarize it in a few bullet points. But I tend to think that whether you are scaling a team from zero to 10, or 10 to 20, or 10 to 100, there are some key elements to consider: culture, people, structure, and process. Culture is the most important thing. It basically sets the tone for everything else in your organization. It defines the values of the organization, what kind of behavior we encourage and discourage, how people should work with each other, and what we can expect from each other. When you're crystal clear about the culture of the organization, you can then start to build the fundamental blocks. For example, what’s the right organizational structure to support the scaling needs and growth of the team as well as to align with product development, priority, and goals? Once you have the right structure in place, you need to put a few key people on the right spot (leadership position). I said that culture sets a tone for everything, and that includes how you hire these people and how you reward these people.

I think another thing to consider is basically the process. In order to protect and maintain your culture and the structure you've built, you have to think about what processes are necessary. There are many common processes, like performance evaluation and reward system, or project management, as well as a technical review system. Once you have the right culture, the right people, the right structure, and the right process, I think everything else becomes easier and more scalable.

Alex: It seems like as you mentioned, from first-line manager to the VP level, or maybe manager of managers, the biggest difference is you learned how to scale the team and this process and our culture to measure the performance of the team. That's your takeaway, right?

Lei Yang:

I think for all managers, your goal is to make the team successful, and that means different things at different levels. When you're managing a team of engineers, success might mean that these people are happy, projects are on track, goals are hit, and the team is shipping business impact. When you're leading a 100-people team, things are much more complicated, because you can no longer have a touchpoint on what everybody's doing. That's when having the right structure and process is important, so that you can actually scale and support the needs of a much bigger team.

Alex: It seems like your leadership style is more like an adaptive style. There are six types of leadership styles. And some starts like coach, but you’re more towards this kind of adaptive style, right?

Lei Yang:

Honestly, I never really think about my leadership style. Interestingly, when I interview manager candidates, a lot of them would ask, “What is your leadership style?” I found that to be a really tough question. Instead, I often tell them what I care about and how I prioritize things. So for instance, I think I'm a people leader. I really care about the well-being of my team, and whether everybody is truly productive, effective, growing, and happy. At the same time, I’m also very result-driven. I want to see the team succeed as a whole, and that really involves a lot of things. So I think you're right, I really have to adapt to the different needs of the team at different times and to different people.

Alex: My last question is, where do you see yourself and Quora in the next 3-5 years?

Lei Yang:

I’ll start with Quora. I'm very bullish on the outlook of the business. A lot of people here in the audience use Quora, but may not necessarily know what is going on inside the company. Granted, we're not a company that pays a lot of attention to marketing and PR, and we probably should do more. The long term goal for Quora is to grow big as an independent business. We're marching towards that on multiple fronts. We are launching in international languages (a lot of them actually). We’re monetizing and growing our revenue. At the same time, we'll continue to grow our traffic and usage. So I think there's a lot of upsides on the product and on the business. I would also call out machine learning is a revolutionary technology that's changing the internet industry, and also changing how we think about knowledge at Quora. There are a lot of great technical challenges for me and my team, which will keep me busy and excited for a while.

Alex: Let’s thank Dr. Yang for sharing her learnings, not only technical learnings, but also learnings about how to be a manager, how to be a managers’ manager, and what she's doing at Quora and her vision. Thank you very much, I learned a lot.

Lei Yang: Thank you.