Research in Academia versus Industry, with Philip Resnik and Jason Baldridge

Guests: Philip Resnik, Jason Baldridge
Hosts: Waleed Ammar, Matt Gardner

How is it like to do research in academia vs. industry? In this episode, we invite Jason Baldridge (UT Austin => Google) and Philip Resnik (Sun Microsystems => UMD) to discuss some of the aspects one may want to consider when planning their research careers, including flexibility, security and intellectual freedom. Perhaps most importantly, we discuss how the career choices we make influence and are influenced by the relationships we forge. Check out the Careers in NLP Panel at NAACL'19 on Monday, June 3, 2019 for further discussion. Careers in NLP panel @ NAACL'19: https://naacl2019.org/blog/careers-panel-survey/ Jason Baldridge's homepage: http://www.jasonbaldridge.com/ Philip Resnik's homepage: http://users.umiacs.umd.edu/~resnik/

Matt Gardner
00:00

Hello and welcome to the NLP highlights podcast where we talk about interesting work in natural language processing.

Waleed Ammar
00:06

This is Matt Gardner and Waleed Ammar, we are research scientists at the Alan Institute for Artificial Intelligence. So a few weeks ago we asked for suggestions on topics you’d like us to discuss on the podcast. One of the suggestions was to talk about academic research versus industry research and NLP is one of the areas where both academia and industry have both made important research contributions. So today we’re inviting NLP researchers who spend time both at academia and industry to help us make sense of the differences and tradeoffs between academic and industry research. Our first guest is Philip Resnik. Philip is a professor at the University of Maryland, College Park. He is affiliated with linguistics and CS departments back in 1993 he spent three years as a research scientist at Sun Microsystems Laboratory. Before then he also spent some time at IBM Watson and VBN. Our second guest is Jason Baldridge. Jason is a research scientist at Google and is the cofounder of People Pattern a start up, which uses machine learning to improve market research. Before joining Google in 2017, Jason was a tenured associate professor at University of Texas at Austin. Philip and Jason, thank you for joining us today.

Jason Baldridge
01:13

Thanks.

Philip Resnik
01:14

Great to be here.

Waleed Ammar
01:15

So could you start by telling us how you decided to do the switch one way or the other and why you stayed since then?

Philip Resnik
01:22

For me, I had looked at academic positions coming out of the phd, you know, at the time, I found it harder to wind up where I needed to be. And uh, you know, there were, there were two bodies involved and I was, uh, I was lucky enough to, to find, um, a really great industry position at a, at Sun, um, working with Bill Woods and Peter Norvig was there at the time and, and a bunch of others. In my case it was, I actually didn’t expect to wind up going back into academia because particularly at the time, that direction was really pretty hard. If you’ve been out for a couple of years, then you’re competing with people who are fresh out. And I had more or less resigned myself to, I’m going to have to, if I want to get back into academia, you know, wait until I can try to make myself a big name 10 years from now and then, you know, get hired as, senior faculty.

Philip Resnik
02:14

And in my case, it was really good fortune, honestly. I had some geographic reasons to change where I was living and I was scrambling around trying to figure out what I was going to do. And in my case, uh, university of Maryland had a job posting that looked as if it was written for me in terms of a joint position between the linguistics department and, the Institute for Advanced Computer Studies. And I can tell you it wasn’t written for me because I know who they offered the job to first. But, you know, it was really, in my case, a very fortuitous thing. Um, but it also had the end result, that sort of progression from industry out of undergraduate for a couple of years at BBN then phd and then back to industry. And back to academia that it gave me sort of a much broader perspective on life in either one or the other. So, you know, these things work out the way they work out in life and I can’t say I’m unhappy it happened.

Waleed Ammar
03:10

All right, great. How about you, Jason.

Jason Baldridge
03:12

Cool. So I think, you know, in my case, I could actually give you a number of examples, but this is not a binary switch, right? Where you’re kind of carrying on multiple interests as you’re going along. So just as an example, I started my phd at the University of Pennsylvania in the linguistics department. I started working with Mark Steedman and then he had an opportunity, well he went to the University of Edinburgh and invited me to go along. So I was like, that’s cool. But right at that time for other reasons to do with my then girlfriend, I was thinking, Hey, maybe I should just skip on over to the Silicon Valley. This was 1998 where there were lots of big opportunities if you could just get into industry right at that time. And so we considered that, but ended up deciding, hey, going to Scotland sounds pretty cool and working with Mark Seedman is awesome.

Jason Baldridge
03:51

So, I did go with that. That said, I then ended up being part of a startup there that we got going. We actually got funding, um, to the tune of 200,000 pounds, um, that was promised, but then my co founders had to sort of, for their own personal reasons, stop that. And I decided not to carry that on much to Mark’s joy. I think. So, you know, that was kind of going on right at that, around that time. And then I sort of happily finished the phd and then did the postdoc at Edinburgh and from there on, I was considering both the University of Texas at Austin and Google actually. So I kind of had that decision right in front of me at that time. And actually Peter Norvig gave me advice then. He was at Google and I was asking him about this kind of decision about, oh, industry versus academia.

Jason Baldridge
04:33

That was very much front and center in my mind. And the advice he gave was actually no matter where I go, I want to work with smart teams of people who are working on interesting problems. And I think you can find that in both contexts. And if you can find that, in either context, that’s great, go for it. So that’s kind of like a really key principle that I used governing all of my decisions going forward. I did decide to go to the University of Texas at that time in part because I thought, well, this is a bird in the hand; I generally understand it’s harder as Philip mentioned, to go from industry to academia then from academia to industry. And so I decided to go for that. At that time. There were other factors. I was actually getting divorced from my wife. We had a two and a half year old daughter.

Jason Baldridge
05:11

And having the university schedule at that time allowed me to have her with me during the summers. And that was kind of actually excellent flexibility that the academic life gave me at that time. Uh, then at the university I then had to take care of some other things, such as increasing my income, uh, to support me and my family. Um, and that’s one of the ways in which Philip and I actually started working together on building a sentiment analysis system for conversing on. Uh, so I was actually moonlighting spending evenings and weekends to do that consulting work where I was actually doing a lot of the programming while still being professor by day. And so that kind of was this constant thing in flux, like, you know, and then to sort of wrap it up, I got tenure and I was sort of thinking, well, what are my next steps?

Jason Baldridge
05:52

Um, and then I’d sort of hoped that I could support my professor had that by having a startup that could of be a thing that I could get going and then, you know, sort of do one day a week going forward. That didn’t really work out with People Pattern, but I learned a lot along the way and got to hang out with Philip quite a bit.

Waleed Ammar
06:07

When did you start People Pattern?

Jason Baldridge
06:08

I think that was 2013. Okay. So then ultimately, uh, I sort of wandered further and further away from just the core academic thing because I’d gotten much more of a taste of doing industry work through the consulting and through People Pattern and for various other reasons I can go into later, I decided I was going to jump and go all the way to, Google.

Philip Resnik
06:26

So just to elaborate for a second on one or two things that Jason said. By the way, Jason mentioned flexibility, which is an important theme that comes up. Also the circumstances of my coming to academia involved divorce and a child and you know, the common statement is hey, academia is great for all the flexibility you get. You can work any 90 hours a week you want. And it’s true, when you have a situation that requires flexibility and fluidity, it gives you a kind of flexibility that is hard to find in other things. And we can talk more about startups too if that comes up because they have some, some advantages and disadvantages that way too. The other thing I wanted to mention though, you mentioned moonlighting, Jason, is that one thing that seems to me to have changed quite a bit is it used to be a very much an either or proposition to choose between academia and an industry where at major research universities, at least it was good

Philip Resnik
07:18

you were able to do working on the side, say like a day a week or something to do consulting or whatever, and you’re starting to see a model now where an increasing number of people are actually splitting things rather more substantially. In other words, being in academic positions but also engaged with industry or consulting or startups at the same time and so the nature of the choices that people have to make has been changing because there are newer kinds of models for dividing your time then I think tended to exist in the past.

Matt Gardner
07:47

I guess one interesting thing that stuck out to me from both of your stories was that how much your life circumstances affected how things turned out. It seems to me like that’s pretty much always, I think, the major driver here in these decisions and it’s good that I think both places will have opportunities for whatever your life circumstance is. I guess it also very much impacted how I ended up where I am right now, which is in a little bit of an interesting, unique situation. For a bunch of family reasons, I decided I wanted to move to Irvine where my, where I grew up and where my parents live and told AI2 that I was moving and they said, hey, how about you open an office for us? So anyways, long story short, I’m, I have an office on UCI campus. I co-advise students with Sameer Singh, but I guess I’m technically a UCI affiliated visiting scholar. Um, but I’m paid by AI2 and don’t have to worry about grant funding. It’s a very interesting and nice mix between academic and industry research, but again, fully fully determined by a crazy series of, life circumstances.

Waleed Ammar
08:49

So this reinforces what Philip was just saying that, it’s not, it doesn’t have to be one or the other, but also I think this is kind of a rare opportunity. Oftentimes people who are already like well in their careers as professors are welcome to join like as 50% or some percentage of their time in an industry. But someone who’s just getting his phd. Yeah, they could do a post doc joint between industry and academia, but I don’t see a lot of opportunities for someone who’s just starting to split their time.

Philip Resnik
09:19

Yeah. I think that’s largely because really any situation where you are working out the fit with an employer, the core question has to do with value and uh, you know, the way you deliver value is different depending on, you know, where you’re working. But really that’s one of the several fundamental things that is always at the top when you’re looking at, you know, they’re looking at you and saying, do I want this person here? And it’s more difficult for somebody who is not as established to create a clear picture of their value and the way that they can deliver it. And it’s much harder to do it in a way that you’re providing two different kinds of value in two different settings. Right. So, Jason, you had established your value, you know, in a startup sense, being able to deliver stuff that, you know, you can build and it works, right? That’s a kind of value that’s crucial. You’d also, through your academic career, established your ability to produce students and publish papers. That’s a different kind of value. Somebody coming out of a phd with a postdoc under the typical trajectory has really not had a chance to build up a solid track record in more than one thing. There are certainly exceptions, but it’s harder and I think that’s why a people who are more established have more opportunities in the way that you’re talking about.

Jason Baldridge
10:36

That’s absolutely true. I’ve actually recently moved to Austin as, uh, some of you know, and that was actually something that would not have been possible for me to do at Google two years ago. When I joined the Austin office had a much smaller number of people, wasn’t as, sort of well built out as it is now. And it so happened, again because of family circumstances, my family wanted to be back in Austin. They didn’t want to live in Mountain View. It turned out that I had the opportunity to come back to Austin and with the goal of not just being a singleton but of actually growing a team here and so it’s a really exciting opportunity. But, it is again, one of those things that comes based on the back of what you’ve done before.

Philip Resnik
11:12

Yeah. Although I will say that even for people starting out, there was a, there is a message here which is, there’s a difference between the jobs that people have available and the job that you want to have and there are circumstances where you really should be going into it thinking in terms of, look, this is the job they say is available. Let’s have a conversation about how to make the job that is the right job for me, also. People sometimes feel as if there’s some kind of a job description or a checklist. You have to do this, you have to do that, but in reality when you’re talking with people, it often is the case that there is a more flexible mindset. You know, if you’re bringing value to the table, people are interested in, well, how do we turn this into something that makes sense for you? And I believe that’s really true at the, at the junior level as well. That’s not just a fact about the senior level when they look at somebody and say, ah, this person is good. It’s not, this person is good. They are this particularly shaped jigsaw puzzle piece and I’m gonna slop them in. It’s not about the shape of the jigsaw puzzle piece, right? It’s about the person who’s good. How do we take the jigsaw puzzle and the piece and make them fit together?

Jason Baldridge
12:19

That’s a great point. Um, in fact, I think a lot of junior people underestimate their ability to guide and dictate what they’re gonna do by showing their leadership early on. Exactly.

Waleed Ammar
12:28

Yeah, that makes a lot sense. I remember when I joined AI2 I was expected to do one job and just like a few months later, I ended up starting like leading a team. And then I gave a course at the University of Washington and I don’t know, I started this NLP podcast with Matt. It’s like, yeah, it’s just like things are evolving in strange ways. So what are other aspects, you already talked about flexibility a fair amount. Can we talk about finance? How does this decision of working in industry or academia affects one’s finances?

Jason Baldridge
13:01

Well, it’s a pretty clear difference. I would say. I mean as a professor you can consult and many professors increase their income by doing consulting and that can bring a lot of additional income to the table. As a person working on consulting, a lot of times with Philip, I was able to earn a substantial portion of my income by working one day a week compared to my full time job as a professor. And that says a lot towards the kind of freedom that you can have for yourself and your family. And that mattered a lot for my family where my wife is a stay at home mom and you know, I actually have four kids, so that’s not cheap. So those kinds of factors can definitely make a difference. The problem is is if you’re being a professor and doing consulting, these are two jobs basically.

Jason Baldridge
13:43

Right, and that means there’s a lot of your time that goes to that and leaves less time for other things such as being with your family. So that’s the additional cost you pay there. I am, I can say that I’m very happy now at Google with one job.

Matt Gardner
13:54

Yeah. I was going to ask if universities are typically okay with you spending a day a week or do you have to do that on top of all of your normal five days or seven days or however much time you spend as a professor?

Jason Baldridge
14:07

Well, UT Austin gives you one day a week that you can do that. Of course is not like your Friday, that’s your Saturday.

Philip Resnik
14:12

In general universities, at least the major research universities are okay with, and even encouraging, of people spending a day a week without asking it. And look, they know that if you’re the kind of person who’s going to go and put in, you know, a day of your time doing this other stuff, chances are you’re going to be devoting a lot of energy to the academic side, also. When I worked at Sun, I discovered early on that they didn’t care if I like, you know, got a week’s worth of work done on Monday and then went away the rest of the week. And this turned out to be a smart decision because people who get, I wasn’t getting a week’s worth of work, but you know, let’s just suppose, you know, people who get their week’s worth of work done on Monday, are not gonna go away and like play golf the other four days.

Philip Resnik
14:55

They’re going to come back and do another week’s worth of work on Tuesday and then do it again on Wednesday. Right? So universities know that by keeping the flexibility there for people, they’re actually doing themselves a service by attracting the kinds of people who are interested in, putting in effort and not viewing the world as an either or, because look, if you are looking at, if what you really cared about was the outside stuff, at least in our field you would go work outside of academia. Right. And so I think that they’re very comfortable and also, universities are very encouraging of the industry relationships that get created there. Whether it’s something, a larger research presence like Google or Amazon or you know, the list goes on, or other kinds of companies. Universities want an ecosystem where their researchers are interacting with the people who value research and who hire students and who fund research and who are consumers of the ideas and so forth.

Philip Resnik
15:57

You know, the other thing that I would add, David Sedaris, the essayist, he writes a lot of funny stuff. In one of his essays, he talks about, sort of, the four burners theory. It’s, it’s worth mentioning if people haven’t heard of it. He actually got the idea from somebody else, but he articulates it really well. You imagine your life as a stove with four burners on it. There’s work, there’s family, there is health and there’s friends and this is a drastic oversimplification of what life is, but it can be really useful for putting things in perspective. And the idea is if you want to be successful at something, you figure out which one of those burners you’re going to turn off or turn way down. If you want to be really successful, you figure out which two. Now, as I said, this is a drastic oversimplification, but anybody I’ve ever talked with about this can immediately identify which burners are the ones that they tend to turn down and have on lower. And so it’s looking at these different job opportunities in industry, academia, startups with your four burners in mind and looking at it. And startups are particularly complicated because you have to distinguish between founding a startup and working in a startup. Founding a startup as Jason and I have both done, is a very different game where you are essentially married to the startup. And so, that idea that you’re doing a startup and other stuff, just to let you know as a founder it’s a much more tenuous proposition.

Jason Baldridge
17:20

Yup. I want to return a little bit to the personal perspective on the financial question, which is it’s not about the absolute numbers, it’s about the financial security that you experience. Personally, like, I would’ve been just fine if I had been on my own working on my university salary and I would basically have been just fine; happy with all this stuff. But, family stuff did enter into the question and I’ll give you one example of where I actually didn’t ask for more financial security or increase of my pay from the university, but they didn’t have my back on something that really mattered. And this was when, my son who’s now 10, I needed to have daycare for him and I wanted that to be at the university. So I actually was responsible. I signed up to get him into the daycare thing

Jason Baldridge
18:03

well before he was born and the actual admit into the universities day care didn’t come until he was like two years old. And this happened at a time, this was like around the 2008 downturn in the economy, my wife had lost her job. We’d sold one of our cars. I was taking the bus in with my son to another daycare that was further away from my office. So I was walking, you know, a mile plus to my office. That’s not such a big deal. I like walking. On the other hand, that meant that I couldn’t easily get to him if something happened and just also would have been really nice to have had him right nearby, at the university. So I just asked like, Hey, can we get the dean to approve some kind of thing for me to get moved up in this darn list?

Jason Baldridge
18:42

Um, I was responsible in all this stuff and they couldn’t do it. The only way they could’ve done it is if I had had a competing offer from another university. And this is often the case with uh, universities. They only, kind of, will bring more to the table if you have a competing offer. That was actually the point at which I talked to my wife and she’s like, sorry, but like I’m not a fan of your university and we need to find some other options. And that’s when I really started looking around for other, kind of, consulting things and that one thing led to another. If that hadn’t happened, I might still be at UT Austin today. I don’t know. Like, you never know what the possibilities are. But that was like an example of where my overall well-being was not, kind of, being taken care of.

Jason Baldridge
19:18

And I was asking for something that I felt was simple and it turns out daycare’s not a simple thing at universities, but go figure. So overall like you know, there’s sort of your personal sense of well-being. Another thing is the security of the people who work with you and often who are either reporting to you or who you are advising. And one of the things that I’ll say has been really nice, since being at Google, is knowing that all the people on my team are well taken care of. Whereas, as a professor, I was often, you know, semester by semester, trying to figure out, oh, am I going to be able to get funding for this person? And you know, they have to borrow money to pay for their tuition because the university messed up and getting their payments and all these kinds of things. And at the startup I was worried about, would we be able to pay them next month? Right. So that’s been a very nice thing in terms of my own mental well-being and respect to the overall financial security of my team.

Philip Resnik
20:05

Staying on the finances one a little bit more, I guess one of the things that comes to mind for me also is why do people do what they do? You don’t get a phd for the money. I mean, you don’t take a grad student’s stipend, right, for five years, you lose five years worth of the kind of income that you could earn for the money for the financial gain. Right? And the jobs that you get with a phd other than the, Hey, I’m going to do a startup and become a zillionaire, you know, with, you know, all the attendant risks and the fact that, you know, 90 whatever percent of those are gonna fail. You’re not gonna make that back in some sense. You can’t plan on making that back in some sense any more than I could have planned to, you know, stay in industry for 10 years and then somehow, you know, assume I was going to be a big enough name to come back into academia.

Philip Resnik
20:52

You can’t, plan that. Right? And so that’s not why you do it. So, the sense of security that you’re mentioning, Jason, is very much more important than the exact dollar amounts. I think that’s actually a really important point. The flexibility to do things outside academia to supplement can help make that work as it has for me too. But, people go into academia and research in general, really, because they want to work on interesting problems that might have an effect on something and work with cool, smart people they respect. Right. I’ve just listed off a bunch of reasons that people do stuff. Money is not even on that top part of that list. You can take a financial hit in terms of opportunity costs, but if the big house was what you were after, you’re probably not the kind of person to have been getting a phd in this field in the first place.

Waleed Ammar
21:47

Fair enough. But, I will say that even getting a small house is not easy these days.

Philip Resnik
21:52

Oh yeah.

Jason Baldridge
21:52

It can be tricky in many of the markets.

Waleed Ammar
21:54

Oh yeah. Thank you very much. I think that’s a very helpful perspective on both sides. One of the main answers I get whenever I ask some of my friends who are in the job market for a faculty position of why are you applying for faculty positions, is intellectual freedom. And it, it’s like related to flexibility, but it’s also different in certain ways. So I wanted to get your opinion on whether this is actually the case. If you think actually as a professor, you feel like you actually have more intellectual freedom, or not.

Philip Resnik
22:26

Well, so this goes back to what I was saying earlier about value right, there is, you always need to deliver something to someone. When students come in, I always talk about sort of the Venn diagram where one bubble is what you want to work on and the other bubble is what somebody will pay you to work on. And the goal is to make those two circles overlap as much as possible. Whether you’re in industry or at a startup or wherever, or in academia, you know, unless you’re independently wealthy or have a patron, you know, the de’ Medici’s, are covering your costs, you’re not going to have complete freedom. That said, you have a kind of control in academia that you don’t have in industry. I would argue, I have argued that academia is actually more entrepreneurial than entrepreneurship because when you’re an entrepreneur, and Jason, you’ve experienced this as well as I have, right?

Philip Resnik
23:21

You’re coming up with an idea and then you have to figure out that it has value and convince somebody to support you to do it, right? And you kind of take that summit piece of new stuff and move it up a curve in some way, right? But then there comes a point in entrepreneurship where that’s not what you’re doing anymore. What you’re doing is now turning that into something sustainable, right? You’re getting the engineering locked down and the sales and the, I mean a lot of that happens along the way. But there comes a point when the core innovation is not as much what it’s about and if you’re really, really lucky, you can be at a company that is continuing to do core innovation while doing that, but that’s really, really hard. In academia, you do that whole first part all the time, come up with a new idea, try to convince somebody that they should give you money to work on it or find a way to work on it out of love for it, even if you don’t have the money, get people interested in being involved in.

Philip Resnik
24:14

And then you take it up to the top of the curve and instead of doing that whole sustainability thing, you turn around and go back down to the next idea and do it again. That is a kind of control that entrepreneurs get in a way that is highly risky, but that academics get in a salaried job. Nobody’s going to just pay you, you’re here forever; you have to keep delivering. Whether you’re assistant or associate professor, or full, there’s always a paying the piper that has to take place, but that cycle of I have something, I have control over it. All I need to do is convince other people it’s worth doing that. That’s something that comes in academia without the same kind of risk or necessarily the same kind of barriers to that that you often see in industry.

Jason Baldridge
24:54

Yeah, that’s, that’s absolutely true. I will say industry can come along with opportunities to have significant flexibility. One big difference is that academia, you get it for life if you stick through it, right? So you know, there are heydays for industry labs like AT&T Bell Labs and things like that where things are great for a long time and then you know they kind of closed down. So there’s always the chance that you get that good window where something, kind of, the getting is good for thinking about doing whatever kind of research you’re really interested in. And I think for that, if you’re looking at an industry place, a really important thing is to consider who are the senior leaders? And for me, I was looking at Amazon versus Google and there were a lot of very great positives about Amazon, but I did know Fernando Pereira and I had a ton of trust in his judgment and how he thinks about research and his experience in industry and in research. So, that for me, was a huge factor in my decision to join Google, apart from other factors. And so that’s one thing, key thing, I would sort of look to. And that has turned out to be true. My, intuitions around this and have found that to be very true. I mean Fernando has been pushing me to solve harder and harder problems and ones that I’m really interested in.

Waleed Ammar
25:58

So is this true for you as well as your direct reports or do you feel like it’s more true for one or the other?

Jason Baldridge
26:03

So we definitely do not have the model where people, say who finished a phd and join our research team are told what to do. That is not usually a good way to work with researchers. We do have projects where there’s some kind of scope that’s been outlined by a senior researcher and that’s often within the realm of some sort of broader set of goals that we’d like to set as an organization. And basically, younger researchers have, or more junior researchers, I should say, have the opportunity to join and choose which ones they want to really focus on, which ones they think are the most promising and that they’ll be a good fit for. And we don’t have any kind of thing like you’re stuck on this project for years on end. So people have a lot of freedom to move from one to another, depending on whether they think the problem is good, the solutions people are approaching are good, the tooling is right, and the team members are people they want to work with. So, I would say within our Google research group, there’s a ton of flexibility in that respect. Kind of going back to what we said earlier, there are tons of opportunities for junior folks to hit at a higher level, I would say, where they are doing more definition, they’re scoping out projects and things like that. You don’t have to do that. But if you do do that, that turns out to be awesome.

Philip Resnik
27:12

So I want to introduce an element of, maybe skepticism is too strong, but at least a distinction between what you have at Google, and this may even apply to Google, as well, in other places. I draw a distinction between research, on the one hand, R&D as traditionally construed and what I would call advanced development, right? And then further along that spectrum you have development, right? Somebody’s giving you, you know, a set of requirements and you’re developing it. The difference between these two is in what it is that’s being valued coming out of it, which by its nature provides the incentive and control over what you’re doing in advanced development groups, which sometimes, and you see a mixture of, some research groups are going to have more of a flavor of advanced development.

Philip Resnik
27:56

Some, you know, development groups will have maybe a little bit more flexibility for research and 15% projects or 20% projects or whatever. But in an advanced development setting, you have to justify what it is you’re doing on the basis of the business need. It’s advanced, meaning you’re on the leading edge or advanced edge of something that is going to deliver value to the company, not just a paper. Right. Whereas R&D isn’t necessarily going to do that. It might deliver a patent, it might deliver a published paper, it might deliver prestige. Right. And it’s pure research. The purest form of research is still relatively rarefied. I think there are a small number of places where it’s really happening and even where it’s really happening, and you can probably speak to this better than I can. Uh, Jason, at least as far as Google is concerned, you know, to what extent does that influence what you call intellectual freedom, right? That, what it is that you’re going to be working on. The piper always needs to be paid one way or another. Maybe I’ll go back to you, Jason. I’m interested in your thoughts on this, sort of, like, this balance between justifying something because it connects to a business need vs. this is a really good idea. I get to work on it. It might or might not actually deliver something in the sense of technological transfer.

Jason Baldridge
29:08

What I can say is that I’m very happy with the situation that I find myself in, with respect to intellectual freedom. There are various incentive structures, which I think you’re alluding to, which is how are you evaluated for promotions and, you know, your performance ratings and this happens in academia. It happens in industry where, you know, there are some number of things that you are doing that people will look at and evaluate to see, like did you do a good job? Right? And it is the case in research and industry at least at Google and I assume elsewhere that doing things like writing papers that are influential in shaping how the field is going forward, are actually considered kind of collateral that matters for these kinds of decisions. So, to the extent that you look at the incentive structures, I think a lot of the incentives that I’m experiencing are very similar to what you experience

Jason Baldridge
29:57

as an academic, say as a professor in a department, or a postdoc or something like that. There are plenty of opportunities at these larger companies, I would say, where you’re not focusing on a specific product need. Google, in the past has had a lot of focus on things that would be research that would be applied toward products. And I think that’s been really healthy for Google, overall. It’s led to a lot better integration of ideas from the research groups going into improving Google products. That said, there’s actually been a shift in recent times to focus on more open ended problems because things like natural language processing turn out to be really hard and if you’re, there’s always these problems where you can kind of look and go like, Whoa, if we

Jason Baldridge
30:31

can just get a bit more data, if we just tweak this model a little bit and then we run the right experiments, we’re going to get something that will improve something that matters to the company, but if you do that, you maybe miss some opportunities to take a longer term perspective that could lead towards larger improvements overall.

Philip Resnik
30:46

This leads to a question you should be asking if you’re thinking about industry, which is what is the tech transfer model, right? Even if you are doing more basic research, what is the model for how that research actually that gets deployed by the company? Do you need to be coding production level code? If not, do you still need to be coding in a language such that your code can be used and adapted or are you just doing a reference implementation such that some engineer is going to translate this. If some engineer is going to translate this, where are you? Do you stay in your group and you’re available to get questions asked of you or do they detail you to go over for, you know, a quarter or a or half a year to be there with the engineering team? There are lots of different models for that. If you’re going to do research in industry, you want to understand what the model for this is.

Jason Baldridge
31:32

That’s absolutely correct.

Matt Gardner
31:33

Can I jump in and refine this question a little bit? So let’s say I am someone graduating with a phd today and I’m on the faculty market and I’m also considering positions at say, Google research or Fair or some other research in industry. What would you say are the practical differences in my job description in the first year?

Philip Resnik
31:52

The immediately obvious thing is in your first year as a faculty member, you are incredibly focused on publishing. If your dissertation stuff was not already published that it’s getting out and building your, sort of, momentum on a research track, creating your research identity.

Jason Baldridge
32:08

Creating your courses and teaching them.

Philip Resnik
32:10

I was going right to their next right. You’re just splitting your effort so that you are involved in that, in the teaching side. If you are as lucky as I think incoming academics should be, then you’re not spending as much time on service. Right. In academia, you sort of have this tripartite distinction between research, teaching and service in order of importance for tenure, and it is not a linear decrease from one to the next. Um, right. This goes back to what you said about Fernando and having good leaders and people, right, who are going to help, you know, set an environment the way that it should be. In an academic position,

Philip Resnik
32:42

if you are looking at an academic position, one of the things you want to understand and ask about, of junior faculty members, or look at the life experience there is to what extent do they have your back? Are they protecting you from being on unnecessary committees? Um, are they enabling you to say no when you need to say no? Um, are they enabling you to focus on the research that is going to move you forward in the way that you need to, as opposed to whatever else is going on, whether it’s a big project that’s running in somebody’s fiefdom or participating in a team to do collaborative stuff, to get some, you know, next star behind a grant. Just a huge difference in what’s happening in your first year in industry. You know, you’re like, hey, this is cool. You’re spending most of your time hands on keyboard developing new stuff. Right. Not consolidating your old stuff of getting on the track the rest of the way.

Jason Baldridge
33:35

Yeah, and that’s absolutely true. Although many people do still carry on the collaborations that they had started during their PhDs and still have the ability to wrap those up, you know, as, as it were, um, after joining an industry position.

Philip Resnik
33:48

I absolutely agree with what you say. My first year at Sun actually was actually very postdoc-l.ike I was very fortunate that way. But I think that there’s probably a wide range of variation across the different main companies that are doing research.

Jason Baldridge
34:00

Yeah. I would like to stress as well, there are differences in being a person doing research in a more product focused setting, which exists both at Google and elsewhere or in a sort of research lab setting. Right? And these opportunities exist in various industry settings. When you go to a more product oriented thing, you probably will be doing more just direct coding, less emphasis on writing papers and all that kind of stuff. One of the things I do think is worth sort of pointing out is a lot of our discussions so far has been kind of the, you’re in a research lab in industry versus you’re going to be in academia, maybe a professor or a postdoc or whatever. But oddly enough, a number of people join research labs and then move into product. And that’s an interesting choice.

Jason Baldridge
34:46

And there are a lot of good reasons for that. And one of those is that a lot of product problems are really darn interesting and then you have like these really rich problems, you have very clear things that you can improve on and a lot of people find that really engaging and exciting, for good reason. And one of the things that I sometimes sort of end up talking to a lot of people who are interested in these kinds of questions is, don’t close off that possibility because often you can get an opportunity, to be, to join a product oriented research group at one of these companies. And I think people sometimes write those off too quickly, but there’s a lot of really interesting things around that and you should talk to some of the people who maybe even made that switch after having joined the research lab in industry and then gone to product area within the same company.

Waleed Ammar
35:29

Yeah, I think it goes back to the value proposition that Philip was mentioning earlier. Even individuals want to see that the work they’re doing results in value for the society and different people value the concrete implementation and concrete heavy concrete solution for a problem, uh, out there. People can use it.

Philip Resnik
35:47

Yeah. I want to actually emphasize this stuff that you’re talking about, Jason. We really should be doing more of it in academia, right? If the kind of person that you’re talking about who is moving from a research group into it, into a product group, is somebody who is going to need to understand a need, right? The problem that they’re trying to solve, the real world problem that they’re trying to solve and connect that with the technological solutions. And sometimes that doesn’t always mean using the most up-to-date and shiniest technological solution for every part of the problem that you’re trying to solve. In academia, personal opinion, too many people doing PhD’s are focused on the methods and trying to increase this, wow. This, let me see, let me see if I can tweak this technique or come up with a new technique to do this without actually thinking or possibly I may be overgeneralizing a little bit caring about like what’s the problem I’m doing?

Philip Resnik
36:40

There are some conferences, I don’t know, some machine learning conferences, I can think of for example, where you know it may be a caricature but it’s like, hey look at this cool new model. Well what problem is it solving? Well who really gives a shit? It’s a really cool new model and the thinking that you’re talking about in industry that is connected with, you know, the researcher who has a connection to the product side and is thinking about the needs is something that I think would benefit the academic research as well. So maybe as you say, you know, hey, don’t rule that out in advance. Maybe you need to go a step further and say, hey, let’s actually rule that in, in terms of what it is that we are trying to get the people in academia to pay attention to, also.

Waleed Ammar
37:18

Thank you. That was very helpful discussion of different aspects that people can think about. I want to make sure that we’re, not spending all the time thinking about, talking about one, one aspect. So what are other important aspects that we haven’t talked about so far?

Matt Gardner
37:33

A career progression. I guess there are a lot of different end goals here, but one possible end goal is to be a well known researcher. I Dunno how other people think of this, but I kind of think of academic as kind of like a pyramid scheme where you get more people under you and eventually you make it to the top or close to the top. How does academia versus industry effect this?

Jason Baldridge
37:50

I don’t know. I’d almost like to say like the goal should be like you want to solve certain problems or certain classes of problems and then that should be your north star rather than, you know, some of these other things and then that may come out of the efforts that you do to solve these long term problems. And like one thing that comes to mind is Mark Seedman’s general way of going about things. And he’s my advisor and I’m a little bit biased, but I’ve always been impressed by the fact that he is usually thinking a decade or more ahead of what many other people are thinking about. And he’s always been true to his interest in the connection between computation language and psychology. And he’s sort of constantly monitoring those things and everything he does. And he’s also tireless in sort of reading the literature in all of these spaces.

Jason Baldridge
38:32

And so what I would actually sort of say is let’s turn that around a little bit and be very reflective about what your research program is and what problems you want to solve. And then you can have flexibility within that if you’re doing a good job along the way, you have flexibility in either context, whether it’s academia or industry, to make progress in some of those things. You’ll have more freedom as a professor to define your longterm research arc. But that’s not to say you can’t kind of piece it together through some mixture.

Matt Gardner
38:59

Yeah, great answer.

Philip Resnik
39:00

Yeah, absolutely. You need to actually figure out what you want first. Um, you need to have your goals in mind and then let the goals drive the decision making. And that is a very general statement, but it’s often, surprisingly often, kind of missed in my experience when I’m talking with people about, you know, well why are you doing this? Why do you want to do that? There is a multivariant optimization problem that goes on where you have to understand what the weights are of the different things that, that, that mattered to you. And actually one of the things you want to do when you start a project, if you’re leading a project actually that relates to this, which is to go around the table and ask each person what is it that you’re in it for? For some people it is, it’s to become well known.

Philip Resnik
39:41

For other people, they don’t really care if anybody knows who they are as long as lots and lots of people are actually using the thing they built. Right? There is a whole spectrum of goals that people bring to the table. It’s not all about being recognized and well known. As for the pyramid scheme. Yeah. I, if I recall correctly, there’s actually a chapter in Freakonomics, the book Freakonomics, that, that, uh, talks about both academia and, uh, and drug gangs as analogous to each other where you have people at the bottom of the pyramid putting up with all kinds of awfulness. And I mean, we have it easy compared to somebody who’s trying to get a phd in English literature. Right. And why do people go through all of this at the bottom? Whether it’s like, you know, being on a street corner or you know, teaching for, you know, a pittance.

Philip Resnik
40:24

It’s because they, they think that I’m going to be the one that makes it up to the top of the pyramid. You know, Mark is a wonderful example. Uh, you know, exemplar for me, he wasn’t my advisor, but when I was at Penn I, it was kind of part of my advisory cloud there. And he is a, you know, a fantastic example of what the top of the pyramid looks like. There are good examples all the way through, at all of these different levels. So you have to, you have to recognize, yeah, there’s a, there’s definitely a pyramid scheme element to it and you ultimately have to balance the risk that you’re willing to take or the things you’re willing to give up for the actual expected value of what you’re going to wind up with, which is the value of it, but also the probability that you’re going to make it there.

Jason Baldridge
41:05

I would add another thing in this, which is after being a professor for over a decade and then doing a startup and now being at Google, one of the things that really stuck out to me is when I worked on the startup, we are actually doing a lot of model development research things that we could have published on and I realized that a lot of that work that I was doing was far better than the stuff I was doing at the university because I had just a bit more budget. I had just a bit more focus. I had a kind of various other factors that allowed me to, I was beating some sate of the art stuff from the literature though I wasn’t publishing it and I realized, you know what, if I can kind of knock this stuff out more easily at a startup, what does that say about doing that work as a professor, some of which I did and I actually kind of reflected that I actually didn’t deal with as thorny longterm problems as professor as I wish I had and one of the kind of things I’m excited about at Google is I actually feel like I’m able to revisit some of those deeper longterm problems for the time being.

Jason Baldridge
42:01

And so that’s been really exciting for me, but I would sort of encourage anybody who’s in an academic position to not get caught up in the cycle of, you know, getting papers out and things like that and to instead have that more longterm kind of view. Right. We actually had a little Twitter back and forth today about some of these topics around publishing an archive, and I do worry about the amount of publications that are out there and there’s a usual story about signal to noise and the incentive structures that we have to pay attention to to get our promotions or to get accepted to the university at all to do a phd. My Gosh, I’m still flabbergasted by this, but if you are in academia, take that long perspective please, please, please. Because that’s the place that should be the sacrosanct area to take the long perspective.

Philip Resnik
42:45

Oh man, it should be, but the reality is so hard. I’m on sabbatical right now and what I mean, what is the point of a sabbatical? One of the traditional points of a sabbatical is to take a step back and refresh and I’m doing that after a really, really long time and looking back and assessing what I was doing and having recognized various treadmills that I was on. You know there’s a government funding treadmill that you have to be very, very careful about with some of the agency sorts of funding, your DARPA and IARPA and, some of that stuff where, you know, the cycle time of the projects and the cycle time of the students is out of sync. And so you know, you get money to do something, you have things you need to deliver. Oops, I need students.

Philip Resnik
43:27

And so you get the students and then the project ends and now you need to keep those students funded as you were talking about earlier, Jason. So you do what you need to and so now, you right, do you have, you need money, you need students, you need students, you need money and you go and go and after awhile you’ve forgotten why you’re doing it in the first place. Particularly for some kinds of funding where the balance between the research and the delivery, the, I’m going to participate in this big shared evaluation or whatever is less than what you’d like it to be. I mean, I’m an audience for what you’re saying. Please somebody in academia, make sure you’re thinking about not just the stuff you know in the way that you’re talking about. I can tell you it’s hard. I mean, I’m doing a reboot myself in terms of what I’ve been looking at. I have been doing that kind of thing over the past, you know, whatever, five years, a lot of stuff during sabbatical. All of a sudden I’m looking at, you know, brain stuff when you know the sabbatical, which I never did before, which is wonderful, but it’s really hard and it always comes back to um, to paying the piper.

Jason Baldridge
44:22

There was a really interesting argument I read a couple of years ago about looking at academia as now a mature industry. Where, in the early 19 hundreds, mid 19 hundreds and so on, it was essentially a startup sort of a situation which was getting tons of funding from the government. And that meant that people could come in, be smart, think deep thoughts, like they could have lots of time just to really focus on their work. And toward the end of the 19 hundreds it was much more harder to get funding. You had to do a lot more of this treadmill that Philip’s talking about and leaving less and less time for doing that deep work. That is what academia should focus on and what academics do best. And one of the things that gets in to that, as well as if you have senior faculty who say became professors in the sixties or seventies they have a very different set of expectations about what the funding story looks like for a young faculty. Right? And then you end up with these generational gaps between expectations about, well, why aren’t you getting funding? Right? You know, like you should be able to get this. Well, it’s a different story today. It’s harder, like there’s less overall stuff being thrown around or there are more people or both.

Philip Resnik
45:26

So there’s a show that you guys should do on the evolution of academia in this regard and sort of where research is taking place and the shift of a lot of the research activity into industry and what the pros and cons of it are. I’m frankly really worried because if it comes down to the incentives and what it is that is supporting the research, whose needs are being supported, companies, even great companies that do wonderful research have a bottom line that’s defined in a very different way from the traditional definition of the bottom line incentives in academia. And so we’re seeing an enormous evolution in our fields in terms of where the research is taking place and what’s driving it. We’re clearly not going to cover all of that in this conversation, but the stuff that you were just talking about Jason, I think is really the beginning of, of that very interesting conversation.

Waleed Ammar
46:14

One of the things that are really important I think is stress level, which we kind of mentioned in passing but didn’t spend a lot of time talking about.

Philip Resnik
46:21

I would refer people to Charles Sutton’s really wonderful session on this podcast about stress. I think that the discussion that you guys had with him was an enormously valuable one. My own thing that I would add, uh, especially since I’m married to a psychologist, um, is that people should recognize, particularly in an academic environment, whether you’re a student or a postdoc or faculty, there are people whose job it is 24 hours a day to help you deal with what you need to deal with. Right? And so people need to recognize that the university ecosystem is one where there are in fact a lot of resources and simply saying, this is what life is like, I’m going to suck it up. It’s awful in terms of stress or depression or problems that people are having. People need to be aware that, there are a lot of resources. And as faculty it’s partly our job to make sure that if somebody needs to get themselves, you know, to a resource that can help them out, we need to make sure that we can help them do it.

Jason Baldridge
47:24

Yeah. And I mean there are cultural differences that I will say, in the kind of overall theme of our academia versus industry discussion here where certainly in academia, as Philip mentioned, you can choose, you have a lot of flexibility to work whatever 90 hour work week you want. And we do tend to do that for ourselves in part because many people who are in this are overachievers and you’re also really passionate about what you’re doing. You enjoy what you’re doing. I often say, hey, this NLP stuff is my hobby. Like I don’t build train sets. I like to do NLP. Um, so I personally will enjoy doing NLP on a Saturday. That’s all right. And in many ways now on the other hand, I have a family and I like to spend time with my wife and my kids and things like that.

Jason Baldridge
48:04

And so, one thing I have found is that in my role at Google in particular, I am much more able to keep things into a Monday through Friday sort of setting and Philip is making a face. Sorry. But yeah and I work, you know, good long hours during those days and I often do read papers in the evening or do things like that. But, and when it’s a ACL deadline, my wife knows that Saturday and Sunday are going to be gone that weekend. We just talk about it in advance. But overall we do have a focus on not having people expecting things of each other on Saturdays and Sundays for example. There is this idea that you do need to have some time to charge back up and also to get sleep. I really recommend this book, why we sleep. I have been so much more protective of my sleep since reading it.

Jason Baldridge
48:49

An,d you will be too if you read it. So read Why We Sleep. And that’s actually a big factor in reducing your stress no matter what situation you’re in. And the final thing is a lot of it comes down to your choices and one of the biggest choices you have is how you spend your time. And in that context, we all know that doing deep work and thinking deeply about problems and not doing things like answering email are the things that we should be doing as researchers. And you should try to find ways to prioritize that if you’re not. And so there’s this book by Cal Newport called Deep Work where he basically just drills in this thing that we kind of all know, but he drills it in, in a way that helps remind you of this. And I just remember Philip, one time you, one of your sons sort of thought that your job was to answer email, right? You remember that story and you’re like, oh no, I have to like change what my son’s perception of what I do is, but we do tend to answer a lot of email as academics.

Philip Resnik
49:40

We’re driven by it. I mean, how many times have you had the experience where you sit down and like, okay, I’m just going to take care of the most important email and then get to this stuff that I’m intending to do today. And then, you know, like you wake up and it’s dinner time or past it, and you’ve forgotten to eat and you’ve got, right. And that’s a real problem. I agree that industry, I will distinguish that from startups and I will distinguish that from founding a startup. That’s a whole different game. Right? But it is true that industry does make it easier to draw those clear boundaries. It is also true that it is possible to do better in academia than most of us are doing. On sabbatical one of my goals has been to move toward the reality that you’re describing instead of my typical reality, like actually reading papers and articles as opposed to just trying to gulp down the most important stuff. So that I’m aware of what’s going on, trying to synthesize stuff, trying to actually just have real conversations and so forth.

Jason Baldridge
50:36

One of the things is if you are in a relationship or you have close people close by who are affected by your academic addiction, pay attention to that. Right? And my wife frankly, was not a huge fan of academia. And the kind of mixture of intensity of research plus the last minute preparations for teaching the class that you weren’t quite ready for tomorrow or the homework that you didn’t quite have ready, all those sorts of things. Those all frankly soured her a little bit on the idea of me being a professor and continuation. I think I could manage that much better now. But you know, young assistant professor, the kind of doing my thing, run, run, run. I burned a few bridges there with my wife on that one.

Philip Resnik
51:14

I agree, you have to pay a lot of attention. But all of this stuff we were talking about when we were talking about goals and how you make decisions and you’re setting priorities and so forth. One needs to realize that is a joint decision making. It needs to be a holistic joint decision making. There’s give and there’s take and not everybody is going to, you know, get everything at one point or another. But, you absolutely have to take that into account when you’re picking things up.

Jason Baldridge
51:42

And to be clear I didn’t burn bridges with my wife. I burned bridges between her and academia. And that kind of meant she really encouraged me to keep pushing and looking for things outside of it. So, we’re good. I listened to her, but yeah, she would not. Yeah.

Waleed Ammar
51:55

Anything else that you feel like we missed in this conversation? I did want to bring up that there is gonna be a panel about careers in NAACL. What’s the day? It’s gonna be a month,

Philip Resnik
52:05

Right, So I am actually going to be moderating a panel on careers in NLP. That’s part of the industry track at NAACL. NAACL is in Minneapolis of course, the week of June 3rd, the panel is, I believe it’s the first day of the main conference on Monday, June 3rd, I believe at 1:00 PM, check the program. This is a, a reprise of a similar panel that we did last year that Jason was on. Go check out the, NAACL blog for information about it and the, and the panelists. And, you know, you’ve already gotten some of two peoples’ perspective here and I promise I won’t talk as much as moderator of the panel. I’ll be helping to make sure that the panelists get stuff out also. But we’re very much looking forward to doing this and it’s not definitely not going to be a repeat of what you’ve heard in this podcast. I think it’s going to simply take it to the next step. So, people come.

Jason Baldridge
52:58

Just to kind of wrap up, I think internships are a great way of testing industry waters and I would say furthermore to the point that I made earlier, don’t just consider doing internships with the research labs. Also consider doing them in product teams where you’ll have a chance to work on research and development around product. That will really give you a sense of what is it like to code at these companies and to solve these kinds of problems where you’re really focused on a particular problem area and Dataset or application.

Matt Gardner
53:23

I’ll, jump in there. I did three internships at Google on products teams and that’s where I learned how to code well and this was like the basis for like the code quality in NLP that a lot of people have commented on. So I would say that these actually helped more than just getting experience. It also helps with your ability to do good research in the future.

Jason Baldridge
53:44

Absolutely. Absolutely. The other thing is no matter where you go, industry or academia, don’t ever forget about your personal life, your friends, your family and things like that. I have talked to senior folks who have kind of turned around and looked back and said, wait, I’ve done all this stuff. I’ve written all these papers, but I feel a little bit alone now. So never forget about yourself, your personal self and cultivating relationships with people. Don’t always chase the papers. Chase your own personal happiness as well.

Philip Resnik
54:12

That’s absolutely right. One thing that will help do that is to just be conscious. We spend so much of our time doing things because, well that seems like the thing to do. Why are you going for, you know, applying for this, uh, for this funding? Well, because it’s there and might somehow connect as opposed to, I really am interested in doing the work that I would do on this thing. Pay attention to those four burners and actually reflect on which ones you’re turning up and which ones you’re turning down at various points. I could not agree more, Jason.

Waleed Ammar
54:45

All right. Thank you so much for joining us today. That was a really fun conversation and I’m sure a lot people will enjoy listening to it.

Philip Resnik
54:51

Thanks so much.

Jason Baldridge
54:52

Thank you so much for having us.