“As you’re approaching your career, don’t spend too much time planning because life happens in unexpected ways. Maybe you have figured this out already, but I think the skill is to create opportunities. When I say create opportunities it does not mean strategically looking to advance in a career but to do good work for a lot of people. They will remember you, and it will be useful.”
This month’s Stern Faculty Spotlight features Claudia Perlich, data scientist at Two Sigma and adjunct professor at Stern who teaches Data Science for Business (Managerial). Claudia started her career in Data Science at the IBM T.J. Watson Research Center, concentrating on research in data analytics and machine learning for complex real-world domains and applications. She received her Ph.D. in Information Systems from Stern School of Business and has over 50 scientific publications to her name as well as a few patents in the area of machine learning.
We’re going to start off with something fun. I saw that you have what you describe as “completely irrelevant fun facts” on your Linkedin profile. Can you share one with us?
Yes, I’ve always been totally crazy about horses. When I tried to figure out what to do with my life, a good friend told me you shouldn’t make your passion your work because it easily stops giving you that feeling once you have the constraint of having to make money with it. Keep your passion as something you can always go to.
The fun fact on my profile is that I adopted a project horse, a rescue, who was standing around in the stable where I started a share boarding just after I finished my PhD and also had a nine-month-old baby. During that time, I felt like I needed to get back into something that was me as I tried to balance work and family. His name is Moon Country but everybody just called him Moon. He’s a thoroughbred that is related to the Secretariat line. He’s not fast, but he certainly has the attitude of a champion. I had adopted him after about a year of working with him. I recently retired him, so now he spends all day walking around in green fields and eating with his buddies.
Tell us more about your career and the experiences that brought you to where you are today.
I grew up in East Germany. I was about 15 when the wall came down. I’ve always been pretty good at, and interested in, math and natural sciences. When I was trying to figure out what to do with my life, my dad suggested that I study something with computers. That was back in ‘92 and he said, ‘Look, computers will be everywhere in the future. If you don’t know right now what you want to do with your life, you’ll have plenty of choices later on.’ So I started my undergrad in computer science in Germany and found my way into an exchange program to attend CU-Boulder. Because I’d always been very interested in biology, I randomly walked into this course on artificial neural networks thinking it would be really fascinating to learn how the brain works. Little did I know that it is really a kind of AI and machine learning. I think I completely failed the entry exam that the guy handed out on the first day, a little test on what you knew about normal distributions and standard deviation and the like. I had no idea what any of that stuff meant, but I stuck it out and learned it, and got really excited about these algorithms that learn from data. It was early on, so we didn’t do anything super fundamental with them, but I found computer science to be a little too abstract for me sometimes and having data as a component and using it to learn something about the world really interested me. So I got into what now is called data science, but back then it was just some form of machine learning.
Ultimately, I went back to Germany and finished my German graduate degree there. But I really loved the American academic system. It’s an environment where you had professors who would actually volunteer to have a coffee with you. That doesn’t exist in Germany. It’s the kind of interaction that you get from the American system that led me to a PhD program here. I asked for a recommendation letter from the professor who had taught my course on artificial neural networks in Boulder. Little did I know that, in the meantime, he had gotten a position as a professor at NYU Stern. So, when I asked him for a recommendation letter he said, ‘Sure, I’m happy to write one, but why don’t you apply here?’ I said, ‘I’m a computer scientist. That’s a business school.’ And he wrote back, ‘So what? I’m a physicist.’ It felt like an opportunity that I couldn’t turn down, so I came to New York and studied for six years at NYU. During the last year of my PhD, I ended up having my first and only child so I took a little bit more time on it (you could either do five or six years). It worked out very well.
At that point, I was trying to figure out what to do with my life because I couldn’t avoid working any longer. The choice was between going into academia or industry. I was groomed for academia, but I had a really great time as an intern in IBM’s T.J. Watson Research Center in Westchester. My one industry offer was from this lab. I took it and became a research scientist, where you work on very applied problems and publish papers, so you’re kind of halfway academic, but you also work on a lot of business projects like the ones that I cover in my course. I also kept very close contact with my academic friends. At one point, my PhD advisor Foster Provost (who wrote the book on data science for business) asked me if I’d like to start teaching one of the sections of the course he’d developed and that’s how I got started with the adjunct teaching position.
What day jobs have you had since IBM?
After about six years, I’d been an advisor to a New York based startup in the advertising space and they were looking for a chief scientist. In particular, they had used some of my ideas from my PhD in their core targeting algorithm and they were very interested in talking to me. While I had a really great time at IBM, I felt like I would probably regret it if it didn’t give this whole startup thing a shot. I took the position and, from then on, spent a little less time on publishing papers and more on putting things into production. I also started to think a lot more about what it means for a company to use machine learning and data science strategically. It was a great time, just at the brink of when data science became the new kind of “it” profession, so to speak. There were a lot of super interesting problems to solve in that space because advertising is still today one of the areas where data is truly abundant.
I also organized a large machine learning conference in 2014 here in New York City called KDD. I put it under the umbrella of data science for social good. We tried to bring together folks that had thought really deeply about some of its application areas. I’m still closely involved with a number of organizations such as AI For Good out of Berkeley. That’s something that I’ve always been very interested in but didn’t have time to do full time. But it’s still something I spent a good amount of effort on.
Fast forward, after about eight years, I decided that I didn’t really want to spend all of my career in advertising. There’s an interesting balance when you think about your career and what you want to achieve. I was always very clear that I didn’t really want to be the CEO or the strategist. I like problem solving, so I want to have a role where I can still do what I love to do – to dig around in data. I needed to find some balance around being an individual contributor but still also having some range of influence through advisory roles and or a team. So I didn’t want to be Head of AI at Spotify; the position I ended up taking was actually in the financial space which I had not been engaged closely with before. I now work as a senior VP for strategic data science, with a small team doing incubation work on new financial models. I also have time and room to play around myself and to work with some ventures and private investments, so there’s a lot of interesting different directions that I get to take it.
Can you share a quick elevator pitch for your course, Data Science for Business (Managerial)?
I have been teaching Data Science for Managers as an adjunct for 10 years, maybe even longer. It’s really interesting to see the development of it over time. Initially, it was a pretty obscure elective. Students were mostly in the Information Systems department, but increasingly it attracted a lot more interest from folks like you, MBA students with much more broader interest coming from different backgrounds.
The course itself is trying to convey some core principles and lessons, like how data science is integrated in businesses to provide value. There’s some focus on the algorithms to the degree that folks understand what they can do, but what I’m really having a lot more fun with is putting it in perspective through sharing a lot of the projects that I did in my day jobs, whether this was at IBM, in advertising, or some of the kind of side volunteer work. I give more context for the challenges of actually translating a business problem into a data science problem. Also, I discuss how to bring together different human skills, from people with more technical background to those with a business background, to build a kind of shared vocabulary so you can actually manage these processes and set the right expectations. I’m trying to give enough technical knowledge for people to feel comfortable with that material, but it’s much more thinking about the bigger picture – implications on the business and a lot of ethical considerations around what can go wrong. And I also establish the vocabulary to translate what data science does, what it can do and what it can’t. I hope that’s consistent with what you think you learned there.
It definitely is. I loved your class. I just wish it wasn’t over zoom but that’s fine.
So I also saw on LinkedIn that you have over 50 scientific publications and you’ve won many data mining competitions and industry awards. I would love for you to talk about something you worked on that you either want to highlight or that you felt was particularly impactful.
I think one of the most interesting things that happened in a business context was when I worked in advertising. We were building targeting models where we were trying to automate how to pick which ad to show to which person. The model would predict who’s most likely to be interested in a particular product. The way these models work is you get data on people sometimes clicking on the ad after seeing it, going to the brand’s homepage or checking out something more about the product. We were trying to predict these kinds of events of people taking actions that would indicate intent.
Were trying to improve these models and, ultimately, over a short period of time, we observed that our models became really, really good. We had some fundamental intuition that you can only predict people so well. I can’t predict what I will do perfectly, let alone some AI. We got really suspicious about it and we started digging around. What we ended up finding was the digital symptoms of a new and very ingenious form of advertising fraud, where you saw these activities that looked like people that did very predictable things; for example, going to certain websites and clicking on ads, or following a link that was embedded in the ad to go to the brand’s homepage. Something just didn’t look right because the other websites they were visiting I had never heard of. It wasn’t the usual stuff you would expect, like YouTube, but sites like Hip Hop Hawk. It didn’t make any sense why anybody going to Hip Hop Hawk should then be interested in booking a hotel at the Hilton or looking at luxury cars. This was one of these detective stories that fascinated me and we found that all of these websites like Hip Hop were fake. No real person ever visited, but there were lots of ad spaces on the site and all the traffic that was going there was manufactured. What advertisers ended up getting was not real intentional interest.
Now we had this huge dilemma, what do you do next? The agency we worked with wasn’t particularly excited about the deep insight that we had blown maybe thousands of dollars on a campaign that looked great on paper, but really hadn’t reached anybody who was ultimately going to buy the product. I could fix it in the system, but then the campaign statistics would look terrible. And then the agency would be really unhappy because they compete for their clients. It took about three or four years, when I was giving talks about this in advertising conferences, until there was a broad acknowledgement of it and we finally received appreciation for it.
The bigger picture of this moral dilemma is that models are better than you would think at finding weird stuff and there are a lot of unintentional outcomes that might cause harm in the process. I wasn’t looking for fraud in my models, I just found them because I made them get me more conversions. Then, I had to try to convey that to stakeholders who were struggling with the nuances and often with the misalignment of incentives. So you think about balancing these ultimately fundamentally ethical questions. It’s maybe harmless in advertising whether we waste money on bots, but the bigger picture, if you think about the discussions around social networks, is that there are a lot of discussions right now about negative impact. For example, what exposure to very highly optimized content does to young females who are using TikTok, or concerns around bias in algorithms that try to match people to job offers and might have a gender or racial bias that wasn’t necessarily intentional. These are things that the technology is capable of and that are really hard to safeguard.
You’ve been in this field for many years, where do you think it’s going in the future now that there’s such an interest in the business world around data science?
That’s an excellent question. The truly valuable use cases may be the ones that are not in the foreground of what everybody is talking about. The field is subject to some degree of hype and I think we’re focusing on the wrong components, both on the good and the bad. If you think about all of the flaws that machine learning has, if you compare them to how bad humans are at the same task, it’s actually worse. Human judgment is, for the most part, terrible, and I think we need to acknowledge its flaws. And we have to come to grips with a more realistic expectation of machine learning. Part of the problem is, of course, that the vendors of AI and machine learning will promise you that it will do great stuff. It might be able to win Jeopardy!, but it still doesn’t really understand simple human context.
Over the next five to 10 years, I think we’ll get a better understanding of where the sweet spot is and where it can make an impact. Not just what’s constantly talked about like self-driving cars. I’m not sure if that’s going to happen. I think we have a much bigger societal question to ask of how we handle it when the machinery goes wrong. But there’s hope. I think we will continue to make progress. I feel that whenever it works well, we forget about how much it helps all of our lives, like when Google tells me that I don’t need to waste my time with the fifth spam call from some car dealership that’s trying to come after me with some warranty offering. I’m personally bullish, generally speaking, on how it will help us in both healthcare and in even critical things like saving energy; for example, how we can be more cognizant of when we are wasting energy and better allocate it. I don’t have a lot of smart devices in my home yet, but I find the idea of less waste very appealing and I do hope that the technology can ultimately serve that cause in an almost unnoticeable way.
When you mentioned that AI is not as advanced as we might think, it made me think of one of my favorite videos that we watched in your class. The short sci-fi film that was written by a machine learning algorithm.
Yes, the Sunspring example is a really clear demonstration of how a lot of these areas mimic human behavior but they don’t have the fundamental capacity of understanding and generalizing it more broadly and putting it into context. So you see how this thing goes off the rails in an amusing and fascinating way. Feel free to add that link to it.
Oh, I definitely will. [Check out the Sunspring film here]
Now we’re on to the part of the interview where I’ll ask you rapid fire questions:
What’s one word that describes you? Cynic
What are you reading right now? Some dragon story
Finish this sentence: At 7am, you can find me… Snoring
What is a cause you’re passionate about? Animal welfare
What do you do for fun (besides horses, because you already gave us that one)? Chopping wood
A favorite movie, TV show, or podcast? Gattaca
What’s one thing you can’t do that you want to learn? Playing cello
If you could meet anyone living or dead, who would it be? My parents, I haven’t seen them in two years because of Covid
I have a couple final questions. What is something your students might not know about you, but you think they should?
That I’m actually a lot nicer than I seem.
Last question, what career advice would you give to your students?
I think the best overall advice is that, as you’re approaching your career, don’t spend too much time planning because life happens in unexpected ways. Maybe you have figured this out already, but I think the skill is to create opportunities. When I say create opportunities it does not mean strategically looking to advance in a career but to do good work for a lot of people. They will remember you, and it will be useful. So that’s one – don’t overthink it with the planning.
I think the most immediate one is to look at your first job as an opportunity to get really good mentorship. Your first job is a place to grow and you should look or evaluate whether this is a job that you want based on whether or not you see people in that environment who can teach you relevant stuff. Whether it is technical, strategic or how to deal with large organizations – give yourself time to grow beyond your initial education. Give yourself three to five years of good mentorship to really kind of push your potential and learn as much as you can.
Thanks so much for that great advice and for sharing your story with us. Sternies, you can reach Claudia on LinkedIn… and try to take her class!