BY ALEX SORIA – DIRECTOR OF DATA SCIENCE
From a very high level, the job of Data Scientist is to seek out patterns and begin to understand them. Sometimes finding these patterns requires testing. In our world these are usually A/B tests or Multivariate tests, but it doesn’t always have to be this black and white. I am curious by nature, so when I came across a situation that I found interesting, I decided to test it. It turned into an interviewing social experiment.
Each year the Elicit Data Science Team recruits from one of the top analytics schools in the nation. This year was my first experience going to the school to recruit and getting to know the program a little bit better. My team wasted no time getting me acclimated with the process. We had 15 interviews to get through that day.
My interview style is pretty organic. I allow the conversation to lead me to the line of questions I am going to ask, and then dig deeper into projects and experience that the candidates have. I don’t have a preordained list of questions I need to ask, as I feel like an open conversation will allow me to see their strengths, weaknesses and full potential. In fact, I hate most of those canned questions and brainteasers. That all changed during my first day of recruiting.
A conversation can go a long way in helping you understand a candidate’s way of thinking, but sometimes you need to jump start the process. An easy way of doing so is by asking one of the aforementioned “canned” questions like “How many golf balls fit into a Toyota Corolla,” for example. This just happened to be the question I asked to one person in the morning session. I wanted to understand how the candidate would think through a situation and come up with an answer where there is limited data and information. The interviewee gave me a great answer and I really enjoyed the way that she went to the white board—without being asked—and drew up her process. She then came up with a pretty straightforward answer. I was impressed.
Later when I hit an afternoon slump, I realized I was still lacking answers from one of the interviewees and decided to try the same question again. To my amazement, the answer was about as close as one could get to cheating off of the previous interviewee’s process. Now let me remind you, at this point I had interviewed 10 people, and not one of the interviews were the same. This was only the second person I had ever asked this question to in my life. So as I’m watching this play out on the white board for the second time, I decided right then, “I have an experiment!” All of a sudden, the day had new meaning!
I had 5 more interviews for the day and decided I needed at least one, maybe two more responses to prove out my theory. So I asked a third and fourth person the same question. Their answers and methods were very similar. For the most part, they all came up with a very similar answer; from the way they drew the car (I never asked them to do this) to the method to calculate the number of balls.
So, was my theory proven? Well, maybe not. I don’t think they were gaming the interview, I mean, how could they? I didn’t ask them all the same question and most of them were late in the day, so how could they know? Did their graduate program teach them that question in particular? Possibly, but not likely. What I have come to realize having been in the analytics field for 10+ years is that the process of teaching someone to think like an Analyst is very hard. It’s typically an innate ability, almost like a curiosity, to question things and look for the answers. I’m sure the vast majority of the students in this program probably already had this hardwired in their brains, but I’m sure the program has had a hand in helping them with their line of thought going from question to answer.
So how do you select a candidate as a future employee from a group of very talented individuals? You look for the one that asks you, the interviewer, the right questions. Those are the ones that are clearly thinking one step ahead in the conversation and those are the ones that are willing to put it all on the line to get the job they want with the people they want to work with.
Gaming the Interview?
BY ALEX SORIA – DIRECTOR OF DATA SCIENCE
From a very high level, the job of Data Scientist is to seek out patterns and begin to understand them. Sometimes finding these patterns requires testing. In our world these are usually A/B tests or Multivariate tests, but it doesn’t always have to be this black and white. I am curious by nature, so when I came across a situation that I found interesting, I decided to test it. It turned into an interviewing social experiment.
Each year the Elicit Data Science Team recruits from one of the top analytics schools in the nation. This year was my first experience going to the school to recruit and getting to know the program a little bit better. My team wasted no time getting me acclimated with the process. We had 15 interviews to get through that day.
My interview style is pretty organic. I allow the conversation to lead me to the line of questions I am going to ask, and then dig deeper into projects and experience that the candidates have. I don’t have a preordained list of questions I need to ask, as I feel like an open conversation will allow me to see their strengths, weaknesses and full potential. In fact, I hate most of those canned questions and brainteasers. That all changed during my first day of recruiting.
A conversation can go a long way in helping you understand a candidate’s way of thinking, but sometimes you need to jump start the process. An easy way of doing so is by asking one of the aforementioned “canned” questions like “How many golf balls fit into a Toyota Corolla,” for example. This just happened to be the question I asked to one person in the morning session. I wanted to understand how the candidate would think through a situation and come up with an answer where there is limited data and information. The interviewee gave me a great answer and I really enjoyed the way that she went to the white board—without being asked—and drew up her process. She then came up with a pretty straightforward answer. I was impressed.
Later when I hit an afternoon slump, I realized I was still lacking answers from one of the interviewees and decided to try the same question again. To my amazement, the answer was about as close as one could get to cheating off of the previous interviewee’s process. Now let me remind you, at this point I had interviewed 10 people, and not one of the interviews were the same. This was only the second person I had ever asked this question to in my life. So as I’m watching this play out on the white board for the second time, I decided right then, “I have an experiment!” All of a sudden, the day had new meaning!
I had 5 more interviews for the day and decided I needed at least one, maybe two more responses to prove out my theory. So I asked a third and fourth person the same question. Their answers and methods were very similar. For the most part, they all came up with a very similar answer; from the way they drew the car (I never asked them to do this) to the method to calculate the number of balls.
So, was my theory proven? Well, maybe not. I don’t think they were gaming the interview, I mean, how could they? I didn’t ask them all the same question and most of them were late in the day, so how could they know? Did their graduate program teach them that question in particular? Possibly, but not likely. What I have come to realize having been in the analytics field for 10+ years is that the process of teaching someone to think like an Analyst is very hard. It’s typically an innate ability, almost like a curiosity, to question things and look for the answers. I’m sure the vast majority of the students in this program probably already had this hardwired in their brains, but I’m sure the program has had a hand in helping them with their line of thought going from question to answer.
So how do you select a candidate as a future employee from a group of very talented individuals? You look for the one that asks you, the interviewer, the right questions. Those are the ones that are clearly thinking one step ahead in the conversation and those are the ones that are willing to put it all on the line to get the job they want with the people they want to work with.
Tags: