Getting the Interview Phone Screen Right

blog
161

Jeff, I will bend the context of your article a bit to pretend that the definition for Q3 was simply “3) Scripting and regexes. The candidate has to describe how to find the phone numbers in 50,000 HTML pages” with no further definition supplied (a better test, I think), as I want to make a point about the assumptions that lie behind the questions (and the possible answers)…

Candidate 1: That’s trivial! I can do that in 2 minutes! (Failed)

Candidate 2: I’ve never had the need to use regexes before, but, hey, that’s what books are for. But first, I have the following questions… (Passed)

Why? Because Candidate 1 will give the appearance of being brilliant and producing quick solutions, but his solutions will probably be rushed, half-assed and ill-conceived.

Candidate 2 asked the following questions during the interview, thereby demonstrating that she understands real world problems and client needs. She is a capable solutions developer, not merely a programmer with a collection of theoretical tools and no practical understanding as to how they might be used…

  • How many of the telephone numbers are badly-formed, broken over multiple lines, or protected from harvesting by Java script or embedded as graphics files?

  • Are any of the pages Unicode and/or in other languages?

  • Once we determine the proportion of telephone numbers that cannot be extracted by regexes, what is an acceptable loss? Are we willing to lose all or them, none, or somewhere in between?

Now, drawing liberally from one of the comments above, the difference between the guy who took days/weeks and the guy who took 2 minutes is that the guy who took longer recovered 99% of the telephone numbers and the other recovered 25% (only properly formed numbers located in the USA stored in plain text).

If this were a privacy issue, for example, “we are required by law to not allow telephone numbers to be published or people can sue us into oblivion (or we can end up in jail)”, the yield is important. And, as we are bending the context for the sake of argument, we also assume Amazon, being an international company, wants international solutions to international problems and does not want programmers who are thoughtlessly US-centric (a plague infecting many international websites).

Now, if neither guy asked any questions about the scope of the problem (and it’s goals) before starting, both are idiots. One may have produced an inadequate solution that comes back to bite the company 3 years later with a massive lawsuit, the other may have wasted time and money unnecessarily by over scoping the problem.

Before anyone says, the scope of the problem is defined by the assumption that regexes are to be used: it is very common for employers to be determined to use Product/Method A to solve the problem (they only have a hammer and all the world is a nail) and it is up to you, as a seasoned professional, to advise them that Product/Method A may not be up to the job and Product/Method B should be used instead and explain why, clearly weighing up the pros and cons.

Sadly, you could eliminate a lot of candidates by asking: Do you pass parameters, or use global variables?; Draw me a data model for a database that holds questionnaires and their results; Do you know what a For-Do loop is and why you would use it?; What is a class?; What does ‘event driven’ mean and how does it affect the way you design your code?

I have met a disturbing number of in-house developers that would fail these questions.

“Watch out for one-trick ponies. Candidates who only know one particular language or programming environment, and protest complete ignorance of everything else, are a giant red warning flag.”

The counter to this is obvious: those who know many are experts in none. Sometimes you want experts, not dabblers.

162

@YerFired:

You said,

Q3: You discover that performing the task takes 100% CPU as well, now what?
The answers I would look for are:
c - here I’m thinking about farming out the work to more than 1 computer, and different computers need to cooperate. It can be as simple as a database, maybe it’s more complex

That’s an interesting thing to look for. I would have looked for something like this:

“First, I’d want to know why it was taking 100% of the CPU’s resources. Then I’d address that. If it’s addressable by hardware or software, we could fix that. Either way, we profile the application to determine which parts are consuming the most processor cycles and optimize the heck out of them. If that still doesn’t work, then we look at the webfarm solution.”

Can the developer see the BIG picture? Is there a problem with the server hardware? Does the problem occur in both the test and production environments? If it doesn’t, there’s something different between them, and it needs to be resolved. If it’s not a server configuration issue, then you profile the application. No line of business application should consume 100% of CPU cycles. HUGE RED FLAG.

163

Aaron G sounds like StrCpy() guy to me. This was the guy that decided that the best way to determine if ANYONE knows how to program is to check their knowledge of Strcpy:

http://davidavraamides.net/blog/2005/05/07/strcpy/

After all, ANYONE who knew how to program would have read this specific book and studied this specific function and be able to regurgitate it 5 different ways!!! Right? This guy is so convinced that everyone is thinking exactly like him, he isn’t able to step outside his own shoes and look at it from the standpoint of a younger person. Similar to you I think, especially with the mocking responses you had to anyone who legitimately questions the validity of certain questions.

Look, you’ve probably been frustrated by clueless coworkers like most of us, but what I think you fail to see is that THESE INTERVIEWS DO NOT WEED OUT ALL OF THOSE PEOPLE - simply the less clever ones. Believe it or not, you’re going to be working with people whose ability and knowledge frustrates you on some level. Trying to design a gauntlet that will make every person hired perfect is not possible - and yes, everything is subjective, but you want to have a reasonable test. You cannot tell me that anyone who knows how to program knows recursion - what about 20 year veteran COBOL programmers who have designed massive systems that didn’t use recursion? The real question is, what does the ability to program entail? It’s a worthy question and should be considered when designing good phone screens.

164

“A lot of people seem to think that perfect answers are needed to all these questions; note that this is a first-stage phone screen, just to decide whether or not to bring you in for interview.”

You are right about the fact that it is not about the correct answers but how people respond. But if you notice in this thread, the response will be that you will lose good programmers if you even try it on them. In theory it should work, but in praktice, if someone asks me these stupid questions, then the first thing that comes to mind is that the people in that company are stupid moron’s so I won’t even bother to procede with the interview because working there will limit my skills in a some kind in mind-killing boring serial work.

Good developers do not think inside the box. They dare to go outside the box too, and try to experiment. Otherwise you just create another clone of a program that has been created a zillion times.

You need innovation. I always use Google since I am creating never before used softwaretechnolgy and maybe someone out there has some good idea that I just take and make it even better.

165

Just get by the “FizzBuzz” question in the phone screen and I’m willing to talk in person. My most recent problem is the human factor of hiring freaks with “insane” personalities ticks that don’t emerge until three months after you hire them. Yikes! How do you interview for that?

166

Umm if the scripting question is soo easy to answer (find phone numbers in 50,000 pages of HTML) could someone explain to me how you would do this. Not that I am going to an interview anytime soon but I feel slow considering many thought it was easy.

167

“We all knew VB, I knew C as well, and for a variety of reasons we wanted to find an additional person who knew C++. Despite clearly requesting a strong C++ background in our ads, candidate after candidate would show up who thought that knowing C++ meant writing C code but using the C++ operators instead of printf and scanf .”

I had the opposite experience, they asked a C++ programmer, but they thought that C++ meant USING the classes from MFC while programming in C. They actually forced me to do it their way since they did not understand what I was doing. Inheritance? Polymorfisme? Private?

168

I dont think you’d get a code monkey for asking the guy what 2^16 is.

I have seen a NASA UAV collision avoidance algorithm coded in C which used ^ for exponentiation, though that was in a research paper rather than production code.

169

Anybody else taken a crack at LittleRascal’s problem? Good problem, harder than it looks, here’s my (probably inelegant) solution in Scheme. I went for stack-intensive rather than having to do any copying/flattening of the data structures, etc.

I’m assuming your trees are binary, for simplicity, and defining a tree as a list with (data left right).

(define global-combos '())

(define left
(lambda (tree)
(car (cdr tree))))

(define right
(lambda (tree)
(car (cdr (cdr tree)))))

(define data
(lambda (tree)
(car tree)))

(define solve
(lambda (tree rest-of-trees current-combo)
(if (null? rest-of-trees)
(begin
(set! global-combos (cons (cons (data tree) current-combo) global-combos))
(if (not (null? (left tree)))
(solve (left tree) rest-of-trees current-combo))
(if (not (null? (right tree)))
(solve (right tree) rest-of-trees current-combo)))
(begin
(solve (car rest-of-trees) (cdr rest-of-trees) (cons (data tree) current-combo))
(if (not (null? (left tree)))
(solve (left tree) rest-of-trees current-combo))
(if (not (null? (right tree)))
(solve (right tree) rest-of-trees current-combo))))))

Then you do such as:

(define trees
’(
(4 (2 (1 () ()) (3 () ())) (5 () ()))
(b (a () ()) (c () ()))
(Z (X () (Y () ())) ())
))

(solve (car trees) (cdr trees) '())
global-combos
((y c 5) (x c 5) (z c 5) (y a 5) (x a 5) (z a 5) (y b 5)
(x b 5) (z b 5) (y c 3) (x c 3) (z c 3) (y a 3) (x a 3)
(z a 3) (y b 3) (x b 3) (z b 3) (y c 1) (x c 1) (z c 1)
(y a 1) (x a 1) (z a 1) (y b 1) (x b 1) (z b 1) (y c 2)
(x c 2) (z c 2) (y a 2) (x a 2) (z a 2) (y b 2) (x b 2)
(z b 2) (y c 4) (x c 4) (z c 4) (y a 4) (x a 4) (z a 4)
(y b 4) (x b 4) (z b 4))

This took me a few hours of muddling, and were I posed this question in an interview I likely would have managed to eke out a few things about “recursion” and “closures” and some graphs on a piece of paper, and probably the start of some incorrect but close solution.

Were you expecting people to solve this in the space of an interview, or was it intended as a thought process thing?

170

billybob,

I don’t know about other people… I think that kind of tasks are not too difficult. It is more to rigorous instead. I don’t see a method other than looking at it one by one (by script, of course). The way to speed this up may be with multiple “crawler” or faster connections.

171

Delmania
You make a good point though I don’t agree with your personal opinion as it stands. You state "He (or she) will get the CS or SE degree"
and speaking for myself the answer is not necessarily. However I do read a lot about Computer Science that interests me , I just don’t feel the need to validate the learning with a degree.(Similar to not going in for a JEE certification for example). I do agree that some people look for theoretical answers and some for real world experience(I’m with the latter) and this post seems to be with the former.

172

Aaron G: I would have thought scheduling a phone interview was professional as well. But looking over the 21 interviews I did that year, there were only 5 by phone, not counting recruiters. I think one or two of those were scheduled. To be fair, other than the soccer example I gave, the rest asked if it was a good time to call.

173

“Umm if the scripting question is soo easy to answer (find phone numbers in 50,000 pages of HTML) could someone explain to me how you would do this”

Use a script (Perl being the obvious choice) that loops through the files, reads in the contents, use pattern matching (regex) to find matching text. Anyone familiar with Perl would consider it trivial.

174

I noticed a lack of questions about Software Engineering practices like requirements specifications, validation, etc.

175

I hate to say it, but if someone makes me write some sample code outside the job interview, I will most likely pass. I don’t have time to write some stupid sample code for you. There are a lot of other companies out there that are probably just as good as yours and they won’t ask me to spend two hours writing a crappy little application so you can see if I can code. Ask me on the interview or don’t waste my time.

I generally agree with everything posted here though. I think it’s ok to have to google for exact implementation. I don’t remember exactly how a radix sort works but I know that it’s faster than quicksort for integers in most cases. I also know that a heap works well for priority queues, and I could generally describe what a heap is and why it’s useful for that. Could I implement one over the phone? probably not, since I haven’t had to do that since college, but it’s not like I couldn’t figure it out if I needed to. If I’m interviewing someone, that’s enough for me in most cases, particularly over the phone. Obviously if I am hiring someone for a particular position that requires a high degree of computer science knowledge, I’d ask more complex questions and expect better answers.

176

The largest hole that I’ve personally seen in the process is the incompetent pre-screening that seems to be done by 1) HR people with no concept of either tech vocab or ontology, 2) automated systems parsing for acronyms, or 3) Possibly monkeys throwing darts at a stack of resumes.

I had a cold call phone interview (they called me) from a company in the Chicago area which shall remain nameless. Apparently they got a hold of my resume somehow. After 10 minutes of harassing me about Extreme Programming (which I’d had no experience with) the interviewer got really snotty with me, explaining that there was no place for me there since I didn’t work in Extreme Programming. She acted as if I were wasting HER time, despite the fact that SHE called me out of the blue. I asked her where on my resume that it indicated that I was an XP expert, or had ever worked in an XP environment. Well, she got ruder still, and that basically ended the interview.

I later realized the problem. Under my skill set description, I had listed XP as one of the operating systems that I was familiar with. Obviously, whatever mechanism they were using to screen resumes for phone interviews was seeing XP and did not have the ability to put that acronym into context.

In another situation I had an HR person tell me that they needed a person to do RDBMs’s. Though when I asked him what that was, he didn’t have a clue.

In yet another situation an HR person told me they were looking for a developer with 10 years .net experience. This was in 2003.

Another time, I had an HR person tell me that they needed a developer to do XML, XSL and other EXI technologies. EXI was a proprietary application used by only one company on earth, the company I was currently working for. It was listed on my resume, so he thought he’d just make up a definition to go with it as if it were an open standard or other language.

If companies want to improve there phone screens, start at the front. Don’t hire HR folks that can’t turn on their computer and don’t understand the context in which they are hiring.

177

As someone already pointed out, it’s much more important to be able to quickly adapt, it’s not about how much specific coding in any language you can retain, “do you know this and that” - yeah, sure, if I can have access to all my normal references: my previous code I have written and then google, then I can do pretty much everything.

The idea that you need to know how to get a phone number from 50 000 files in an hour without any references to be a good developer is just plain silly. If you are hiring people based on that I feel sorry for you, your company will have hard time adapting when the “new thing” hits and changes everything.

178

“I want programmers who are good at what they do and are eager to learn to be better at what they do. Google teaches nothing except cutting and pasting.”

And being so narrow minded teaches nothing but ignorance. My answer to that is, I would never work for someone like you, you settle for mediocrity and I’m above that.

"good programmers write good code; great programmers steal great code. "

179

"good programmers write good code; great programmers steal great code. "

Then who wrote the great code?

180

Steve,

I did massive page searchings last year. It was around 30,000 pages to look for (not 50,000) and definitely not on Perl. I have to leave the 50,000 challenge to Perl experts.

Jeffry,

I think it is because most comment came from non corporate programmers.