Getting the Interview Phone Screen Right

blog
81

IMHO, the better test in an interview takes the form of “here is a [very basic] API. Implement [simple algorithm using recursion] with it.” It’s not a phone interview question, but it’s a first-live-interview question, which might be a showstopper of the entire process (no reason to waste anyone else’s time if the first technical interview bombs).

The trick of course in making the API simple enough (without stripping it clean) and finding an algorithm to implement which is straightforward enough to be explained completely in a few minutes.

Bonus questions include: what would you change about the API to make it perform better? Where can additional safety be introduced?

This seems to be a key differentiator between a developer who can dive into our code base, figure out the dependencies, and innovate, and the developer who knows only basic CS 101 stuff but can’t apply that to a real codebase (which is acceptable in a fresh-from-college candidate, but a major thumbs-down if the resume lists years of business experience!), and the developer who has coasted along on the coattails of others his entire career.

82

Why not just watch the homeboy (or homegirl) code on his own system using Yugma or similar?

83

@Lawrence, you got a point, I’m quite bored. However now that I think about it, it’s not because of the incompetency around here, the other guys here are not incompetent, they could all answer these questions. But the general lack of soul, you know the will to just try stuff and challenge themselves everyday. I think interviewers need to worry about that above knowledge base.
Your also right about getting a new Job. I’m just wasting my time here. Gonna start job hunting today.

By the way I was wondering why in the world would anyone have 50.000 html pages. If you expect someone to solve the problem with scripts, shouldn’t you first ask yourself why you didn’t use a script language and regex to create those 50000 pages, making the whole issue of replacing a phone number a matter of updating a field in your database :slight_smile:

84

Well Jeff it was bound to happen I finally disagree with what you say. When I first graduated school, I would have had that total vacuum in RegEx’s my OO Design skills would have been poor. I could do it but I couldn’t do it at the level I am at today. I think you’ve three very important things here.

  1. The job context. I could probably figure out and test and develop you code that uses a RegEx to solve your phone book problem in maybe 5 hrs if I had to use RegEx. However writing code to drive a digtal earth reference model do you know how many places I have had a need to use RegEX ? Maybe 1 or 2, in the last 3 years. Then there is co-workers and the internet. So I think the context in some areas is pretty important.

  2. Secondly Interpersonal relationships. If you’re not a team player and you want to sit in a closet and code. Great I don’t want you on my team, I don’t care how good you are, END OF STORY! There is no test for that.

  3. You’re ability to solve problems. I am not interested in if you can do what’s been done 100 times out there. Can you really really think critically ? I suggest the thought analysis process is the most important. I’ve seen developers who’ve had 10 + yrs who could code anything you gave them but couldn’t have an original thought if they’re life depended on it. Again these are old problems that have been solved 100’s of times.

I think that these test degrade our past experience and references. If I have 3 or 4 + years of experience and half decent references, why assume that it’s just luck that I’ve stuck around ? Obviously I must have done something right. If you want me to write a test to get a job then don’t ask for a resume. I wouldn’t waste my time on an employer that asked me to prove myself in an interview. That’s what a resume and references are for.

I wouldn’t go so far as to lead pipe someone but that would be the end of the interview because, I wouldn’t work for anyone who felt my resume/references were good enough to talk to me but not good enough to consider hiring me until I proved my worthiness.

Good Post, interesting insight but I am in 80% disagreement with what you’ve said here.

Chris

85

Hi Jeff,

I usually agree with you on everything, but here I have to side with the mob :slight_smile:

Your questions are really textbooky and I would fail at least 50% of them (as in, not having a clue without Google). I have no uni degree, six years of real world development experience, and currently being self-employed.

If you’re doing a phone screen of an fresh out-of-CS-student then your questions will suffice - in checking whether he/she was paying attention in certain classes, but for any other … geezzz…

Take a CS major, submit him/her to 10 years of real world development, idiotic requirements, even bigger idiots with money and of course users - and look at textbooky stuff slipping away …

86

“Design a deck of cards that can be used for different card game applications.”

This could be a black hole of design in itself. You could sit there for the whole hour coming up with useful stuff to cram in there.

“Design an OO representation to model HTML.”

Correct answer: Google up the DOM. Cut, paste, done.

“Last year my team had to remove all the phone numbers from 50,000 Amazon web page templates”

The real task here is to make it so they don’t have FIFTY THOUSAND frickin’ templates sitting around. Who designed that monstrosity??

“What are some really common data structures, e.g. in java.util?”

Well, not being a Java programmer, I’ll need Google to answer that…

“A candidate should be able to demonstrate a basic understanding of the most common data structures. More specifically, the big ones like arrays, vectors, linked lists, hashtables, trees, and graphs.”

This could be an issue, particularly since till this moment, I was under the impression that a “vector” was a mathematical n-tuple representing an offset in an n-dimensional Cartesian coordinate system. This is despite the fact that I have used “vectors” in the sense the interviewer meant countless times.

“They should also know the fundamentals of ‘big-O’ algorithmic complexity: constant, logarithmic, linear, polynomial, exponential, and factorial. If they can’t, that’s a huge warning flag.”

…that the candidate doesn’t have a degree. Which you may or may not care about.

87

Atario,

It’s not about java.util.*, it’s about knowing basic data structuring libraries in your language of choice.

About the big-O, you probably don’t have to know how to write it down. You just have to know that it if you want to look up a value in a sorted linked list, it will take twice as long for a list double the size (in average). While if you had to look up a value in a binary tree, it would only take 1 extra iteration/function call when the tree doubles in size.

Caring about big-O is not such a huge issue when there are few items. But when the amount of data starts to increase, a badly chosen data structure will slow down the application.
See also http://www.codinghorror.com/blog/archives/000957.html about the rationale of caring about big-O in sorting algorithms.

88

Watch out for one-trick ponies. Candidates who only know one particular language or programming environment, and protest complete ignorance of everything else, are a giant red warning flag.

And watch out for those that say they know a huge list of programming languages. They probably had contact with it in College or something, but they can’t possibly know all of them.

89

Having done quite a lot (thousands, probably) of interviews (I did it for a living for a few years before going back to uni for my CS degree), I would say that Jeff/Steve is/are generally on the right track. However, I would emphasise that you need a mix of general questions (eg questions 1 (modified to your target language set), 2, 4 and 5) and the specific (question 3).

A question requiring you to write a regular expression does NOT count as general knowledge. You should know what one is, but the specific ability to write one depends on how often you use them (I haven’t written one in years, so I could no longer do it without looking up the syntax). But if you need to know them to do the job you’re applying for, then it’s a fair question.

It’s a common trap to think that all good candidates (for any job, not just software) should be exactly like you in knowledge and outlook (although nobody could be quite as good as you, right?), so you need to guard against that. The regex and data structure questions particularly have that sort of smell about them.

Personally, I’m also inclined to ask some process oriented questions, eg patterns, unit testing/TDD, refactoring, SE methodologies etc, as that tends to give you a different perspective on the candidate than the detail stuff.

Questions about the interviewee’s personal life/interests can generally wait until a face-to-face, unless you’re interviewing someone a long way away, in which case you should be more thorough with your screening.

Also, the interviewer should NOT be doing most of the talking, although they should be controlling the direction of the interview (programmers in particular can tend to rabbit on a bit!). The phone interview is to find out about the interviewee, ideally quickly, to see whether they are worth meeting. The interviewer jabbering away does not contribute to that. You do need to mostly ask open questions that require an answer beyond yes or no. For example, ask the candidate to explain something, or talk about their experience with something, rather than if they’ve heard of it.

Also, the phone interview is good time to check out any red flags raised by the CV - long unexplained breaks, overlapping time periods, lots of very short jobs, impossible amounts of experience with a technology, and so on

90

Write a program to calculate and display the shopping days til Christmas in any way in the language that the shop uses the most…

To keep it simple, any day except Christmas is a shopping day.

It must return the correct answer.

If they can’t do that then they can’t handle:
dates,
leap years,
logically thinking out the problem.

Have them submit the code.

Look for:
Good usage of components (good)
Correct handling of leap years (good)
Good OOP principals (separating the interface from the function from the UI) (good)
Good re-usability (very good)
Good coding style (good)

Just a little suggestion…

91

The comments from this post reflect the ongoing battle between those who believe that software developers only need real world experience to do their job, and those who believe that a person needs a Computer Science or Software Engineering degree to do well.

My personal opinion is that a person who wants to be a professional software developer and not just a computer programmer will do both. He (or she) will get the CS or SE degree, and then try to work in all aspects of software development, not just implementation.

For my part, this post only emphasizes my desire to hit the Computer Science Theory books again to get a better grasp of the fundamentals.

92

@Atario:
"This could be a black hole of design in itself. You could sit there for the whole hour coming up with useful stuff to cram in there."
Which would neatly show that, in a real project with shifting requirements, you’d be incapable of solving basic problems without overcomplicating them.

"Correct answer: Google up the DOM. Cut, paste, done."
And? That’s correct (I think), it solves the problem, and all you might need to do to show off is highlight the nouns you’d turn into objects.

"The real task here is to make it so they don’t have FIFTY THOUSAND frickin’ templates sitting around. Who designed that monstrosity??"
Converting 50,000 Amazon web page templates; a long, slow tiresome process that will cause more trouble than it solves, and will never be complete.
Writing the regex; about an hour’s work, including debugging.
Pretending this problem doesn’t come up all the time; Priceless.

"Well, not being a Java programmer"
What, you’ve never encountered or expect to encounter Java in any project? You can never be trusted to work on a single project involving Java? If you and Java come into contact, the universe would warp, and life as we know it would become lavendar-fresh bath salts?
I’m not a “C++ programmer”, but my current project involves C++. Even if
"I’ll need Google to answer that…"
If you need Google to answer the question “What are common data structures?” (like Lists, Linked Lists, Vectors, Maps, Hash Tables), how in the name of the Gods are you going to program if your net connection fails?
java.util was a clue, not a requirement.

"This could be an issue, particularly since till this moment, I was under the impression that a “vector” was a mathematical n-tuple representing an offset in an n-dimensional Cartesian coordinate system. This is despite the fact that I have used “vectors” in the sense the interviewer meant countless times."
So, if they asked you to perform a simple task at work, you’d make some clever quip about double-meanings rather than actually doing what they asked… That’s actually good for an interviewer to know.

“They should also know the fundamentals of ‘big-O’ algorithmic complexity: constant, logarithmic, linear, polynomial, exponential, and factorial. If they can’t, that’s a huge warning flag.”
"…that the candidate doesn’t have a degree. Which you may or may not care about."
No, that they have no grasp of code efficiency. If you don’t know why an exponential-time algorithm is bad news, I really don’t want to use your sorting algorithm.
You don’t have to know the full terminology, but words like polynomial and exponential are taught at high-school level. And appear in every text-book on the subject. And are bandied around by so-called “real programmers”.

This is why they need to start asking these questions. It exposes not only any areas of ignorance, but also your whole attitude to problems and authority. Which is far more useful. You can learn regexes, learning how to deal with other people is harder. More useful too.

93

Many people are single skilled like one-trick ponies. I strongly agree with the interview tip, but we need to understand that it is not the single skill that really is the issue, but instead is the attitude of rejecting the means of utilizing other languages as an alternative.

Colin Joss
Colin.Joss@gmail.com
East Lothian, Haddington
United Kingdom

94

Interviewers typically have to offer a lot of leeway for the nervousness factor, although some may have the wrong perception about the quality of programmers applying for jobs because they simply aren’t able to see both sides of the table anymore. I see some of that arrogance leaking out in some of the comments – Yes, of course there are people out there that just don’t have the chops to be good programmers, but I think this is getting exaggerated a bit by looking at random experiences from interview performances. I say this having been on both sides of the table before. Believe it or not there are good programmers who don’t do great with interviews in the style they are often done, and this can’t be ignored as simply ‘disqualifying a good candidate occasionally’ – especially when it’s admitted the system is flawed AS IT IS and people get through those phone screens who don’t have the skills desired.

For you to accept that interviews are the best predictor means that anyone who fails an interview must be a bad programmer – yet in reality most programmers have flubbed interviews before, or done badly when presented with certain areas or questions they were unsure of (combined with the nervousness of being there). It seems a lot more shaky when you look at it in light of people’s realistic experiences and the ‘hit and miss’ nature of the whole thing.

Especially considering all the people who make it past phone screens anyway (part of the original topic) - designing a better phone screen doesn’t mean potshotting more random technical questions, many of which aren’t related to tasks you actually will DO and work with on the job.

95

"Write a program to calculate and display the shopping days til Christmas in any way in the language that the shop uses the most…

…
Good OOP principals (separating the interface from the function from the UI) (good)
…" - Mac

Umm, why would this little exercise involve objects? This could be done in one line of Perl.

96

This post reminds me of a series of blog posts I wrote a few months back. I basically railed against everything Steve Yegge thought was good.

I still disagree with his philosophy.

http://codeeleven.blogspot.com/2007/11/want-job-reverse-string-sad-state-of.html

And a follow up:

http://codeeleven.blogspot.com/2007/11/whiteboarding-code-part-duex.html

97

I did a phone screen a while ago. The guy was monosyllabic, which I thought might be interview nerves, so I tried to draw him out a bit:

“What’s the technology you’re most enthusiastic about? What do you feel you’re a real expert at?”

“Hibernate”

“Ok…” (I wrote a book on it) " so what version are you currently using?"

“Er… er… um… I dunno.”

“We’ll get back to you.”

98

I think that all of these questions are fair to ask a potential employee. The interviewee may not be able to answer all of the questions, but a lot can be determined of the person just by watching / listening to his or her reaction. For example:

For the telephone problem, the interviewee may not know how to use RegEx. Uh-oh… but it’s possible to do this anyway… He or she could describe that they’d open each html.file and turn each line into an array of characters (ie. ‘O’, ‘n’, ‘c’, ‘e’, ’ ', ‘u’, ‘p’, ‘o’, ‘n’, ’ ', ‘a’, ’ ', ‘t’, ‘i’, ‘m’, ‘e’, ) and then loop through, one character at a time, seeing if it found a character that is 1 OR 2 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8 OR 9… then see if the next one does, and the next, etc, using a large string of ANDs and ORs.
This method, obviously, would be terribly inefficient, but it would show that the person was able to work around a barrier.
An undesirable candidate would not be able to work around a lack of regex knowledge.

Of course, Google solves everything, but…

It would be a bit like math tests where you show your work. Even a wrong answer still can get half credit for trying hard.

99

sobani:

“It’s not about java.util.*, it’s about knowing basic data structuring libraries in your language of choice.”

In retrospect, I may have been taking the “e.g.” as a direction to the interviewee and not the interviewer, as the original author may have intended. I was taking it as a literal and not a compiler directive.

"About the big-O, you probably don’t have to know how to write it down. You just have to know that it if you want to look up a value in a sorted linked list, it will take twice as long for a list double the size (in average). While if you had to look up a value in a binary tree, it would only take 1 extra iteration/function call when the tree doubles in size.

Caring about big-O is not such a huge issue when there are few items. But when the amount of data starts to increase, a badly chosen data structure will slow down the application.
See also http://www.codinghorror.com/blog/archives/000957.html about the rationale of caring about big-O in sorting algorithms."

I’m not saying knowing about big-O is not important, just that knowing the notation and terminology (“logarithmic”, “geometric”, etc.) probably shouldn’t be the real target of the interviewer.

Tom:

“Which would neatly show that, in a real project with shifting requirements, you’d be incapable of solving basic problems without overcomplicating them.”

It all depends on what you’re trying to accomplish, doesn’t it? If you’re writing a class for use in your company’s lone card game, you’ll want to write it differently from, say, a class for sale to card-game developers worldwide.

“And? That’s correct (I think), it solves the problem, and all you might need to do to show off is highlight the nouns you’d turn into objects.”

I meant that the answer would be that it has already been done – as it says in a different spot, “don’t reinvent the wheel”. Especially since modeling HTML is a pretty massive undertaking to give someone in an interview question.

“Converting 50,000 Amazon web page templates; a long, slow tiresome process that will cause more trouble than it solves, and will never be complete.
Writing the regex; about an hour’s work, including debugging.
Pretending this problem doesn’t come up all the time; Priceless.”

If you have 50,000 templates in use, you don’t have templates. You have a mess. And I don’t think anyone here is saying that messes don’t exist. It’s just that there’s “doing the right thing” and theres “slap-dab it till it doesn’t error out”. Not that we haven’t all been guilty…

“So, if they asked you to perform a simple task at work, you’d make some clever quip about double-meanings rather than actually doing what they asked… That’s actually good for an interviewer to know.”

Too clever by half, apparently, since it failed to get across to you that emphasis on specific terminology will mislead the interviewer to think that the interviewee doesn’t know something, when, in fact, they know it by a different name. This is a problem with 99% of all computer “skills assessment” testing, by the way.

“This is why they need to start asking these questions.”

My point is: they do need to start asking new questions, but they also need to be careful what they’re really asking.

100

Wow - Ron - how amazingly patronising of you…

“And finally, for those who can’t do these problems easily: even if they look stupid to you, the subjects they cover would likely broaden you as a programmer and increase your productivity.”

I wouldn’t be so bold to state that every good developer should be able to spit out a f*cking “Virtual Zoo program” on the end of a telephone.

and… Jon Raynor - what you wrote was a Function not a Method… In order for it to be a Method, your code needs to be inside a class

i.e.

class MathFunctions
{
    public int AddNumber(int a, int b)
    {
        return a + b;
    }
}

I would also recommend using camel casing on your Method decleration, include some comments, provide better variable names for your parameters and change the method instantiation type to static.

i.e.

/// summary
///     Class that contains general mathematical functions, 
///     accessible to all classes (Public).
/// /summary
/// remarks
///     Dated    : 24th Jan 08
///     Author   : First, LastName
///     AddNumber: Method that returns the result of adding two 
///                integers together
/// /remarks
public class MathFunctions
{
    /// summary
    ///     Adds two integers together and returns the result
    /// /summary
    /// param name="ValueA" type="int"
    ///     para
    ///         a value to be added
    ///     /para
    /// /param
    /// param name="ValueB" type="int"
    ///     para
    ///         a value to be added
    ///     /para
    /// /param
    /// returns
    ///     An integer value of the two parameters added. 
    ///     Note: there is no overloading, or error trapping 
    ///     in this method.
    /// /returns
    public static int addNumber(int valueA, int valueB)
    {
        return valueA + valueB;
    }// end of addNumber Method

}// end of MathFunctions class