Tech Leadership Spectrums — Assessment Tests
This post is a continuation of a multipart blog series about Technology Leadership Spectrums. The previous post discussed high-bar vs. low-bar recruiting. It also discussed hiring for professional experience vs. candidate characteristics and interviewing with empathy vs. apathy. This post takes it a step further and evaluates the use of assessment tests in the recruiting process.
As stated in the previous post, two of the most sought after candidate characteristics are aptitude and creativity. The most common way of evaluating these characteristics is via whiteboard coding challenges.
On the one end of the spectrum, employers don’t use any assessment tests. On the other end of the spectrum, employers use many assessment tests. This case might involve whiteboard coding during a video chat followed by several more challenges during in-person interviews.
The current consensus seems to be on the Many tests side of the spectrum. However, the use of whiteboard coding is a point of contention. On the one hand, you have tech giants like Google who are trying to assess millions of candidates in a fair and repeatable process. On the other hand, you have candidates spending an exorbitant amount of time preparing for the coding challenges, not to mention the stress of performing these oral exams. From a candidate’s perspective, whiteboarding and waterboarding not only similar in name but also in the torture they exude!
A typical way for software engineers to prepare for whiteboard coding is with the Cracking the Coding Interview book. It is nearly 700 pages long and requires about three weeks of full-time study! Such a commitment can favor younger engineers without mortgages or families. The following book reviews illustrate the frustration that candidates have with coding interviews:
Perpetuates the insanity of programmer interviews …the interview-prep industry has completely decoupled itself from the actual job of programming!… I’ve been writing software for a long time… I’ve worked at some well-known companies, and I’ve interviewed a LOT of people. But I’m here to tell you that even I can’t pass one of these interviews without studying. That’s a bad thing. If the goal of an interview is to identify competent programmers, we’ve gone far, far off the rails with these kinds of interviews. …the presence of books like these create a vicious cycle: prep book gets written; interviewees study/memorize answers; interviewers make questions “harder” to compensate; new book gets written! It never ends. The grinder continues to turn, and whereas ten years ago you could get a good job with some string or linked-list manipulation questions, now you’ve got people who consider whiteboard coding of topcoder elite questions to be the baseline measurement of programmer competency. That’s nuts. …If you interview programmers, please, stick to questions that demonstrate actual day-to-day work competency. And yes, if you’re interviewing and you have the leverage, stand up to companies that try to abuse you with this kind of demeaning nonsense… If we are to be professionals, we have to demand the career respect afforded to professionals. That includes not being treated like children when we are interviewed. (Tim)
Contributes to ludicrous tech interview process This book was the worst investment of my career. The time I put into this book helped me get further along in these insane interview processes but ultimately did not help me find a job. Not to say this book is trash, but for me, I put months of studying into this book and I did not pass an Amazon or Google interview. Yes that is not the books fault, just pointing out that it is not the end all book and that for different people (like myself) I would have been much better off spending my time hands on building real things rather than this interview fluff. I feel strongly that tech interviews are flawed and often ridiculous and [that] studying some of these problems that a regular developer never comes across in real work… helps promote that interview process. (Adam Pritchard)
Despite these frustrations, the conundrum remains. How do popular tech companies filter through millions of applicants per year? For these large tech companies, maybe the least bad solution is, in fact, the coding interview. For smaller companies, a less extreme evaluation method might be more appropriate.
Dan Kador seems to have found a reasonable middle ground as he explains in the following interview:
[The recruiter] will do a phone screen to… make sure that the opportunity is interesting from both directions before [either party] invests more time. We want to be respectful of people’s time… When I say “engineering phone screens,” I don’t mean technical, run through coding problems. It’s more just one engineer to another — let’s get to know each other, tell me about the problems that you are working on, tell me about the things that you have been interested in. In general, we have completely shied away from any of the stereotypical Silicon Valley interview type questions… We don’t try to trick people. We don’t try to stump people with tricky algorithms. None of us are actually implementing sorts at any point in our careers here. (CTO Connection Podcast — min 33:00)
Ed Johnson has a similar take as he portrays in this interview:
[Ed] The biggest thing that I learned after a lot of recruiting in corporate is that a purely technical interview does not cut it. Where you put the semicolons in the C code doesn’t mean jack! What I need is initiative, drive, and the ability to solve problems independently. That weighs so much higher… I will still give coding questions, but that will not be the only ones.
[interviewer] Are your coding questions around practical things that people are going to build on the job, or are you having people write out quicksort on the whiteboard because I have never done that past my CS degree.
[Ed] No, I would give a more high-level design kind of question. I almost hesitate to even get to the the full code level, but I need to know how you think. (CTO Connection Podcast — min 7:15)
An alternative to a high-stress whiteboard interview is a take-home assignment. It allows the candidate to complete the coding challenge from the comfort of her own environment. Doing so mimics a more realistic situation than whiteboard coding. In the real world, software engineers almost never code on a whiteboard.
However, take-home assignments are not without their challenges. For example, how do interviewers know that a candidate completed the assignment himself? In most situations, a code walkthrough can ascertain that question. In addition, how long should the assignment take? A lengthy challenge increases the likelihood that a candidate won’t bother with it. Lastly, should the assignment be timed? A timed challenge also increases the likelihood that a candidate will forgo the pressure cooker.
Online work samples might be an alternative to both whiteboard and take-home challenges. Examples of online work samples include blog posts, GitHub contributions, and Stack Overflow answers. Employers could give candidates a choice of taking a coding challenge or proving a work sample. Work samples are not perfect either. Employers who choose to accept work samples forgo a standard way of comparing candidates. But they might also increase their talent pool by being more accommodating.
In this CTO Connection interview (min 30:30), Michael Lopp states, “If you apply at Slack, and you say, ‘Hey, I have a GitHub repository with X, Y & Z,’ we’ll look. Because what better way to see if someone is a good writer than reading their writing or a good engineer by looking at their code?”
For other philosophies related to technology leadership, please check out the parent post as well as the following posts:
