Home › Posts › Jul 06, 2023

Conducting a Great Technical Interview

If you have ever been through the "getting a technical job" gauntlet then you have, more likely that not, gone through The Technical Interview. This interview usually comes after one or more Not Technical Interviews and serves to determine if you are allowed to wear the badge of Technical Enough.

I've had the pleasure to conduct technical interviews for many candidates ranging from engineers to managers to curriculum developers. I say "pleasure" because we conduct our technical interviews as conversations, not exams, and it makes for a much more pleasant experience for all parties involved. I work with some truly world-class people here so we must be doing something right.

While the technical interview is an important filter, it's harder than it might sound to get it right. This isn't the same conversation you have with the person next to you at a tech conference, trading war stories and poking around for something you have in common. This is a vital component of culture building and setting your team up for success and you often have less than an hour to do it.

There are other things going on here that makes this task difficult, besides the baseline pressure to get it right. Being an interviewer is, more often than not, a temporary second job that appears out of the blue. Just like all interruptive work, the work involved can put project timelines at risk (oh no, not the due date) and create more opportunities for context switching (the one true enemy of productivity). Depending on the importance this task is given, explicitly or not, it can become just one more thing in the way of "real work."

There is also this uniquely awful social component to interviewing where the candidate feels on trial, not sure if they should answer the question being asked or if there is a meta-question hiding in there somewhere. On the other side of the Zoom, it's often hard to let go of the need to look or sound smart and project the right authority since, hey, you already have the job they want. Awkward moments abound as everyone navigates all this in real time.

We have the makings here for a truly unpleasant situation so how have I learned to enjoy giving technical interviews? Like so many things, it comes down to identifying the job(s) to be done (JTBD).

Effective technical interviews come down to two important things that you, the interviewer, most own: having a great technical conversation and finding out what you need to know.

The former is not guaranteed and can be harder to get right if you're naturally introverted. The latter is straight-forward but easily overlooked if you're not deliberate about it. Both, I promise, become much less vague and anxiety-causing with intentional, directed preparation.

As a team, prepare before all interviews

This preparation starts with the team that the candidate is applying to join. Using the public job description as the foundation, write a list of the technical-specific qualities, experience, and knowledge that this person should have to be successful, if hired. These should be forward-looking, job-specific, and honest. Think about the things they should be able to do walking in the door as well as the things that they need to learn along the way.

As an example, let's say you're hiring for a mostly front-end programming position at a senior level. The job description might say "4 years programming experience with React, Vue, or similar." You should expect that the folks applying will have this experience on their resume in one form or another. But what else?

You might be looking for leadership qualities since you expect grow the team with a more junior hire next
You might be looking for experience with legacy front-end code to help modernize a particular application
Your team might be taking on a backend service in the near future so some full-stack knowledge would help
Design resources might be slim so you're looking for someone with an eye for UX

You should be able to boil this all down to a succinct (3 seems like a decent minimum, 8 seems like a lot) list of qualities, called a rubric, that will allow the team to compare candidates in a consistent and meaningful way. Post-interview, the team should score the candidate on these qualities individually then, after submitting their feedback, can more easily decide whether to move forward with someone.

Coming up with a coherent evaluation is now just a matter of generating the right questions. Either individually or with the team, write out a list of position-specific technical questions to ask during the interview. These will be the bank of questions you'll pick from for all candidates so write as many as you can but base them on the rubric.

For example, if leadership is one of the qualities you're looking for, a question could be:

Tell us about a project that you lead, whether it finished successful or not. What did you learn? What would you have done differently?

If you're looking for UX experience, a question could be:

Tell us about a project where you were a part of deciding the end-to-end user experience. What was your role in that project? What did you learn?

Write as many down as you can and then refine them. Think about the kinds of answers you might get. If the person you're speaking with doesn't give you a lot to go on, what could you ask to get them to expand? If they didn't have that specific experience, what else could you ask about in that vein? Expect to re-write each question a few times and try asking them to your teammates to see how they land.

You should end up with enough questions for all interviewers to choose from. If you did this individually then you'll want to combine them together and remove or combine related or duplicated ones. As a general rule of thumb, you should end up with no less than 12 questions total, distributed between the interviewers. Make sure they are ranked in some way from most important to ask to least so the least important ones are your backups if the interview is moving quickly.

Besides the main JTBD of finding out what you need to know, this bank of questions will make sure your team:

... is consistent with your questions across candidates.
... avoids awkward silences where you're not sure what to ask.
... has potential questions for future open positions.

As a final preparation step for all interviews, decide on who will lead the interviews or whether it will rotate and write down the opening to use for all interviews. Both of these tasks are easy to do and will greatly reduce awkwardness and increase consistency.

Having a designated leader for the interview keeps it on track and on-time. Writing down your opener makes sure you set the right context and reduces anxiety for the candidate. If they know what to expect, they can relax a bit.

As an example, here's the opener I've used on the last several interviews I lead:

Optional "Can you tell me how to say your name correctly?"
Introduce yourself, title, history, then pass the mic to another interviewer.
"This is a conversation, not a test. We’re looking to hear about how you approach technical problems and come up with solutions. We are looking for details: specific approaches you would take or have taken before."
"We’re going to be taking notes during the interview. Don’t worry, we’re paying attention!"
"This is, of course, a two-way interview. We’ll leave 10-15 minutes at the end for questions from you but if have more questions about the company, the team, or the position, please send questions via email and we’d be happy to answer them."
"Do you have any questions or specific considerations before we get started?"

The leader should typically be someone who has attended at least a couple of these interviews before but it certainly doesn't have to be the most senior person on the team or the person who has done the most interviews. Consider rotating between team members so everyone gets experience.

As an interviewer, prepare for specific interviews

At this point, you should have a very clear idea of what you're looking for and a discrete list of questions that will get you there. Now it's time to really get to know the person that you'll be speaking with.

I usually do this the day of the interview, often blocking out an hour before the interview so the information is fresh in my mind. I do some combination of the following, depending on the candidate:

I read their resume top to bottom at least once, usually twice
I get familiar with the companies they worked for and what they do
I review their GitHub contributions, if they have any
I read a few posts from their blog, if they have one
I'll look through their LinkedIn, if they have it

What I'm looking for here are ways to personalize the questions that I have and hooks to keep them talking if I'm not getting a lot out of them. If you want, but don't need, experience with, say, data pipelines, then look for experience in their resume. If there is not a hint of data experience, and that's not a show-stopper, then don't ask about it. If they built Kafka topics and Airflow jobs for a year, then don't ask if they have data experience or not, ask them about "position ABC at company XYZ" and use the words they used. You'll show them that you cared enough to do work ahead of time and avoid wasting valuable time.

Be careful here, though, because the more personal you get, the less consistent you will be across all the interviews. Use the information you glean to pick your questions and tweak them, not to ask easy questions you know they have an answer for (or to ask questions you know they won't have a clue about). Use what you learn to preface questions, focus them, or eliminate them altogether.

One more thing I do at this stage is to write down an extra question or two just to make sure I have enough to work with. I find that that I'm the most nervous when I don't have enough questions to ask. 9 times out of 10 the interview is at risk of going long but every once in a while you get someone with terse, efficient answers and you're looking at the clock with 20 minutes left and nothing left to ask. Add a few questions at the end of your list so you end up disappointed you didn't have more time rather than stressed out that time is crawling.

During the interview

Hopefully your anxiety about potentially altering the course of a person's life is reduced a bit but, like all things that happen during the messiness of being a human, you are not guaranteed a fun time just because you followed my advice.

I find the hardest part about the interview itself is navigating the awkwardness I've talked about throughout this post. There are so many dynamics at play that it can be hard for all parties to stay focused on the task at hand. This often culminates in a situation that makes the candidate ramble, meander away from the question being asked, clam up, or misspeak.

The most important thing to keep in mind is that signs of anxiety are not indications that the person is a bad fit for the job.

In fact, being nervous could be a sign that they are a perfect fit because they want it so badly. I can think of a myriad reasons why someone have a tough time being evaluated:

Maybe this is a really important company or position to them
Maybe they're getting a vibe from you that it's not going well
Maybe there is something totally unrelated going on in their life
Maybe they're worried about the next question
Maybe they have an inferiority complex
Maybe they're introverted and these complex social situations are difficult to navigate
Maybe you're nervous and it's making them nervous

Whatever the reason is, it's critical to understand that it's up to you and your team to help them get through it, it's your burden to bear. You have the advantage of preparation, more authority to interrupt to get things back on track, and less at stake. Internalizing this will make it easier to do everything you can to keep things comfortable.

One thing I like to do, usually as the interview leader, is to come up with a personalized (not personal) ice-breaking question that gives me some information but should be easy for them to answer. If they're a regular open source contributor, I'll ask them a question about a specific library they're active in. If I see something interesting (and job relevant) on their resume, I'll ask them to talk about it. What you're looking for is a question that makes it easy for them to start talking and an early win that boosts their confidence. I can usually find a segue from that to a more pointed question, which has proven to be a great way to ramp up.

After that, the main thing I focus on doing is listening, smiling, and nodding. I want them to see me paying attention and plugged in. This is a challenge if you're a note-taker, like I am, but I can usually type a few key words out quickly and plug right back in. All of these things are signals to the candidate that you are interested in what they have to say and you care about what they are telling you. This boosts their confidence and allows them to do the best that they can.

And, really, at the end of the day, it all comes down to listening. We've all had the experience of spacing out for a few minutes and losing the thread on a meeting. When it's a weekly team sync or a topic that might not be directly relevant for your day-to-day work, that's forgivable and not too difficult to recover from. But remember: fighting awkwardness is your responsibility. If you're not listening, you're going to stumble over the next question, appear disjointed, or leave a big block of radio silence that your teammates might struggle to fill. If you pay close attention, on the other hand, it should be easy to find threads to pull on.

After the interview

At this point, I've certainly gone on long enough about this but I wanted to give you one more tip to make interviews more valuable and less painful. As soon as the meeting closes, go back to your notes (or start them) and fill out everything you remember about their answers, your impressions, and their performance. You're probably going to have a "thank god that's over" feeling as soon as you're off but every hour you wait to write down your impressions makes them less accurate and more vague. Get it over with and then you can move on to that Slack message.

I hope this was helpful for anyone privileged enough to be in the position to make these kinds of decisions for your company and team. I've grown to enjoy interviews over the year and it all comes down to practice and preparation.