Apr 12, 2017

Beyond the Whiteboard: Software Engineering Interviews at Zymergen

As a relatively young startup, we recognize that early decisions in a fast-growing company have a way of defining company culture for years to come. We also know the engine of growth for a startup is its employees; even more than raising capital, attracting high-quality talent is imperative. It’s the talent we hire who will embody the company culture we’re aiming to build — one that is collaborative, pragmatic, and takes pride in producing high-quality work.

We’ve previously talked about “Finding a Startup with Values that Matter”, and for us, building a company that reflects our values starts with our hiring process — not just who we hire, but how we hire. We’ve asked our CTO, Aaron Kimball and our VP of Software Engineering, Dmitriy Ryaboy, to provide some insight into our approach to building the software engineering team:

Beyond the whiteboard

In the Bay Area, we face an intensely competitive hiring field for top software talent. We feel this hiring pressure acutely, as Zymergen has always been as much of a software company as it is a biology company. From the beginning, we wanted to be able to hire programmers from a wide range of experiences and backgrounds, not limiting ourselves to those already familiar with genomics or bioinformatics.

Zymergen was created to tackle an incredibly complex issue — we want to build a general-purpose approach to designing, engineering and optimizing biology. We believe that to make progress in this ambitious mission, we must attract and cultivate talented individuals diverse in thought and in experience (sometimes referred to as “cognitive diversity”) who fully embrace collaboration. The key element we hire for is ability to succeed in a highly interdisciplinary, collaborative environment. We value skills like being able to explain complex subjects in an accessible way, as well as to listen and quickly absorb new information. We also value practical, real-world experience in building systems — knowledge of how things fail, how to design for simplicity and robustness, and a certain amount of wariness when it comes to unproven technologies. Strong coding ability is only one of the multiple dimensions along which we need to evaluate candidates for the software team. With this in mind, we reinvented our on-site interview process, eliminating traditional whiteboard coding tests, focusing instead on technical communication, adaptability, and creativity.

Our on-site interview does not include the “Valley standard” whiteboard coding component. We decided early on that the ability to solve programming puzzles under artificial time constraints without the tools one would normally use to write code simply did not seem to be a good measure for evaluating the skills we were looking for. From our personal experience in top tech companies, as well as widely-cited studies on the subject, we knew that this type of test yields a number of inherent problems of bias and is at best loosely correlated with on-the-job performance.

Instead of repeated bouts of coding on the whiteboard, our on-site interview consists of a technical presentation, followed by a series of conversations about technical subjects. Our goal is to understand the breadth of the candidate’s experience, their approach to designing real-world solutions, and their curiosity.

We ask our candidates to give a presentation on a project they’ve done in the past that showcases their abilities as a well-rounded engineer. This is an opportunity for candidates to authentically present themselves in their best light by speaking on a subject they’re already familiar with, allowing them to create a basis for the subsequent one-on-one conversations. We make sure expectations are clear well in advance and even send a simple example slide deck to help explain what we want to see. The hiring manager is available for the candidate to ask questions as they prepare the presentation, and offer to give feedback on proposed topics and preliminary slides.

The presentation is followed by a series of one-on-one conversations. Each interviewer has a preassigned topic, such as “software architecture” or “testing and reliability”. We train all interviewers not to ask questions that test for recall of specific and somewhat arbitrary facts that are easily discoverable online (“what is the worst-case performance of merge sort?” “what does NUMA stand for?”). Instead, interviewers encourage candidates to talk through problems, sketch/diagram solutions, and generally have as much of a collaborative problem-solving experience as possible. The “testing and reliability” interview becomes less “can you find a tricky edge case” or “have you ever used Mockito for Java unit tests?” and instead becomes more of “let’s say I’m writing a Slack bot that lets users trade stocks — how would we test it?” Such a question acts as a prompt for a technical collaboration, rather than an interrogation; it allows the candidate to demonstrate, for example, how they think about reliability and reproducibility.

In addition to questions that seek to determine a candidate’s technical expertise, we also try to find out how they approach work in general: How do they approach planning? What happens when someone on the team has a strong opinion different from theirs? What do they do when they think the team is making the wrong decision? For all such questions, we try to get real, specific examples from the candidate’s past, and gently steer away from discussions of generalities.

This interview format has allowed us to hire talented people who might not have been able to show their full potential under the artificial conditions of whiteboard coding tests. Through our process, we’ve been able to learn something new from every single interview candidate.

A new Zymergen engineer doesn’t just need to learn about our internal systems, coding conventions, and processes. They also need to absorb the vocabulary of a whole new field, understand the basics of strain engineering and industrial fermentation, and learn to combine and apply these concepts to correctly and effectively develop new software tools for this domain.

We designed our interview process to identify and attract people who would be excited by such challenges, and thrive in such an environment. We’ve tried to be thoughtful about building a hiring process that levels the playing field for candidates so those with the right skills can shine. This meant changing the standard interview playbook. By allowing time and space to have quality, in-depth conversations with the candidates we interview, we are able to get much more of a complete understanding of what an individual is like to work with than we did when we subjected them to repeated tests of impromptu coding skills.

As a result, we’ve built a team of passionate individuals who are great communicators, careful and considerate thinkers, and excellent programmers.