Designing Interviews - Scaling Startup looking for Experience

Designing Interviews is a series that looks at different stages of tech companies from initial founding all the way up to unicorn size.

This article focuses on the context of a scaling tech startup. Generally the ambition for such a company is to grow while maintaining the iteration speed and culture that has allowed it to find its product market fit. This is a difficult process because the faster you grow the more time existing employees have to spend onboarding, teaching, and learning from new hires - but there is also an existential need to gain experience to solve business, product, and technical hurdles.

Designing and running an effective interview pipeline is hard. The goal is to create a balanced, thoughtful process that attracts the right talent, keeps candidates engaged, and sets them up for success once hired. That takes considerable continuous effort, but as long as you keep your situation and values in mind it can work. If this situation speaks to you and you're looking for help, feel free to reach out.

Imagining a Startup

When designing an interview pipeline, the first step is to assess where your business stands today. For the purposes of this post let’s imagine a startup called Xazzy that wants to save the world with more Jazz music - the founder fell in love with Jazz at an early age and it has always improved her mood. Getting more people listening to it, writing and playing jazz, sharing it just brings joy to everyone. Xazzy found product market fit by selling their system for use in elevators, waiting rooms, lobbies, etc.

They’re gearing up to scale but haven’t gone through a rapid growth phase before. They’ve built their engineering team through job postings and investor networks but lack experience hiring for senior roles, particularly in areas outside their expertise, like large-scale production environments. Due to that lack of experience there are some challenges like outages around releases that are affecting customer satisfaction and could brand reputation at risk if they continue.

Let's round out this scenario:

• Mission: Save the world with more Jazz music
• Values: Joy, Reliability, Compassion, Ownership
• Team composition: A solid team of mid-senior engineers but very few with production experience at serious scale.
• Challenges: Release-related outages and increasing demand for scalability, which necessitates more frequent releases.

Sounds like Xazzy needs to improve stability and engineering maturity. They also need to speed up product iteration while protecting a positive reputation.

Realizing we need more experience

Looking at the goals, from a technical standpoint stabilizing their backend infrastructure and raising the engineering bar seems worthwhile. Let's assume we confirm that with engineering and get alignment that a senior or staff-level backend engineer with experience in Go and scaling large systems would be the most immediate and impactful hire. Now, the question is: how do we interview someone who is beyond our own individual experience?

Its likely that among the shared experience of the existing team you have some talented and valuable people or Xazzy wouldn't have gotten this far. So part of the process needs to lean on their strengths while learning through doing. Thoughtful feedback is the best way to ensure your team is learning as much as possible, so paving the road and incentivizing it can be worthwhile. Pairing up to do interviews can help build confidence, mitigate individual biases, and ensure there is always a notetaker.

Job Description

Writing a good job description and title for the intended role helps attract good candidates. It doesn't do much to weed out spam and therefore I wouldn't be concerned with putting effort into it with that goal in mind. [[4]] Here's my list of things to check:

• Would my prospective candidate be searching for the job title I'm leading with?
• How can I quickly interest them that this is a good place to work and they'll have the outcomes they want? Investing in the copy about the company's brand here actually matters. [[5]]
• What is the candidate going to work on? People want to know what is expected of them.
• Requirements - make sure they're actually requirements and they're realistic. 20 years of Go experience simply doesn't exist yet, and years of experience is a very rough filter. Add qualifiers like production experience to indicate what you're really after. The best requirements are exciting to the best people for the job. [[4]]
• Benefits - don't make it a mystery whether you do or don't have benefits. Hiding the lack of benefits will always lead to resentment, and omitting them when you do offer them misses a chance to build excitement. 10% 401k matching? Amazing. 20 weeks of paternity leave - wow that place cares about it's people. Xazzy doesn't have that but it offers free Jazz lessons in the office with local bands.

Designing the Interview Loop

Xazzy has started to get a few resumes sent in and we've got to assess candidates promptly in order to grab the right talent. What kinds of interviews should they do?

While it might be tempting to cover all the bases with multiple rounds of interviews, keep in mind that every additional step risks alienating talented candidates. Skilled candidates often prefer shorter and more focused interview loops. They also appreciate transparency and being able to ask questions to understand the company better, so its in the company's best interest to allot time for that. Its also great to send out or even publish information publicly on how you interview - Block has had a few engineers write about their experiences and published information about levels. A number of companies' interview processes are written about at interviewing.io including Google, Meta, Netflix, Apple... but part of Xazzy's appeal is its not a huge tech company, so it should stick with a short loop that evaluates what it values most.

After workshopping with the team Xazzy decides it really needs an engineer with good system design, a match for its fun jazz culture, a solid coder, someone who has a proven track record of solid execution. They also want to balance how long the interview loop is, so they decide to start with:

• 15-20 min casual conversation
• Take home assignment that use's Xazzy's REST API to mix and play jazz styles and stream some audio.
• 60 min system design (10 minutes for questions)
• 45 min culture interview (10 minutes for questions)

After running with this on a few candidates they find a lot of candidates dropping off on the take home assignment. There are so many good articles that consistently point to measuring the process and re-evaluating against the results. So Xazzy changes it up:

• 15-20 min casual conversation
• 45 min live coding pair programming against Xazzy's REST API
• 45 min system design (10 minutes for questions)
• 45 min culture interview (10 minutes for questions)

Closing

Once you’ve found a strong experienced candidate that matches what you want, move swiftly to close. It’s essential to keep communication clear and move the process forward smoothly. Long, drawn-out negotiations or delays can cause candidates to lose interest or accept other offers.

Xazzy found a great backend engineer that passed its interview loop and is excited to join. It's their first time working for a startup and they're coming from a more established company. The founder takes some time to get on a call with them and tell them about their vision and why this engineer is going to have great impact. She talks about the challenges she sees on the horizon as well as ambitions.

Every company’s situation is different, but the more transparency you offer prospective candidates, the more likely they are to trust you.

Onboarding

Finally, onboarding is critical. Provide the new hire with the necessary tools—both technical and cultural—to succeed. This includes setting them up with their equipment, documentation, and meeting schedules. Have a plan for their first day, week, and month and make sure they're aware of it. The faster they feel comfortable and prepared, the faster they can start contributing to the business. Common pitfalls are:

• Delays in ordering a work machine, or having it specced incorrectly. Hash this out the day they accept the offer.
• Background check verification. Some companies require this; make sure the candidate is aware before they sign an offer.
• Missing onboarding documentation - where are the data and tools they need to do their job? The more they are provided with the better.
• No introduction - make a public announcement about the new hire where other team members can see it. Setup meetings with their peers. This is crucial for remote employees where social expectations can be unclear.

By structuring the interview pipeline with these stages, you’ll be able to hire the right senior talent, stabilize your infrastructure, and continue scaling your company with a strong and collaborative team.

References

1. "Why engineers don’t like take-homes – and how companies can fix them" - interviewing.io [[1]]
2. "The other half, or why interviewers aren't always great and how to make them better" - interviewing.io [[2]]
3. "We have the best technical interviewers on the market. Here's how we do it." - interviewing.io [[3]]
4. "How to write (actually) good job descriptions" - interviewing.io [[4]]
5. "3 exercises to craft the kind of employer brand that actually makes engineers want to work for you" - interviewing.io [[5]]
6. "What do the best interviewers have in common? We looked at thousands of real interviews to find out." - interviewing.io [[6]]

[[1]]: "Why engineers don’t like take-homes – and how companies can fix them" https://interviewing.io/blog/why-engineers-dont-like-take-homes-and-how-companies-can-fix-them

[[2]]: "The other half, or why interviewers aren't always great and how to make them better" https://interviewing.io/blog/the-other-half-or-why-interviewers-aren-t-always-great-and-how-to-make-them-better

[[3]]: "We have the best technical interviewers on the market. Here's how we do it." https://interviewing.io/blog/we-have-the-best-technical-interviewers-heres-how-we-do-it

[[4]]: "How to write (actually) good job descriptions" https://interviewing.io/blog/how-to-write-good-job-descriptions

[[5]]: "3 exercises to craft the kind of employer brand that actually makes engineers want to work for you" https://interviewing.io/blog/3-exercises-to-create-the-kind-of-employer-brand-that-actually-makes-engineers-want-to-work-for-you

[[6]]: "What do the best interviewers have in common? We looked at thousands of real interviews to find out." https://interviewing.io/blog/best-technical-interviews-common