đź’» Conducting a technical assessment
A technical assessment gives you the opportunity to test whether a developer has the required technical skills and ability for the job.
If your goal is to build a high quality product, everybody on your team should be able to make meanginful contributions and be able to succeed on a technical level. If this is lacking among your team members, it can severely limit your ability to move quickly. That’s why an effective technical assessment process is so important.
When deciding on how to do this, the question that you should be asking yourself is: “Am I trying to filter people out, or am I assessing to find out what people’s strengths are?”
Sequencing also becomes a consideration at this point:
- An assessment that is administered closer to the beginning of the process should be designed to determine whether somebody possesses the baseline competencies that are necessary for them to perform well in a given role.
- One that is carried out towards the end can be more focussed on confirming what a developer's strengths are in order to ensure that they are given work that engages them from day one.
Something that a lot of companies often forget at this stage is just how competitive technical hiring is. Any developer that you’re talking to is most likely engaging with other companies as well.
Having a cumbersome, poorly designed technical assessment strategy is a good way to make sure that your process finds itself at the bottom of a developer's to-do list as they prioritise processes that are more respectful of their time.
Proper briefing is also super important when conducting any kind of technical assessment. A poorly briefed assessment project that you think shouldn’t take more than 4 hours can end up taking the whole weekend to complete if you’re not clear on exactly what the deliverables and goals of the project are.
Keep in mind that developers rate irrelevant tech assessments as a top pet peeve in a hiring process — avoid this by paying close attention to your brief, as well as how much time it will take.
More resources
Want to know developers' top pet peeves in the interview process? Read more in the 2023 hiring reports for the Netherlands and South Africa.
Technical interview​
A technical interview aims to assess the depth of the developer's knowledge about a particular technical skill. It is potentially the most abstract of the options available to you when looking to assess technical ability. While it can be the most convenient from a candidate experience perspective, it’s also often reliant on the person running the interview being highly technical themselves.
The effectiveness of a technical interview hinges entirely on the strength of the questions that you ask. While it might feel good to ask a trick question that somebody outside of your building might struggle to answer, you generally want to stick to questions that will allow developers to showcase their own skill.
If you decide to adopt this strategy, it’s really important that the interviewer has a good grasp of the problem that they’re asking the interviewee to solve. If they lack in-depth understanding themselves, they will will struggle to drive the discussion and get all the information they need to assess the developer's technical ability.
Example technical interview process
Read more about how to set up an efficient technical interview, including:
- A downloadable rubric for scoring the interview.
Whiteboard interview​
Traditionally, this type of interview refers to a candidate solving problem while writing code on a whiteboard.
The classic combination of whiteboard and marker can be a really useful tool to determine somebody’s technical capabilities, as long as you make sure to be extra clear up front on what it is you’re assessing. Presenting somebody with an empty whiteboard and telling them to start designing solutions can be stressful at the best of times and will probably not result in them putting down their best work.
The whiteboard interview is most effective at establishing whether the developer can logically structure a solution to a problem, which should be made clear up front or even in advance of the interview itself.
Remember that your goal should be to assess somebody’s potential rather than ask them to showcase how well they can perform in a made-up test scenario; you’re conducting a technical assessment, not planning a technical ambush.
More resources
If you would like to carry out a remote whiteboard interview, you can use tools like Miro or other online whiteboards. Read our top tips on getting started with Miro here!
Take-home assessment​
A take-home project is a piece of work that you assign to a developer to complete in their own time.
As with all of the other types of assessments, it’s important to be explicit about what it is that you’re assessing, what the deliverables are and how long you expect somebody to set aside for the project.
A well-briefed assessment project can easily be used as a jumping-off point for a technical discussion during an interview. Getting somebody to talk you through their solution can be incredibly insightful and can tell you a great deal about their decision-making and communication style.
Make sure you assign projects that are relevant to the work your company does. This has the added benefit of giving the candidate a glimpse of what it’ll be like to work for you.
If you’re trying to hire people who enjoy solving interesting problems, make the assessment as engaging and representative of an actual day’s work as possible. This is especially important if it’ll take hours to complete your assessment.
It’s worth noting that many developers rate take-home assessments as the most effective way to assess their technical skills — it’s well worth the effort of setting it up as part of your hiring process.
Questions to ask when discussing a completed assignment​
- What tradeoffs did you make given you had limited time to complete the assignment?
- How did you choose which areas to prioritise in the assessment?
- Given an extra hour, what would focus on?
- What was your thinking around using XXXX rather than XXXX?
Example take-home assessment process
Read more about how we use the take-home assignment in our hiring process at OfferZen, including:
- The benefits of the take-home assignment for our tech hiring team
- How we design and rate a completed take-home assignment
Online coding assessment​
If you’re trying to optimise your hiring process to be able to handle really high volumes of people, then an online test can be a good way to accomplish that. These kinds of assessments function well in a remote hiring process, and are usually designed to be completed in the developer's own time.
This makes online coding assessments much easier to fit into a schedule. It provides a well-defined block of time within which to focus while also preventing overly meticulous developers from taking a weekend to complete what should be a 2 hour assessment.
On the other hand, this approach is quite impersonal. If you decide to include this in your interview process, make sure that somebody takes ownership of it. It should be their responsibility to maintain the integrity of the assessments. This means switching up your tests on a regular basis, moving people through the pipeline and being available as a human to talk to if the candidate has any questions.
Additionally, it’s worth mentioning that there will usually be costs involved in using an online code assessment service, either in the form of a subscription or per-use fee.
The automatic scoring systems that many of these online testing tools use often lack the ability to pick up on nuance, meaning that smart developers often don’t get results that reflect their true ability. This can be counteracted by making it part of your process for a human to review the submitted code.
Pair programming​
A pair programming assessment involves assigning somebody from your existing team to collaborate with the candidate to solve a single problem. This can happen remotely, or by physically sitting at the same workstation.
This technique can an extremely effective way to gauge a number of key metrics, from technical ability and problem solving skills, through to communication ability and collaboration. It does however come at a significant cost in terms of preparation and the involvement of multiple members of your existing team, so it’s not to be undertaken lightly.
A common theme so far has been that briefing is critical to an effective technical assessment, and pair programming is no different. Well executed pair programming assessments will always be explicit up-front about the differentiation between the assessors and the people that the candidate will be programming with.
Poorly run pair programming sessions can easily result in a developer being hesitant to explore all of the solutions at their disposal, which is understandable when you consider how easy it is to feel like the person sitting next to you is judging your every keystroke.
No technical assessment​
While doing nothing is always an option, it could also cost you dearly. One bad hire early-on can do a lot to hamper your ability to hit product roadmap goals, not to mention the potential cascading negative effects on your company culture.
You might be saving time in the short term by relying solely on your intuition, but we strongly recommend doing some form of due diligence to ensure that the person you’re adding to your team is able to contribute meaningfully, shore up existing weaknesses and help you build an awesome product and team.
Platforms you can use to conduct technical assessments​
Comparison of technical assessments​
| Assessment | What does it assess? | Company time investment | Candidate experience |
|---|---|---|---|
| Technical Interview | Ability to communicate problem solving process as well as high level technical understanding of topics. | Dependant on the number of team members involved. | Dependant on the skill and level of preparation of the interviewer. |
| Whiteboard Interview | Ability to logically structure a solution and communicate your thought process. | No setup time outside of coming up with the problem. Can be conducted by a single member of the development team. | Generally positive as long as there is a proper briefing around expected parameters (eg. code vs pseudocode). |
| Pair Programming | Ability to solve problems collaboratively and communicate decision making. | Potentially high. A number of members of your development team will potentially be working with the candidate. | Can be quite daunting if it isn’t explained that the person they’re paired up with isn’t necessarily assessing them. |
| Online Assessment | Proficiency in a specific language and its syntax. Useful for measuring computer science fundamentals. | Minimal. Best practice would be to have a dev team member at least double check the results of the marking algorithm. | Quite impersonal, but potentially convenient due to being able to fit it into their own schedule. |
| Take-home Project | Ability to assess a project brief and model a solution in code. Ability to budget time to have the project completed before the deadline. | Upfront time investment to set a sane assessment as well as rubric for scoring it. Assessing the project and delivering useful feedback can be time consuming. | Dependant on the quality of the briefing. A poorly briefed project can result in candidates spending too much time answering questions which might not be relevant, which is frustrating. |
More resources
Is your technical assessment ChatGPT proof? With the advent of AI tools like ChatGPT, you need to rethink your approach to assessing developers’ skills. Here are key takeaways from Karin Bothma, Engineering Hiring Manager at OfferZen, on how to do this:
- ChatGPT is Breaking the Technical Assessment — What To Do About It
- Struggling to assess devs accurately in the wake of recent AI developments? Get practical questions and guidelines on how to approach it in our 2024 Hiring & retention report.