The tech hiring market has seen a downturn from its historic highs in 2021, and has been rocked by a series of layoffs and hiring freezes. If you’re scaling your development team, but are struggling with budget constraints or working with a smaller recruitment team, you need an efficient tech hiring process that leverages all the resources at your disposal.
How you adapt your tech assessment approach in response to shifts in the market and technological developments such as AI is critical to the success of your hiring process. In our latest Untold Stories in Tech Hiring event, we unpacked how companies can best leverage technical assessments as part of their hiring strategy.
The session was hosted by Hung Lee, Curator for Recruiting Brainfood. He was joined by panellists Matthias Schmeißer (Global Director of Talent Acquisition at emnify) and Ilya Sakharov (CTO at Codility).
Key factors impacting the tech assessment process
Ilya unpacked the macro-economic factors that are affecting assessment processes right now:
The market is currently saturated with senior developers as a result of widespread layoffs in Big Tech, with fewer companies hiring developers in volume anymore.
“If your hiring volume is lower, the costs of mistakes is higher,” he said. Companies are hiring more carefully to get the best quality candidates on board for their most critical roles.
In this context, investing in a great candidate experience will help hiring teams get the edge in hiring the best developers that are now on the market — especially for more senior developers.
At the same time that these market factors are playing out, we’ve seen the rapid development in fields such as AI, with LLMs (Large Language Models) becoming more sophisticated and widely used by developers in their day-to-day work.
“LLMs are disturbing the world of development. They are amazing tools, and they’re here to stay. It has tremendous implications on how we do things in software development, because it is abstracting the code generation away from people,” said Ilya.
When it comes to future-proofing your tech assessment process, hiring teams need to have a solid response to developments in AI, and ensure they are providing a excellent candidate experience along the way.
Strategies to respond to AI developments
There are two main ways you can respond to the developments in AI when hiring developers, according to our panellists: You can either come up with strategies to combat it, or you can work with it.
Working against AI
In this more reactive approach, you can proof your technical tests by running them through AI models, to help you learn what is easily solved by LLM models.
Ilya used an example from the process they followed at Codility, a platform that helps you assess developers’ technical skills. Codility vetted all their assessments with the help of OpenAI, the makers of ChatGPT, in the process making a tool available to customers to filter assessments based on ‘AI resistance’. Some tasks are easily solvable by AI, while others are more highly resistant.
- Easily solvable tech assessments include algorithmic tasks where you ask candidates to give you a certain output based on input (eg. ‘Create a function that does X’).
- More resistant tech assessments include multi-phase tasks where each phase depends on the previous, and context is necessary to solve the problem.
However, Ilya argued that this can lead to a ‘cat and mouse’ game with AI models that are always increasing in sophistication. Something that is resistant to AI today could be easily solved by it tomorrow.
Working with AI
A proactive approach would be to treat AI like any other tool you should leverage, just as developers are currently using it in their day-to-day work. When hiring, this has the following implications:
Assess the candidate's ability to leverage LLMs
Tools like GitHub Copilot and ChatGPT are becoming as widely used among developers as Google.
“This is something that we need to embrace and add to our assessment, it opens up the door to measure how people interact with these tools,” said Ilya.
For example, hiring teams can test how a developer would optimise their code using AI tools, which will give you a more complete picture of the candidate and how they would operate with AI as a day-to-day activity.
Following this approach has led to the creation of entirely new roles such as prompt engineering.
Follow a skills-based hiring approach
With the advent of LLMs, you can spend less time testing code generation and narrow technical skills. Instead, you can dedicate more time testing for crucial soft skills and whether the candidate will be a good addition to the culture of your team. This amounts to a skills-based hiring approach, where you test comprehensively for all the skills a candidate would need to function well within your team.
“In the past, we focused most of our assessment on technical skills, and less on social or transferable skills. We need to get a better balance, and consider the environment that people will function in: Will they add value to the current team setup and their way of working?” said Matthias.
In the hiring process, practically this could look like:
- Adopting psychometric or personality tests.
- Following a STAR approach in your interview questions. Instead of just asking about someone’s responsibilities in a previous role, ask them to unpack their actions and the results of specific situations.
- Ask developers to present something to the rest of the team, use pair programming, or ask them to explain technical concepts to a non-technical person.
Ultimately, you also want to get an idea of how the candidate collaborates with others and helps colleagues improve by sharing their knowledge and best practices, especially if you’re hiring seniors.
“Validating someone’s collaboration skills is even more important for senior positions than the ability to generate code,” said Ilya. “You want to test how people tackle a problem together: How they optimise, iterate and create a solution in collaboration with someone else, which will be an everyday activity.”
“Having an assessment process that is very close to the real-world dynamic is going to give you a better window into candidates’ abilities, and a more accurate hire at the end of the day,” added Hung.
Strategies for a good candidate experience during the tech assessment process
Providing an excellent candidate experience during your tech assessment process increases the chances that you make the best possible hire, lessening costly hiring mistakes and building your employer brand in the current hiring climate.
Here are the panellists tips and best practices to get it right:
‘Pitch’ the opportunity before sending an assessment
“There’s a fine balance of when to engage with people and ‘pitch’ the company versus when you start assessing them,” said Matthias.
This matters especially if your employer brand is less well-known: You first need to show candidates that your company and opportunity is worth the time it takes to complete a tech assessment.
“The first rule is to analyse your own programme in the developer community, and figure out exactly where you stand before you think about what sort of tests you can deploy,” added Hung.
Prepare candidates on the process
First have a conversation with the candidate to explain your process and the people involved in it, before moving ahead with your assessment.
“Even sending a small video to explain what each session is about will do magic,” Matthias said.
Give comprehensive feedback
Learning opportunities needs to happen on both sides during the hiring process. Have open feedback channels between yourself and the candidate. This extends to providing comprehensive feedback to candidates that you reject in your process.
“If candidates can learn something from you by going through your test, that’s golden. You can create a great bonding experience by talking them through their results,” said Matthias.
These recommendations are backed up by data from OfferZen’s State of the Developer Nation reports: In both Europe and South Africa, transparency about the role and interview process is the top factor that developers appreciate in a hiring process.
Conclusion
Key takeaways to future-proof your hiring process:
- When responding to developments in AI, you can either work against AI (a reactive approach) or work with AI (the proactive approach).
- Leverage AI as part of your hiring process means you will test candidates’ ability to leverage LLMs and follow a skills-based hiring approach.
- To provide a good candidate experience: Pitch the opportunity to candidates before sending an assessment, prepare them thoroughly on what to expect, and give comprehensive feedback.
Ultimately, keep developers’ experience top of mind to future-proof your tech assessment process.
This means adapting to the ways they’re working in their day-to-day lives with tools such as AI, testing for the real skills they’ll need on the job, as well as taking simple steps to provide an excellent experience of your hiring process.
Read more
- ChatGPT is Breaking the Technical Assessment — What To Do About It
- How to Conduct an Effective Tech Assessment. This chapter from our Ultimate Developer Hiring Guide provides a breakdown of common types of tech assessments, including online assessments, technical interviews and take-home assessments, to help you choose the most effective method for your hiring process.
- Candidate Experience Journey Best Practice
