There are some good stories to share from the Bitcoin bubble of 2017. Luno's story is about the difficulties we had getting everyone through our virtual door at the same time. Let's just say, we didn't manage on our first try. Here's what went wrong and what we did to make that virtual door less congested.
Luno is an international cryptocurrency wallet and exchange that serves a growing number of countries in the EU, as well as South East Asia and Africa. One of the biggest challenges we face is identity verification, an important aspect of any fintech company and what turned out to be a serious bottleneck for us when Bitcoin reached peak popularity at the end of 2017.
Since I'm part of Luno's onboarding-pod, this was my team's problem to solve. We have a developer for each platform (web, iOS and Android), a designer, a backend developer and a product owner. We take care of the entire onboarding process from verifying your email address and cell phone number to levelling up an account so that local currency deposits can be made to buy Bitcoin.
The crypto-hype broke our verification process
Crypto markets can be highly volatile: In 2017, there was so much public hype around cryptocurrencies that it caused a 3000% increase in the value of Ethereum when some major wall-street players announced they were getting involved. Rising worries about North Korea's nuclear threat sent even Japanese investors, usually known to keep cash "under their mattresses", flocking to alternative assets like crypto.
This excitement should've been an easy win for us as a crypto-wallet provider, right? Wrong. Our trouble started in the fourth quarter of 2017. The market price for most cryptocurrencies was "going to the moon". Suddenly, Luno was flooded with customers reaching their deposit limits but urgently wanting to buy more Bitcoin. Our exchange had been designed to scale up, but our verification processes couldn't handle the suddenly exponentially grown influx of requests. Queues began to get backed up. What used to be a healthy maximum two-day turnaround-time for customers now turned into a week, two weeks or even a month for some unlucky souls. Why? Let's take a look at our verification process.
Auto-humatic verification - Susan and the humans
For the most part, identity verification is very straight forward:
- A customer's ID number and name are captured.
- An identity report validating an applicant's address, date of birth, name and mortality is retrieved from a localised third party identity verification service.
- Verification proceeds if a match is found.
- Supporting documents may be requested.
This usually runs automagically. We've even given the myriad of automation solutions throughout our infrastructure an affectionate name: Susan. "She" is responsible for anything we can train her to do. At the time, along with the verification procedure mentioned above, she had the ability to review any documents uploaded through advanced machine learning techniques and Google Cloud Vision. She could also check for simple things that would reject documentation, such as black and white documents, name spelling mismatches, identity report mismatch and cases where an identity was found on a sanctions list.
Only things that could not be auto-classified needed to be handled manually.
In 2017, however, the amount of unclassifiable applications grew exponentially: Susan kept passing roughly the same percentage of uncertainties to her human counterparts for manual evaluation and they simply don't work nearly as fast as she does! (Sorry, humans.) For some customers with all their docs in order, the turnaround was still near instant. (Go Susan!) All the others, however, needed to go into a queue to get human eyes on their situation and this queue became longer, because of innocent mistakes on one hand and purposeful submitting of fake documentation on the other.
That's how our response time grew from hours to days to weeks - not ideal at all.
Why did we even need a multi-level verification system? Why didn't we just switch off verification, or at least loosen the reigns and allow more people to buy Bitcoin? We certainly would've seen more happy customers!
Why crypto-wallets like Luno need complex verification steps
Luno has a somewhat complex relationship with the banks. We have hundreds of thousands of customers depositing and withdrawing money in their local currencies to and from our bank accounts. To protect themselves from fraud, embezzlement and partnering in criminal activities, banks are regulated to track every penny coming in and out. Exchanges like Luno are not yet regulated the same way by government, but to maintain a healthy relationship with the banks, we must regulate ourselves. As long as the banks are happy that funds can be accounted for, Luno can continue to be a customer.
Alleviating pressure with more humans
That said, in December 2017, about four months after the crypto-craze had started, the length of our verification queues were exasperating. Each agent was responsible for around 4000 verifications. It was too much but loosening verification was absolutely not an option. At the same time, the continuous incoming stream of new customers didn't really allow us to redevelop Susan either. The quickest and, in the end, only viable solution at the time seemed to be: Add more humans!
We decided to hire temporary staff to come in as data capturers. Every available space in the office was somehow occupied: People on laptops in the kitchen, on the floors in hallways, outside on the balcony. With this temporary solution in place, we managed to get through the end of 2017. But what happened next?
Making verification processes scalable
By January 2018 the hype was over and we were settling down to a stable and steady growth. That's why we started 2018 with a design sprint: As onboarding-pod, our team's purpose was to:
- Come up with one or more creative solutions to our identity verification problems and then
- Iron out as many of the wrinkles we could potentially face before beginning to build or code anything.
After a design-sprint with the entire team, one of the areas we identified for improvement was, unsurprisingly, automation. That's why we started looking at Onfido, Jumio and a few other ready made solutions such as Trulioo, Checkr and Civic. This involved scanning through a lot of documentation and implementing them on a trial basis. In the end, we settled on Onfido, because:
- They had international support.
- Their pricing system was one we could work with.
- We could integrate with them directly server to server, keeping unnecessary SDK from being added to our frontend.
- They have success stories from comparable clients.
In addition, Onfido kind of works like Susan on steroids. (Sorry, Susan!) It's trained on a far larger sample of international data and much better at handling edge cases, so that we could get results far faster and far more accurately. Onfido is very good about picking up on any discrepancies in the document type and rejecting fraudulent cases without the need for our human team to get involved.
Testing Onfido
In order to test whether Onfido would hold up in Luno's specific use case, we first ran a couple of experiments:
The rate at which Onfido could update the status of a document made it look quite good:
- Onfido can indefinitely scale up the number of documents it can handle per second.
- Humans, on the other hand, can process only a limited number per minute, require training, office space, equipment and - of course - sleep.
What about the quality though? Can machine learning really identify everything a trained human agent can? To test this, we added customers verified by Onfido to a quality assurance queue. The reviewers compared Onfido's judgement to our policy rules and the decisions our own agents would have made. To our amazement and delight, Onfido didn't even break a sweat! In the first 19 out of 20 cases Onfido was spot on: The one gray area case was a selfie on top of a mountain - unlikely to be a live photo from the registration, which is one of the requirements we have.
Collaboration of the giants: Susan and Onfido
So has Susan "cached" out, sipping virtual cocktails somewhere on a virtual island? Not quite. Since Onfido is a paid service, it only makes sense to have Susan handle fast, easy and obvious wins. Anything she has a problem with, she hands off to Onfido for a second opinion. Our human agents now only become involved if both of our automagic giants are unable to make a decision!
Andrew is one of the "new recruit" software engineers at Luno as they rapidly expand. He's into web tech, mostly interested in delivering app-like experiences on mobile web in emerging markets.
