How I Helped Non-technical Users Identify App Issues with an Open Source Android Library

Engineers can often spend quite a bit of time helping team members identify the source of app performance issues. I decided to improve this process with an open source Android library and make it accessible for non-technical users. Here's how I went about that.

While automatic testing is proven to really improve software stability and to reliably prevent most issues from reaching end users, manual testing software is still necessary. There are always issues that automation can’t cover, so different teams will be doing their own tests continuously, for example:

• Developers test app behaviours described in bugs to ensure a development is well-executed and investigate specific app features.
• Quality Assurance specialists (QAs) and test engineers are constantly testing developments and executing test cases to ensure that apps work as intended.
• Team members from other departments test parts of apps related to their responsibilities, for example:
  • The Content Management System (CMS) team tests to check if the content is being loaded as required.
  • The design team tests to check if a feature’s design is consistent with the mocked designs provided.
  • The security team tests to ensure the security standards are maintained.

Even though the task of pinpointing the problems is usually assigned to QAs and test engineers, other teams often need the assistance of a developer to get to the root of the behaviour.

For example, when I was working on an Android app for banking platforms, I had to meet with a member from the CMS team to assess if the interaction with a specific Software Development Kit (SDK) was working as intended. Just by looking at the logcat, my colleague was able to understand all he needed.

This gave me an idea: If he had a tool that allowed him to check the logs he saw in the logcat, I would have more time to focus on other work, and he could get quick answers without having to wait to meet with me.

We had to make the process more efficient for all parties involved.

Figuring out what the solution needed

To reduce time spent on analysing behaviours in our app, I created Hermes: Hermes is an open source Android library that aims to empower all team members to more easily and independently perform software analysis. It enables all people who test our app to see curated information relevant to them.

To maximise the impact of this tool, it was paramount to talk with team members and really understand what information would make their jobs easier. For some teams, having tracking information can be highly beneficial, for others it might be unnecessary. Of course, every team and every product is different, so depending on your case, you might need to share different pieces of information with your team members.

After speaking with my team, we came to the conclusion that making the following changes would impact their jobs the most:

1. Configurations: Allowing testers to check configurations enabled them to map unintended behaviours to specific configurations, resulting in a clearer view of issues from the get-go.
2. SDK Interactions: Because SDKs bridge work between different teams inside and outside of the organisation, it is vital to allow testers to confirm if there is irregular behaviour in the interaction with the different SDKs. Making this information available allowed team members to easily check if specific problems were originating from a specific SDK, or if it was an issue on the app itself. This allowed us to, at times, skip the app-dev analysis process and go directly to the team that maintains the SDK to resolve the problems.
3. Services input and output: Similarly to the SDK interactions, having access to Services inputs and outputs allowed testers to easily assess if a specific problem was in the backend or frontend domain without the need for developers to get involved.
4. Fragment backstack: Fragment navigation can be messy, specially on Android apps that don’t yet support Android’s Navigation Component. Having fragment backstack available allowed us developers to more easily understand issues such as unsynchronised data between fragments and issues on a fragment’s lifecycle.

Until then, only the development team had access to this information.

Building Hermes

With the knowledge of what everyone needed, I had to find the places in the app where this information was available, and submit it to Hermes. It is important to not send unnecessary information, otherwise the team members will find it hard to parse through the data’s pollution. This is how I submitted events to the Hermes tool:

Hermes.i()

    .message("AllProducts API call")

    .extraInfo(result.data, format = DataType.JSON) // Formats the content as Json

    .submit()

The team members who would be using Hermes were not necessarily all technical. For that reason, it was necessary for the information in the events to be written in an accessible way.

To make the information more digestible for non-technical people, I added two main features to the tool:

1. Submitting and displaying events - Before, testers and QAs couldn’t access information about events without developers. To solve this, developers now place events in key locations. By submitting and displaying events, the event logs are displayed in an accessible way at runtime, so that team members can easily extract meaningful information from it and produce more informed analyses.
2. Sharing events - Previously, developers would have to dig for information on the actions testers took before being able to analyse the problem. By adding the option for all team members to share events, team members can now share single events or the entire event stack. This gives developers more information, which allows us to analyse the behaviours more easily and resolve problems more quickly.

Hermes removes step B, saving time and effort for developers, QAs and testers.

Potential for Hermes improvements

Of course, there are plenty of other ways in which one could improve Hermes to boost efficiency even further. Three features I think are especially exciting:

1. Supporting a Timber.Tree implementation - If, like in my old team, the teams already use timber as a logger, it could make sense to provide a Timber.Tree implementation that maps all Timber logs into Hermes events. This way, registering events in Hermes library can be done by placing logs.
2. Event filtering - This would allow events to be filtered by type or tag that can be placed in the events. With this feature, non-technical users can visualise events with specific types or tags, making it easier to detect the events they’re searching for.
3. Invoke commands - I’ve previously noticed that QA and testers sometimes really wish they could just “spawn” the BottomSheet or Dialog that is being tested, or even simply clear a specific SharedPreferences file to redo a test case. By placing transient and temporary commands, attached to a LifeCycleOwner, developers would enable QAs and testers to interact with the app in a more direct and open way. This will enable QAs and testers to easily perform test cases that otherwise could only be done by developers themselves, modifying the code-base directly.

Resources:

Check out the Hermes repository for library setup information.

Rafael Ramos is the co-founder of Neroes and an Android Engineer currently working for one of the largest online luxury fashion retail platforms. He always works towards process optimisation to allow users to receive more with less effort.

You can check out more of his work here.