Scrum and Scrum variants make up for about 70% of today's Agile methods and practices globally. At the same time, there are still many problems and failures during its adoption and implementation. In this article, I will discuss how to identify and categorise the different problems arising during Scrum adoption according to who they affect and how to overcome them.
What is Scrum and why should teams adopt it in the first place?
Scrum was developed in the early 1990's by Jeff Sutherland and Ken Schwaber. As an Agile methodology, Scrum is lightweight and simple to understand, but extremely difficult to master.
While it was initially developed for the software development environment, it is often referred to as a project management method that focuses on people over processes. This makes Scrum a collaborative and engaging method that can be used in any environment for almost any project.
It allows teams to work in an iterative and incremental environment, where the re-evaluation of requirements and progress is "baked-in". In other words, changes are expected and this is where communication and collaboration within the team does it's "magic".
Why use Scrum?
I've been involved in software development for a number of years and I've often seen that many companies and teams don't understand how the adoption of Scrum can be beneficial. After doing some research for myself, I realised that it can improve conditions on an organisational, team and individual level. The top 5 benefits elicited from about 10 different sources are:
- Quality: The iterative nature of Scrum allows a quick response to resolving defects, with potential for lowering product maintenance budgets.
- Improved teamwork: Teams communicate better as they are working together to achieve common goals.
- Improved time to market: Product increments are released faster and, as a result, changes can be factored in sooner with less of a challenge.
- Improved work environment: Allows the Scrum Team to work within a structure that encourages the sharing of ideas as well as open communication.
- Self-determination: The Scrum Team, with emphasis on the Development Team, has the autonomy to decide how best to complete the Product Backlog items within the confines of the Scrum framework.
What problems arise during the adoption of Scrum?
Organisational factors
There are several things organisations are doing that hinder a successful Scrum adoption. By 'organisation' I mean the place that is represented by the individual and team. The organisation determines the policies, roles, and structures within which the team and individuals can carry out their duties. The problems that can arise are the following:
Problem #1: Organisational structure
What is the issue?
A common problem within organisations is when they have a hierarchical structure where individuals and procedures still follow a top-down approach. For example, when an individual within the Development Team can't consult with a senior manager regarding a particular problem they encounter. The common blocker here is that they first need to follow the correct communication channels within the organisation's structure.
How to overcome it?
A start would be to consider creating an open and integrated environment. To do this, you can:
- Remove unnecessary "chain of command" structures by implementing a flat structure,
- Encourage cross-departmental communication, and
- Remove unnecessary department segmentation within the building.
Problem #2: Organisational culture
What is the issue?
When perceptions and beliefs have worked well for previous problems encountered within the organisation, they are taught to new members. This is because they are seen as the correct way to perceive, believe and think in relation to problems. An example of this would be a Development Team that has become cautious of implementing new technologies. This is often because the current systems are working and they are in a comfort zone.
How to overcome it?
A change in culture allows the opportunity for innovation to thrive within the organisation. This may require the company to:
- Encourage members of the organisation to think independently, and to constantly be looking for new and innovative ways to improve upon existing ideas or solutions.
- Allow the Development Team to have "Champions" of a particular system or technology. Champions are the individuals who put in time and effort to research and re-engineer existing systems, or to implement new systems and technologies. They also strive to improve the team's knowledge and skills.
- Allow systems that are not critical to existing operations to act as "guinea pigs" for the process of rewriting, re-engineering or upgrading.
Problem #3: Management support
What is the issue?
Innovation is not encouraged when management penalises employees for mistakes that they have made. For example, if management expects the team to create systems that do not have flaws, this comes at the cost of allowing the team to be innovative.
How to overcome it?
Management should support employees during the innovation adoption stage. In a nutshell, an idea goes through many stages before it gets implemented. It goes from the initial idea to being diffused throughout the organisation, to either being adopted or rejected, before the implementation prospects are even determined.
Many mistakes are expected to occur, and having the management team's support allows organisations to look at innovation adoption from a positive perspective. For example, don't view minor failures as a negative outcome, rather encourage this innovative behaviour.
But remember to emphasise that quality should still remain a priority in the long run.
Team factors
Another barrier in Scrum adoption are problems within the team that are causing them to operate sub-optimally. The team I mean here is the Scrum Team that mainly consists of Product Owner, Scrum Master and Development Team. It could however also include other stakeholders who can add value to the Sprint, such as Project Managers, CTO's and QA's.
Problem #1: Communication
What is the issue?
Lack of communication is detrimental to the success of any group working together to reach a common goal. Team members who do not communicate correctly can cause the sprint goal to fail.
How to overcome it?
- Encourage open communication, an important component of any successful team.
- Consider rotating the individuals leading the Scrum events e.g. daily-standups, retrospectives, reviews, etc. I have seen that it is a good idea to give individuals, who usually shy away from meetings, the opportunity to facilitate them. This usually encourages them to be more open to subsequent communication responsibilities.
Problem #2: Teamwork
What is the issue?
Teamwork challenges are a recurring problem within Agile development, as they affect the team's ability to reach the objectives that have been set. Team members that are not willing to share knowledge, or who are afraid of asking for help because it's seen as a weakness are both examples of teamwork challenges that could crop up. This mindset is counter-productive for the team and the organisation as a whole, because the sooner team members are "in the know" the quicker the tasks can be resolved.
How to overcome it?
This problem can be solved by encouraging individuals to respond constructively to their team members, respect their opinions, as well as support each other. As a Team Lead, I would often reiterate to the Development Team that they need to communicate and ask for help. Especially if they do not know something or are struggling. This helps the individual and the team to progress at a much quicker rate. "Super heroes" who suffer in silence, take on all the responsibility and delay task completion are actually not doing the team any favours.
Problem #3: Specialisation
What is the issue?
Specialisation doesn't allow for interchangeable roles. A typical problem I've experienced is when individuals with a specific skill set are not available when there is a problem. This means the rest of the team needs to jump through hoops because only the specialist resource can resolve the problem quickly - like Neo in the Matrix.
How to overcome it?
Do not have a team that is made up of specialist individuals. For Agile development to improve team performance, the team needs to be self-organised. That is, the individuals within the group need some degree of skill-overlap. A way of achieving this could be to:
- Make use of pair-programming, or
- Have monthly knowledge-share sessions where team members can share what they've learnt while completing a task.
Individual factors
Even individual behaviour within the Scrum Team can affect the team's success in adopting Scrum processes:
Problem #1: Escalation of commitment
What is the issue?
The individual within the Scrum Team who persists with a task even though it is not adding value to the project. Yes, this problem could also apply to the team - or the organisation for that matter. It has been identified here as an individual problem because it is often the developer who doesn't want to admit defeat and look for alternative solutions.
For example, a junior within my team took a few days while trying to integrate to a API using a specific method. After my inspection, I informed the junior that the solution would not work with the current approach, and that he should use another approach. This resulted in us losing time on a project that had a very tight schedule.
How to overcome it?
The Scrum Master should spot and share these impediments with the team, if they were not able to identify them themselves. Development Team members can be made aware of any issues with the help of Scrum events (e.g. Daily Scrum) and artifacts (e.g. Sprint Backlog/Scrum Board). This will prevent further resource wastage, but also creates a platform for the team to suggest alternative solutions.
It is important to approach these events constructively, so that each individual knows that asking for help is encouraged when they are struggling.
Problem #2: Experience
What is the issue?
Individuals who do not possess the necessary skills and knowledge to perform on the job. Since Sprints are usually short cycles of iterative and incremental development, the teams that generally perform well using Scrum are made up of more experienced individuals. In my personal experience, I have had to make adjustments to factor in the inexperience of the juniors and interns on the team. These include extending the Sprint duration or having fewer daily Scrums, which has the potential to lead to other problems.
How to overcome it?
This is a really big challenge to overcome, as experience is not a process or mindset that can be quickly altered to produce better results. The following options could expedite the transition to mastery:
- Training is a necessity, whether its done during or after normal work hours. I have seen that by encouraging continuous learning, an organisation can demonstrate that it is interested in the individual's personal growth. This can be done by offering training opportunities, course subscriptions and celebrating course completions.
- Mentoring allows the inexperienced individual to open up to learning from others within a safe environment that was created for this purpose.
Problem #3: Over-engineering
What is the issue?
Doing more than what is required by the client is often a result of:
- Lack of communication between stakeholders,
- Bad planning or
- Limited domain knowledge by the Scrum Team.
For example, over-engineering could be the result of a developer not understanding what was required from the Product Backlog item.
How to overcome it?
If anything relating to the Product Backlog item is unclear to the developer during the Sprint, they should be able to liaise with the Scrum Team to clear any confusion. A senior or team lead can usually elaborate on what is required. Otherwise the individual can consult the Scrum Master, who should take the problem to the Product Owner and other stakeholders on behalf of the Development Team.
For the sake of brevity, only three problems per factor have been included in this article, however, I could have included many more. As an example, over-engineering could have been mentioned as a Team factor as well, since inexperienced teams within the Scrum methodology are culprits of the problem. But, great communication is like good nutrition which prevents diseases from developing, and cases of over-engineering can be resolved through this wonderful gift.
Resources
What resources can help teams, individuals and organisations overcome these problems?
- Scrum guide: This is the go-to guide for Scrum introduction as it is a vital resource for entry-level Scrum certifications. This document is written by the "founding fathers" of the Scrum Agile Methodology, namely; Ken Schwaber and Jeff Sutherland.
- Scrum - A pocket guide: A good short book to read for anyone who wants to get a basic understanding of Scrum. It's written as concise and complete as possible.
- The state of Agile annual report: The largest and longest running survey on Agile, in the world! View the latest 12th annual report, which discusses the benefits of Agile adoption, Agile methods and practices, Top 5 Agile techniques, Agile project management tools, to name just a few.
Other sources that you can take advantage of include joining local communities like SUGSA (Scrum User Group South Africa), and keeping up-to-date with the annual Agile Africa Conference proceedings which attract industry experts to speak about their experiences.
Ridewaan Hanslo has been actively involved in the IT industry since 2002, with experience and credentials which include: being a Senior Software Engineer, Cyber Security Specialist, Advisor, Online Tutor, Speaker, Mentor and Researcher.
As a Professional Scrum Master who has been an avid Scrum practitioner and researcher, he has written and reviewed papers on the topic. If you want to see some of his work that contributes towards this article, visit this page.
