In many data science projects , I've found myself collaborating with people from both different disciplines and different countries, all individually great in their respective fields. Although collaborating can be challenging, it helps bridge domain knowledge gaps, completes projects faster and reveals useful insights. In this article, I will discuss the challenges that I have encountered and share tips on how to work around them.
When I was selected to be part of the Data Science for Social Good Fellowship at University of Chicago, my team partnered with the city of Jakarta, Indonesia to build a machine learning system to improve traffic safety.
My teammates on this project came from different places in the world: a PhD Physics student from Portugal, and two students from the USA - a PhD Stats student from Colorado, and a Jurisprudence Law and Public Policy student from San Francisco. They were all experts in their respective domains.
Additionally, the teams were assisted by an expert in public policy, a traffic engineer, and various partners from Jakarta who monitor and analyse the data on a daily basis.
Seeing the challenge and value in collaboration
The setup I found myself in enabled me to see both the difficulty and intrinsic value of cross-discipline and cross-national collaboration. In my case, I was the only member of my team with a deep computer-vision and machine-learning background. Thus, to complete the project in 12 weeks, the team had to work together in order to learn from each other, find common-ground, and debate viable solutions.
This kind of teamwork and cross-disciplinary collaboration is really exciting because it fosters a range of benefits. Some of the ones I experienced include:
- Innovating through novel questions: When you are collaborating with other people who are domain experts, you are in a position to ask questions that haven't been queried yet. By giving someone in the team the confidence to ask the "stupid" questions, or allowing the expert to share domain knowledge that is "common" for them, the team is enabled to deep-dive into far more innovative solutions.
- Becoming a better teacher: Since the end-product is usually a computer system that requires more programming resources, you will have to teach your team how to program, or how to use a certain programming library. You might even have to explain a certain tool to help with the amount of work coming your way. In this way, it's a forcing function for you to become a better, more patient teacher.
- Developing adaptability: You also learn to work with people that have different personalities and explore different solutions. This, in turn, allows us be more adaptive individuals, work better with people in general, and work on larger projects in the future.
While all these benefits are attractive, we had to find solutions to some key challenges before we could truly "enjoy" their value. Here is how we experienced and worked around some of these challenges.
Not communicating optimally
When working with a team that has diverse skills and backgrounds, I realised that everyone usually has preconceptions that inform how they interact. This can create barriers between teammates. Similarly, depending on who naturally emerged as the team leader, some team-members take on a certain "superiority." If this is not broken-down, the team will divide into smaller groups and it'll be hard to work together. Everyone should feel free to contribute, be heard, and constructively debate existing ideas.
Tip: Spend time with your colleagues and get to know them better. Find setups that encourage interpersonal interactions. Go for coffee, spend lunch together, or make social plans after work.
This helps with communication, confidence and conflict resolution within teams, and it should be the project manager's responsibility to facilitate team cohesion early on in the project. Simple tasks such as a discussing and analysing each other's strengths, weaknesses and goals can help break the ice, and allows teams to get to know each other a little bit better.
Knowledge imbalances
Often when collaborating with people on a task, there are times when one party knows more than the other. I have also realised - when working with people who individually think they are great at what they do - that no one likes to be labelled as the learner and have their ideas disregarded. This makes it difficult to work together effectively and share ideas, because feeling vulnerable is not always comfortable. However, we need to feel safe enough to admit that we do not understand something, rather than spending valuable hours trying to figure something out on our own to protect our egos.
Tip: You should have the patience to walk your team through grasping the basics and allow them to feel comfortable to explore and learn. Don't forget: you were also in their position once.
In situations like these, it is first-and-foremost the responsibility of the more knowledgeable person to guide the team to some "knowledge common-ground." This is where everyone can begin understanding something comfortably enough to ask questions. Then, it is the team's responsibility to acknowledge that they each have a skill they are bringing to the table, in order to build a constructive and collaborative working spirit. If we ignore this responsibility, then only one person becomes the expert, which blocks people from feeling comfortable enough to be vulnerable and ask for help.
The "My idea is better!" problem
At the Jakarta project, we had video data without labels and very limited labelling resources. One party suggested that we turn the videos into stacks of image frames; in doing so, we could randomly select frames and label as many different objects as possible for a better object detection model. The other party argued that we needed to label video segments instead of image frames. Although both arguments were valid, we had to pick one.
Tip: If the team thoroughly interrogates each solution together, then, depending on the facts and feasibility, a conclusion can be reached amicably and efficiently.
Different solutions arising for the same problem is good for project morale, as they allow the team to discuss the project requirements thoroughly. Important things are often discussed, such as ethics, needs, wants and dreams. However, deciding on the "best" solution can be difficult, especially when there are no pre-existing reference solutions.
Both sides should be given a platform to debate why their method should be picked and how they propose to reach that goal. Explaining one's rationale gives teams a chance to do groundwork, learn new things and teach the team new things. This will inevitably make the final solution rock-solid and water-tight idea - more-so than if it were never discussed in the first place.
The ego
Lastly, working with people who individually think they are great at what they do can result in competing egos. Sadly, there is no way to completely solve this; but there are ways to make sure this doesn't affect relationships, or the project as a whole. If ideas or egos do clash, constructive debates are usually a good way to overcome it. Nevertheless, prevention is still far more preferable than cure.
Tip: Team leads and project managers should aim for prevention and early intervention, and regularly speak to their team members to monitor how different egos are working together.
Collaboration helped our project thrive
Our project thrived from having a mixture of representatives from different disciplines, and different countries. We could pool our experience and our knowledge, and push the project into the right direction. Cross-discipline collaboration helped us tackle different technical aspects, and cross-national collaboration helped think about the problem from different social perspectives.
As my example illustrates, in order to design innovative solutions, we needed diverse problem-solvers with different skills in the same room. While one thought of the technical solution, the other could consider whether it is socially acceptable, and the other domain experts could ensure that the solution is feasible. Instead of one person being relatively good at doing everything, you can have multiple stakeholders who are experts at different parts of the process.
Although it can be challenging, breaking discipline boundaries and collaborating is always a fruitful task. Individually we can use our skills to solve problems, but collaboratively we can change the world.
Raesetje Sefala is a computer science Masters student at Wits University in South Africa. She is using satellite images and computer vision to study the effects and evolution of spatial apartheid in South Africa. Her research interests include applying machine learning to help solve problems experienced by countries in the developing world. She also enjoys working on computer vision problems, spatial data problems, and a combination of the two!
