What is collaborative modeling?

Collaborative modeling is about including stakeholders in the design decision-making and modeling process, and managing social dynamics and aligning mental models. This goes beyond the software architecture domain, but we will focus on that specific domain in this book. We see a trend where teams get more autonomy on their own software delivery lifecycle, including software architecture and design that impacts the teams’ software they build. In order to come to the best solutions, they need to model that solution together with their relevant stakeholders.

Designing a software system is difficult. Understanding the context in which a software system lives is more difficult. In order to understand that context, developers need to communicate with the business. But there is a lot that can go wrong while communicating with the business. For example people can misunderstand what is being said, they are scared to speak up, tensions could grow, and conflict can arise. In order to avoid these miscommunications, it is better to use visual techniques to make assumptions more explicit which will improve collaboration between developer teams and the domain experts. That improved collaboration will let software teams understand better the context and underlying problems the business has. That understanding enables the software teams to design and build sustainable quality software products. In this book we will teach you how to apply different visual techniques and how to facilitate better communication via collaborative modeling. In figure 1 you can see an example of a collaborative modeling session.

Figure 1. Example of how a Collaborative modeling session might look like. The picture shows a group of people collaboratively visualizing their conversation which enables them to make better design decisions through a shared understanding.

Collaborative modeling is a visualization technique to analyze complex and conflict-laden decision-making processes with all relevant stakeholders and decision-makers to create a shared understanding. Collaborative modeling goes beyond the software architecture domain, but we will focus on that specific domain in this book. We see a trend where teams get more autonomy over the software systems that they build. And building software systems includes software architecture and design. And if we do not let software teams create a shared understanding through collaborative modeling with stakeholders, how can teams know how to design and build sustainable software that is flexible enough to change with the business needs?

A brief history of software design

Part of our job as consultants is to facilitate collaborative modeling sessions between engineers and stakeholders. During these sessions there is a lot of individual and group behavior that results from the interaction between people. We refer to that behavior as social dynamics. What we notice is that most engineers struggle with these social dynamics when introducing collaborative modeling in their team or organization. Which makes a lot of sense, because most engineering studies don’t teach a lot of social sciences. We believe that those struggles are one of the main reasons most engineering teams aren’t successful at starting collaborative modeling. We think that it has something to do with where we, as an industry, came from. Twenty years ago Waterfall was the de facto standard for most companies developing software. The Waterfall model splits the activities of building software in small sequential parts. The waterfall model was developed by copying the manufacturing and construction industry’s project management. It was successful in that industry because it gave the client certainty of costs and time, but as a downside left little room for changes. The model was introduced in the 1970s when most computer programmers created, edited, and stored their programs line by line on punch cards. Changes after the design were costly and took time, so the waterfall model made sense.

The Agile theater

Winston W. Royce, in his 1970s paper “Managing the development of large software systems,” was the first to write down a detailed diagram of the process without calling it waterfall, which you can see in Figure 2. However, he also felt it had significant flaws stemming from the fact that testing only happened at the end of the process, which he described as “risky and invites failure.” Fast forward roughly 50 years, and we still base most of the software development approach on the same diagram. Yes, companies did transform to an Agile way of working by splitting up the work in smaller batches and delivering those smaller batches to the users faster so they were able to adapt quickly to the users feedback. Still, we observe that it is usually an Agile theater, implementing scrum in the development teams, but keeping the hierarchies and the idea-to-production value chain in place; only the development teams are doing agile development whereas the business side of the company keeps a very waterfall mentality.

Figure 2 – Detailed diagram of the waterfall process

We believe this mindset is the result of how software engineering is perceived or misunderstood by people who need software to run their organization. These organizations want to know how long it will take and what the costs are, similar to the manufacturing and construction industry. They let architects gather business requirements and design the system and let those designs be built by software teams. When someone without an understanding of agile software development with a waterfall mentality wants software to be built, they will perceive any Agile artifact like collaboration as a waste of time. That person sees it as a waste of time because in their mind they just need to give a list of requirements to calculate the costs and estimate the work, and then let it be built.

The difference between designing and building in the manufacturing and constructing industry and the software industry is changing requirements. For example if we want to construct an office building, we can gather requirements and design the building upfront. During the time of construction there is hardly any change needed in the design. There might be some adjustments, true but the main purpose and its functions of the building hardly changes after design. Which is different for most software systems we develop. As soon as a user interacts with a new feature of the software system, that feature might be exactly what they want, or there is potential and it needs improvement, or it is not at all what they want. Usually it is the latter two, because we are building a solution for the problems of our users, which hasn’t been built before. Unlike an office building, which we have been doing for many years, and has a lot of good and best practices on how to do it. But in the software industry we are mostly building software that has not been built before. Because if it had been built before, we could just copy-paste it or just buy it off the shelf. So you can expect a lot of new insights once users interact with our software. So when building software we can only know a clear solution in retrospect.

So creating software requires a different approach to design than we do in manufacturing and constructing. Designing software requires an Agile approach to architecture. This means collaborating on design, experimentation, emergent design, prototyping a solution, and a deep understanding of the problem by the engineers building that software. When organizations start to work in an Agile way, most of them do not know what to do with the software architecture and usually the software architect keeps designing and gathering requirements in a waterfall way. That can create a bottleneck for the development teams. The teams either need to wait on the software architect or they start building the software without doing software architecture because it is considered not being Agile. This is what happens often, a company’s software architect leaves, and the company finds itself without the necessary software architecture knowledge in the team. However, a key aspect of Agile Architecture is that the team has the knowledge to take up the role of software architect.

Architects as the change makers to an Agile Architecture

If the role of software architect and the knowledge to fulfill that role has to reside within the team, we need enablers that make it possible. Collaborative modeling tools, such as EventStorming and Example mapping, are one of those enablers. They are straightforward, easy to use and allow you to collaborate on the software architecture. When you are collaborating together, those social dynamics we mentioned earlier pop-up. We experienced this first hand and we were at first unable to deal with that. We got stuck in situations where nobody was talking or deciding, where we kept cycling back to the same topic and the same decision. Situations where we thought we made a group decision, but in the end experienced that most of the team did not agree and delivery was postponed. Or in instances where two people cannot agree, and start pushing their opinion in the architecture. Dealing with these social dynamics during collaborative modeling is essential for making sustainable design decisions. Design decisions where we include all the knowledge the group has to offer and bring out the full potential of that decision, and are supported by the entire group. And that is something we do not talk or teach enough of in software architecture!

The software architect must play a vital role in changing these social dynamics. Because software architects are usually people who are already involved in designing software in the organization they can be the enabler. They can be the catalyst that drives teams and the rest of the organization to start designing software in an Agile way. Software architects can let teams make their own design decisions, advise them and not impose design on those teams. Software architects can do that by starting to facilitate collaborative modeling with the teams. Facilitate collaborative modeling to let them be able to design software together with stakeholders, decreasing the feedback loop from requirements to production, creating higher value more quickly, timely and reliable to their users. But just being a facilitator is not enough, software architects must also understand and be able to teach the teams collaborative modeling and its social dynamics challenges so teams are able to do it for themselves. When teams can design software for themselves it removes the bottleneck for changing software in the team, and frees up the software architect outside the team to make sure teams stay aligned with each other and with the vision and goals of the organization.

You don’t need to start collaborative modeling with stakeholders, it is sometimes safer to do it within the team first, so members can make mistakes and learn from them. These successes will eventually lead to other teams doing the same and domain experts starting to join. Not only can software architects facilitate and lead collaborative modeling sessions, we taught a lot of other roles in software development to do this. Product owners and product managers, CTO’s and Engineering Managers, User Researchers and Business Analysts, basically everyone who is involved in the software development process. As long as you have someone with software design and architecture experience, anybody can facilitate and lead collaborative modeling sessions. Everybody can teach teams and others to run collaborative modeling and the social dynamics involved.

The software architect as the catalyst for better design decisions

Everything we discussed so far leads up to our mental model of collaborative modeling sessions. This book is about looking at software architecture in a new (or different) way. Not just from a technical perspective, but also from a business perspective. All whilst taking into account the social dynamics that affect both collaborative modeling sessions and architectural decisions. This is what makes this book different; we didn’t find many books on software architecture that talk about being able to adapt when company strategy or business strategy changes. From experience, we would find this very valuable and relevant. Which is why we ended up writing it ourselves. In this section, we’ll give you an overview of what to expect in the book.

Collaborative modeling, design and architecture

As mentioned in the previous sections, we find it essential for teams to make collaborative design decisions themselves because we have experienced the opposite. We experienced the frustration when provided with an unrealistic solution by an architect from above (in the famous Ivory Tower). We had good suggestions to add to the solution which the architect was unaware of. It made us build unsatisfying software for ourselves and for our users. We wanted to change that; we wanted teams to be their best version, to build software they could be proud of, the best software there is for the users. For us, collaborative modeling helped us to be the catalyst for that change in our own careers. We experienced how it helped in reaching the full potential of a group and its wisdom. How it led us to better models and solutions.

Figure 3 Mental model of a collaborative modeling session.

Figure 3 shows our mental model of a collaborative modeling session. It presents the facilitator as a catalyst that helps the group make sustainable design decisions. On the left of the facilitator, you see the heuristics, practices, principles, and tools we bring to the table for collaborative modeling. Let’s say that these are the ‘chemicals’ that are useful to do collaborative modeling. We need to mix these different chemicals, so we can align with the group’s needs. With chemicals, we mean various things like specific modeling tools like EventStorming, Wardley Mapping, Example Mapping, Business Model Canvas, and more. Disciplines such as Domain-Driven Design can also be seen as a chemical. In addition, we also can use several styles to provide a format to these sessions. We consider these different styles, including ‘modeling together alone,’ ‘split and merge,’ and ‘diverge converging groups’ chemicals too. Last but not least, heuristics are important chemicals. Throughout the book, we will share and explain some of our personal heuristics that are hopefully inspiring for you.

This means there is a lot to deal with as a facilitator of collaborative modeling sessions. Which tools best fit the needs of the group? Which style or formats would work best for these people? How could they best be combined? All things you need to think about beforehand, be able to anticipate and use in a flexible way when required. In line with our mental model, you could say that mixing these chemicals (answering the previous questions) is hard because they might just explode and can seriously damage the group. Collaborative modeling sessions can be a pressure cooker, and adding the wrong chemicals can have serious personal consequences for people. That is why we are writing this book on how to use and mix these chemicals yourself. We will give you our list of potential chemicals, but it will be necessary to start setting up your own table, add new chemicals and mix them! Get used to the chemicals, and experiment with them but do so in a safe environment where you can fail without risk. This will enable you to become that facilitator as a catalyst yourself.

The collaborative modeling process and the potential benefits of facilitation

Collaborative modeling sessions are very fluid, and we adjust where needed by adding different elements, or chemicals. The collaborative modeling process itself is more structured to ensure successful sessions and follow-ups. We will describe how to prepare, do a check-in, facilitate the session, do a check-out, do a retrospective and communicate and document the outcomes and results. You see it on the poster in figure 1.5 in the top left corner, just like a periodic table poster that hangs in all chemistry labs. We need certain elements in the process to make it a success. We use these elements together with chemicals so that the collaborative modeling group can design their shared model.

The impact of social dynamics on collaborative modeling sessions

However, lurking in the group’s shadows is a lot of social dynamics like ranking, cognitive bias, conflicts, and polarities that can congest the flow of a session and stop the group from moving forward and making sustainable design decisions. We use the metaphor of using a flashlight to observe the shadows and manage the potential inner demons people have, as you can see in the model. As shown in our mental model in figure 1.5, these inner demons can overshadow the group’s potential, knowledge, and wisdom to create a shared model if they are not addressed properly. It is hard for a group to see these shadows and inner demons, which leaves room for the facilitator to make the group aware of them and help them deal with them. If we do not let people deal with their inner demons, they will get stronger and bigger until they eventually step out of the shadows and turn up in the group. As you can imagine, this will highly impact the outcome of the group. We will dive into conflict, ranking, cognitive bias, resistance and polarities. In these chapters, we will also provide techniques to address and manage these social dynamics and inner demons that live in the group’s shadows. One important thing for every facilitator is that sometimes groups are not ready to face their inner demons. Even though you might see what is keeping the team down, in the end, it is their decision to deal with it or not.

If we are not careful, these inner demons can also affect us as facilitators. If you are a problem solver and see the group unwilling to deal with their demons, that can be frustrating. Now we must be careful not to be swallowed by the same demon. We must deal with our own demons to stay neutral and to be the best facilitator the group needs us to be. A facilitator who loses neutrality can affect the group and feed those demons. At which point you are not helping the group deal with them anymore. We will elaborate extensively on which demons you can find in a group. Because the only way people will fight the demons is if we, as facilitators, create a psychologically safe environment where we can deal with them. And a significant part of the book will be about this subject, and in most chapters, you will find us talking about it.

Collaborative decision making

That shared model we created in collaborative modeling sessions can then be used by the team and stakeholders to make collaborative design decisions. More specifically, design decisions regarding changing the business model or design decisions on how to build the solutions. We want to ensure the decision is supported by the entire group. It does not mean the whole group had to be there during the decision or that everyone has the power to make it. But it is crucial to facilitate when a decision is made to get people involved in the decision to know what it takes for people to go along. Of course, we would always love to have everyone involved in deciding, and we believe that given enough time, we can make it happen, but we don’t always have enough time.

Now with our mental model (Figure 3) and this introduction, we provided a complete overview of what will be discussed in this book. And we hope you will stay with us in our journey to become the software architecture catalyst.

What will readers learn from this book?

A common topic for the past years has been “How do we collaborate on making architectural decisions?” Especially how do we let autonomous teams do their own software architecture, without imposing design on a given team?

The hashtag that has been coined for this process is: #AnybodyArchitecture. Collaborative modeling and being able to facilitate collaborative design decisions with it by being aware of ranking, cognitive bias and being able to resolve conflicts and manage polarity together with the teams is a key enabler of making that happen!

• Prepare and facilitate collaborative modeling sessions with tools, such as EventStorming, Business model Canvas, Domain storytelling, Example Mapping and Wardley Mapping.
• Pick and apply software design decisions and modeling heuristics.
• Structure, visualize and make sustainable software design decisions.
• Best practices for dealing with resistance, and to get all needed insight from the group.
• Create awareness around biases and ranking during a meeting.
• Best practices on following up after a meeting, so everyone stays involved.
• Document the meeting and report to stakeholders.
• And much more!

Summary

• Collaborative modeling is a visualization technique to analyze complex and conflict-laden decision-making processes with all relevant stakeholders and decision-makers to create a shared understanding.
• Collaborative modeling creates and maintains a deep and shared understanding of the domain, directs conversations through visualization, enables exploring alternative views and creates a shared and sustainable design decision.
• Collaborative modeling requires buy-in from the domain experts if you want to gain all benefits, and is also being perceived as a “big investment” by higher management
• Most companies did not change to an Agile way of software development, ending up with a so-called Agile theater where design is still being done up-front.
• Software architects can be the enabler of that change to an Agile Architecture through using collaborative modeling.
• Software architects need to understand and learn how to facilitate the social dynamics that are a big and important part of developing a software system, and they can negatively influence decision making.
• Collaborative modeling can help bring these social dynamics challenges to the surface, where they can be dealt with. This will lead to better design decisions and clarity in a team.
• Through facilitation we can become the catalyst for change in bringing software architecture ownership to the teams.

If you want to learn more about the book, check it out here.