The Virtual War Room
The war room is a time-honored tradition of problem solving. When there’s a major crisis situation, operators, sysadmins, IT specialists and product owners join forces in the same room. Developers participate in cases that require their specific knowledge and even the new kids on the block, DevOps and Reliability Engineers, join everyone in the room and everyone maximizes brain power regarding the problem.
A few years ago when IT centers were much more centralized it was common for all the relevant siloed teams to send their representatives to the war room. There would always be someone from Ops, someone from Networks, someone from Databases and so on. Only the application representative might change from crisis to crisis, depending on the actual application impacted.
The benefits of bringing together the brains of everyone who can help solve the problem are obvious — it’s easy to communicate, share ideas and collaborate when everyone is adjacent to each other. When the application owner suspects that the app slowness is caused by a network issue it’s important to have the network representative within hand’s reach so they can check using tools unavailable to the application owner.
There is also often someone in the war room whose job it is to document the tasks performed by others — who is checking what, which solutions have been tried — and then he relays this information to other stakeholders who are not in the war room. Too often this is a manual process; steps may be missed. When researching exactly what happened after the event, it is often difficult to understand the actions that occurred in the war room.
Keeping up with the ever-changing event becomes a burden. Someone joining late must be brought up to date, often distracting the ongoing conversation.
Today’s systems are becoming ever more complicated and less centralized. It is more of a struggle to find out who are the right people to join the war room. Organizations often take the “better safe than sorry” view and their war rooms are full of people who are there “just in case the section they’re responsible for ends up being a part of the crisis”. I’ve seen war rooms where some people are busy working on multiple tasks simultaneously while others are simply reading their emails…
There are also downsides to having everyone grouped in the same physical location:
- Major incidents, with multiple participants, can lead to a cacophony where people are talking over people and people are not listening to each other. Human beings are not skilled at managing multiple conversations simultaneously. Even worse, you run the risk of people with louder voices overpowering others.
- While people may be physically together, they often work independently and do not share information. Even worse, the information that is shared is often written on temporary whiteboards & flipcharts or consists of someone “showing their screen” to someone else. The information is not recorded (or is difficult to record) for future reference.
- Worse than not working as a team to solve a problem is when small groups splinter off and try to solve the problem in their own way. Not updating other people makes it difficult to keep track of what’s going on and who’s touching which piece of the puzzle.
Physical rooms are limited in size. As problems grow faster than the available real-estate there is often a struggle to find enough chairs for everyone in the same room.
- Multi-national organizations may find themselves solving problems that cross physical borders and need cooperation between people who simply cannot co-locate.
- Different groups may set up physical war rooms in different locations because they think they are dealing with different problems. It then becomes difficult to unite the disparate teams.
- As companies encourage remote work for multiple reasons (at least one of which looms large as I write this in March of 2020), they may find that many potential war room members are working from home and cannot join in.
There are solutions which transform a physical war room into a virtual one:
The first is simple conferencing — whether it’s a phone call, a video conference, screen sharing or any combination, conference calls allow people to work together while not being together. However, conference calls suffer from many of the same issues that physical rooms do.
- They do not solve the problem of having many people working together — either some conversation is drowned out by other conversation or multiple disconnected conversations occur simultaneously.
- They do not solve the problem of onboarding new people to help solve the problem — it is time-consuming to bring new people up to date, especially if the various tasks being performed are disjointed.
- Even worse, when some of the members of the war room are physically present and others are “merely” virtually present it creates a hierarchy of attention. The virtual members must fight for attention as the co-located members can easily step back and have invisible, whispered, conversations.
- As long as the center of the attention is the physical room, virtual members will not be full participants; while they might be able to see a white-board through the conference camera, there is no way to grab a marker and start writing on it!
Another solution, which resolves these issues, is commonly called ChatOps and it is the use of a collaboration platform to create a virtual war room where the various team members collaborate.
Unlike a physical room or a conference call, it is easy to manage simultaneous conversations in ChatOps by dividing the written conversation into channels or threads. Even better, since the conversation is (by definition) documented in the collaboration platform, it’s easy for people who are on-boarding into the war room to read what was said.
The use of a virtual war room is a great equalizer — people who are physically present in the building and people who are on the far side of the continent are on equal footing and there is no “louder” or “nearer” person who can drown out conversation. While it is not possible to have multiple verbal conversations simultaneously, the use of a-synchronous text messages and conversation threads means that multiple options can be raised in parallel and dealt with in sequence.
The use of virtual war rooms, using the same collaboration platform that the entire organization uses means that it’s easy to pull in people who might have balked at being summoned into a war room (e.g. developers who might feel out of place besides network operators).
Of course, a virtual war room lacks the immediacy of a physical one which is where the benefit of using a collaboration platform and the use of bots comes into play. ChatOps opens up the capability of automation which injects useful information into the conversation. Unlike collaboration using physical screens or screen sharing, a ChatOps solution can inject images from multiple applications simultaneously. The information added by ChatOps is permanent and is not be limited by the size of a whiteboard.
ChatOps is the integration of development tools, operations tools, and processes into a collaboration platform so that teams can efficiently communicate and easily manage the flow of their work. It does not replace any specific existing tool or role but enables the creation of a virtual war room where collaboration is faster, simpler and easier — both for humans and for the bots that help them.
Once the problem has been solved, the conversation history audit can be used for post-mortems and retrospectives to analyze how the problem was handled, get to the root causes and find improvements for the future.
This virtual war room makes problem-solving simpler (and therefore faster) because it is a more efficient way for people to multi-task without interfering with one another. It is self-documenting which makes learning and improvement easier. And as a final important benefit, it allows people to collaborate efficiently without forcing them to be physically together.
Bring your plan to the IBM Garage.
Are you ready to learn more about virtual war rooms?
We’re here to help. Contact us today to schedule a time to speak with a Garage expert about your next big idea. Learn about our IBM Garage Method, the design, development, and startup communities we work in, and the deep expertise and capabilities we bring to the table.
Schedule a no-charge visit with the IBM Garage.
