Knowledge is power.
Organizations that institutionalize and share knowledge (thus creating processes that can be explained clearly and reliably to their employees) generate value, consistently.
When that knowledge isn't shared and documented, but is instead concentrated - "siloed" - within a group of specialists, it lives only as long as the tenure of those specific people. Transparency and collaboration suffer, and stale ideas circulate. Knowledge silos are stealthed bottlenecks to rapid iteration and adaptability, in even the most agile of organizations.
History is full of examples of knowledge silos resulting in unnecessary delays, wastage, and valuable insights being lost to the annals of time. Take, for instance, Fogbank - a lining material for nuclear warheads that cost the US millions of dollars (and a decade!) to reverse engineer, because they hadn't documented the original manufacturing process in the 1980s, and people on that team with specialized knowledge of the production had retired, died, or left the agency.
What can we do about silos, then? How can we improve cross-functional collaboration, and preserve domain-specific knowledge that is painstakingly learned over time? Let's find out with Swimm, an innovative knowledge management system that is language agnostic, supports imports from Notion and Confluence, and champions the principle of Continuous Documentation.
But first, let's address a burning question.
"Why break down silos if they get results?"
On the matter of silos, it's easy to focus solely on short-term profit and overlook the bigger picture. After all, some technologies may be lost knowledge today, but weren't they incredibly successful in their heyday? So why are knowledge silos a problem, as long as they produce results for the organization right now?
The problem with this "silos get results; don't fix what isn't broken" viewpoint is that there are dangerous, and sometimes outright false assumptions with that claim. Silos may get results, but they definitely limit your organization from iterating quickly, scaling, and expanding. Your organization becomes the opposite of Agile.
1. Your specialists themselves become the bottlenecks, and your Bus Factor plummets.
Wherever knowledge silos exist, over time, specialists or experts in a particular team become the only ones who can perform certain tasks, or make certain decisions. This creates a bottleneck where work can only be done when those individuals are available, blocking progress, slowing down rapid iteration and prototyping, and leading to unnecessary delays.
If anyone from this team was to be laid off, retired, or became otherwise unavailable (illness, vacation, etc.), there would be a significant knowledge gap that severely impacted your organization's ability to function effectively. This is known as the "bus factor": the number of people who you can afford to lose by - hypothetically - being hit by a bus, before your project was in jeopardy. Obviously, higher is safer, but with silos, that number is dangerously low.
2. Silos assume that regressions don't exist.
The silo mentality discourages documentation because, "Whatever I'm building now is going to get replaced by something strictly better anyway, so why bother creating documentation?" And so domain-specific knowledge lost due to turnover, retirement, or other factors isn't seen as that big of a deal.
In reality, technological regressions do exist; technology doesn't only move forwards. For example, Ancient Roman concrete used to build seawalls was actually far superior to modern construction, but was lost knowledge until very recently. In software engineering, this often starts with hacky, poor quality code thrown together at the last minute to meet deadlines, which remain in the codebase and ossify, even though they were always meant to be rewritten properly later.
3. The pooling of specialists tends towards tunnel vision.
The silo mentality assumes that a group of specialists working in isolation will always come up with radical, innovative new ideas and never get fixated on a particular way of doing things that has worked for them, historically.
When specialists work in silos, they often become entrenched in their own ways of doing things, develop tunnel vision, stagnate, and lose sight of the bigger picture. Unsurprisingly, this leads to a lack of collaboration, slow development, and missed opportunities for innovation or adaptation to changing trends - because why trust ideas if they originate from "outsiders"?
This shortsightedness and mistrust within the organization is a significant barrier to progress, despite the potential for fresh, better, more efficient solutions. It hurts morale, and stifles innovation.
4. The trap of Vertical Specialization.
Silos assume that specialized knowledge within a vertical is only valuable within that vertical, and there is no need for cross-functional knowledge sharing; that dev teams are able to understand and modify any part of the codebase at will, themselves, and collaboration isn't essential to understanding different parts of the system.
Of course, the fact of the matter is that any modern software project subscribes to continuous feedback and improvement, so cross-functional collaboration and knowledge sharing across different teams and departments are critical. A feature that may seem simple on the surface may have far-reaching implications across different verticals, or parts of the system.
Also, vertical specialization creates a lack of shared accountability and transparency within teams. Without horizontal collaboration, it's too easy for team members to resort to blame games and tribalism, ultimately leading to delays.
Besides, even if the original assumption were true, and a specific piece of information really was valuable only for a specific vertical, it still may not be transferred when different people are assigned to this vertical, or when people switch teams/projects, sending us back to square one.
5. Critical expertise lasts only as long as employee tenure.
In today's fast-paced and competitive business environment, employee turnover is an inevitable reality. Combine this with the fact that in siloed organizations, critical knowledge and expertise are tied to specific individuals - specialists who build domain-specific knowledge over time - rather than being documented and shared across the organization.
Together, this is a disastrous one-two punch that means that when employees leave, they take with them the specialized knowledge and expertise they have acquired during their tenure. From the organization's viewpoint, critical expertise lasts only as long as employee tenure, and it's vital to take steps to retain that knowledge before it walks out the door.
So, yes, silos slow down innovation and adversely impact the developer experience across the organization. Breaking them down is crucial for fostering collaboration, knowledge sharing, and - ultimately, what we're all here for - faster and more efficient software development.
How to break down silos most effectively?
Simply relying on conventional strategies like regular team rotations, standups, and weekly meetings may not be enough to fully dismantle silos.
While team rotations can provide employees with the "bigger picture" and promote cross-functional collaboration, they can just as easily lead to disruptions in project continuity and prevent domain-specific knowledge from accumulating in the first place.
Similarly, weekly meetings where everyone shares everything can be a waste of time. As counterintuitive as it sounds, not all knowledge is useful, or even needs to be shared in the first place. Agenda-driven meetings are often time-consuming, and critically, may not even lead to meaningful discovery of knowledge at all.
So, what's the best way to break down silos?
The answer is something that stops them from ever being relevant. A company-wide culture of knowledge sharing and collaboration. So, build an inclusive environment that makes it easy for employees to both share, and discover knowledge.
You'd need to :
- Create strategies to promote this new culture of knowledge sharing,
- Use a framework, or tool to implement it (Swimm),
- Regularly review and reiterate to make it better.
What could such an approach look like? Let's find out, using Swimm as the framework mentioned above.
Building a culture of knowledge sharing with Swimm
What makes silos relevant? Documenting knowledge is a chore; discovering knowledge is a pain. Personnel outside of product teams lack context, and the risks conventionally associated with sole responsibility make employees hostile to collaboration.
Swimm can help negate each of these points, systematically, removing power from silos.
1. Simplify documentation as a process.
The most common blocker to knowledge sharing within organizations is that documentation can be difficult and time-consuming to create and - more importantly - keep up to date.
So the biggest win possible would be to invest in tooling that simplifies documentation as a process, removing its barriers to entry and making it easier to engage with.
With Swimm, that's exactly what you get, and it's incredibly easy to get started. Swimm's editor (that can do rich text as well as enhanced Markdown) makes it a cakewalk for anyone to create code-coupled documentation that directly references live code within the repo (code snippets, functions, variables, or any token within code at all), explain it using images, videos, and Mermaid diagrams, and interlink it to other internal documentation, wherever it makes sense to do so.
Swimm also provides standardized templates for creating and maintaining technical reference and API docs, data flows, high-level overviews of the system, design docs, research plans and proposals. This saves time by eliminating the need to start from scratch every time something new needs to be documented, no matter in which vertical or department.
All docs created with Swimm are plain old Markdown files, saved in a .swm folder in the same repo as the code, and viewable/editable outside Swimm. There is no vendor lock.
But writing docs is only part of the puzzle. As any developer would tell you, the only thing worse than no docs is outdated docs - and this is another area where Swimm comes in clutch. Its patented Auto-Sync feature automatically keeps the live code snippets referenced within docs up to date, and alerts you whenever code changes are enough to cause a significant drift with the documentation. For things Swimm identifies as breaking changes, it does not Auto-Sync them but rather prompts for a human's intervention. In the latter case, Swimm's GitHub App will fail verification checks, and block commits and PRs, prompting you to manually resolve docs being out of date before proceeding.
2. Improve Accessibility & Discovery
The second most common blocker is that documented knowledge is difficult to discover. It's frequently scattered across different platforms - for example, design docs in Confluence, instructions for setting up dev environments in a company Google Doc, some implementation bits in code comments, and tips and tricks for daily development within emails and Slack communiques - and even if centralized, they are not reliably searchable.
Swimm addresses this challenge by making documentation visible, accessible, searchable, categorizable via tags, and available where it is needed the most - in the Integrated Development Environment (IDE) itself, right beside the code.
Swimm - Visual Studio Marketplace
Swimm - IntelliJ IDEs Plugin | Marketplace
These IDE plugins allow members of any team, organization-wide, to easily find relevant knowledge while working on the codebase, without having to constantly Alt+Tab between different platforms or Ctrl+F through multiple documents.
3. Document new problems and solutions immediately, and include context.
Every time someone works on something that nobody else has context of, have them walk through the changes they made, explaining why they chose that particular approach. It is crucial to document new problems and their solutions immediately, when the context is fresh in the mind of the decision makers, while also providing context for both the problem and the solution. This provides transparency to future team members, letting them have insights into the thought process and decision-making that went into solving a problem.
With Swimm, it's a cakewalk to create documentation where you can just reference live code, and create links between different sections of it - even across different microservices and multiple repos - explaining how all the pieces fit and work together using markdown or rich text, images, videos, flowcharts, sequence diagrams, etc. Even better, with Swimm's IDE plugins you won't even have to Alt + Tab to create documentation, you can do so right in the IDE, even while you're coding - so that you can really document every important decision as you implement it.
Walking through live code to explain flows and patterns in a project, across multiple files and repos, pointing out often unintuitive interactions in a single, logical flow - all of this mirrors an experienced developer sitting with the team and being a mentor, sharing knowledge, and creating an environment where the pursuit of knowledge doesn't have to involve being thrown into the deep end of the pool and being expected to sink or swim.
This is what Swimm calls walkthrough documentation, bridging the gap between code comments and high-level design docs. By leaving breadcrumbs for future team members, Swimm helps to ensure that the knowledge gained through iteration, experimentation, and problem-solving is always preserved, and always easily accessible to others.
4. Make Offboarding More Effective
"Offboarding" is just the process of managing an employee's departure, and for our needs specifically, we're concerned with the knowledge transfer stage of the process. Good offboarding flows make sure the departing employee leaves with a positive experience while also capturing their knowledge adequately, ensuring the organization can function in their absence.
If knowledge silos exist in your organization, properly offboarding your specialists becomes an essential - but often stressful - item on your checklist. With Swimm, here's what you can do to ease the process:
- Instead of asking the departing employee to document everything before they leave, you can create specific Doc Requests for sections of the codebase the departing specialist had expertise in, and assign it to them via the Swimm web app's UI. You can track offboarding progress this way, and these requests will be marked as completed once its associated doc is merged to the default branch by the people it's assigned to.
- You can have someone sit down with the departing specialist, and together, create Swimm Playlists that enshrine their knowledge.
Playlists are a special kind of documentation from Swimm that are essentially similar to Spotify or YouTube playlists, but for knowledge sharing. These step through a sequence of documents, links, videos, Markdown files, or diagrams to explain something - whether that's instructions for setting up the local dev environment, explaining data flow for the project, or just a historical record of engineering decisions made, with their context.
Taking the effort to create these Playlists pays off by 10x'ing the onboarding of new hires, later.
Offboarding is difficult because you're never quite sure when you've got everything you need, and to make matters worse, you'll often have a limited window of opportunity to complete the process. Also, there's the human factor. Not everyone who leaves does so on good terms, and even those who do may not have the motivation to participate. You can't guarantee participation, but it is crucial to make that effort, and Swimm helps.
5. Encourage Shared Ownership
To build a culture of knowledge sharing where anyone can speak up and contribute, you also need a culture where blame games do not exist.
This means shifting responsibility from the person who wrote a piece of code/suggested a new idea, to the team instead. Especially for meaningful features, you need everyone to buy in and be involved in both planning and reviewing, with no risks or blame present. Encouraging a culture of shared ownership with pair programming and inclusive code reviews can get you there.
Pair Programming
Pair programming involves two developers working together on the same code - one typing, the other providing feedback in real-time. Not only does this help establish shared ownership, it is also a win-win scenario:
- It sets up a natural transfer of tribal knowledge/skills from a specialist to another team member, organically breaking down silos.
- As the two developers bounce ideas and suggestions off each other to debug a problem, it creates a shared understanding of the codebase, pre-empting silo behavior entirely.
As a result, tribal knowledge is actually passed on - not hoarded - and individuals are less likely to work in silos.
Inclusive Code Reviews
Unlike traditional code reviews - which are an often intimidating, painful process where one person's work is essentially judged - this would be a team effort, with multiple people involved in both planning and reviewing. For major features, a few team members can brainstorm ideas, share what they know, discuss objectives, and develop improved workflows going forward.
Swimm makes code reviews fast and painless. Any time a PR is open, a document can be scaffolded from it with one click. Within it, the dev can walk reviewers through the important changes/additions in the PR using code-coupled documentation, diagrams, and images.
This way, developers can describe their code with far more elegance than code comments or pages upon pages of discussion within PRs ever could, and also make sure that context and reasoning behind changes/additions are documented for posterity. Later, Swimm will update the document as the code evolves, so it can serve as a relevant resource, rather than a historical artifact.
Conclusion
Think outside the box when it comes to knowledge silos. Understand the root cause of silos developing, rather than taking a bulldozer to them by default.
The traditional strategies for demolishing knowledge silos - regular team rotations, mandates to document each and every little thing, and long, agenda-driven meetings - are that proverbial bulldozer. Rather than making employees hopeful, and buying into the process, it makes them wary and hostile, instead.
Consider a holistic approach. Create a company-wide culture of knowledge sharing and documentation instead. This post has hopefully given you an idea of the playbook you can use to promote such a culture in your organization, and Swimm, the framework you can use to implement it. All that remains on your part is vigilant review and reiteration on the process to maintain it.
If you can do that, you won't need bulldozers. Silos will break down organically.