Organizational silos is what you need

I have been working on several big IT programs over the last years. We are talking here about big, ambitious programs typically run in large corporations

Seeing these program struggle to keep with their initial ambition, and even deliver a fraction of the original value, is common. These programs tend to be very big and very difficult to steer effectively

I would like to focus on one area often heard of within these programs:

We don’t succeed in making things work because people do not talk to each other. They are working on silos.

Actually, I would like to challenge this belief and go as far as saying:

Organizational silos are what you need to make big programs work effectively

Silos as the root cause of problems ?

One could argue that the primary reason these big programs tend to fail is their size. I would agree with that. Having smaller, manageable programs is a nice goal. Yet sometimes, big programs are what you end up with and you need to organize them.

Most of the time, people complain about silos because a silo by itself is not able to provide value to the company.

This is often the case when you organize your teams by technical layers or by competencies/skills

We call these silos Horizontal silos

A classic example of these bad silos is when you separate you frontend developers from your backend developers. Each team can produce a piece of your applications, but none produces something that can be used by the end user

Similarly, when you separate your testing or integration teams from your development teams, you require the development team to pile some development work and then ship it to the testing team to validate it.

When problems occur (integration is slow, many bugs remains in production…), you blame it on the « silo mindset » in which teams do not talk to each other.

Outsourcing strategies adds to the problem

An extreme version of this is when one of your « bad silos » is outsourced. While you already have difficulties in making people integrate their work together, you now have the added problem of synchronizing the output of an external company with everyone else’s.

The real problem here is that each team needs the output of the other teams to provide value within the program.

Besides, their work usually need tight integration with the other teams (specific API development, behavior and requirement misunderstandings…).

These teams really are dependent on each other. They cannot produce value on their own and need constant interaction.

Indeed, these teams operate in silos and this damages them

Classic strategy to break silos

A strategy to break these silos and force people to talk to each other is to locate everyone in the same place.

We are talking here about big programs, often involving hundreds of developers.

This means creating big setups where all the developers and their managers can be co-located so they can « talk to each other » and accelerate integration and dependency resolutions.

While this is a good strategy for small teams, in big programs with multiple subsystems, it tends to be counter productive for several reasons:

• It requires co-locating a great number of people in a single location, which is difficult to achieve when people usually come from different countries and even from different companies
• This setup is not sustainable over time and is usually the solution adopted to solve a crisis. It is not a stable organization
• Putting many people in the same space might improve point-to-point conversations, yet tends to produce an overall communication mess. Everybody being able to talk to everybody, there is no longer any interface or point of contact
• In computer science architecture, this is known as « global variables ». You loose the control of your program by having communication patterns all over the place.
• This creates a gigantic monolithic system

Conway law

A famous saying, known as Conway’s law, states the following:

Organizations are constrained to product designs which are copies of structures of the organizations

What this sentence means in our context is that by creating gigantic gathering of people in a common place, your architecture will eventually become a monolith:

• All components talk to each other
• No interfaces defined between components
• Fragile system: if one subsystem fails, the whole system is at risk
• Heavy system: tends to be very difficult to deploy and operate

Vertical silos is what you need

As we can see, creating big gatherings of people together does not always convey good result, and putting people in « silos » is not working either.

There is a better way.

And the better way is actually to go find « good » silos !

What you need is first to group people according to the amount of common value they provide. This is usually at the feature level.

The feature is usually what brings value to the final user, and can be at many levels:

• Use case
• An application type
• A complete business process…

You will also want include in these groups any functional level that is related to the feature: architects, product managers, tester, devs

By grouping people working on the same feature, you create a « silo » able to create value for the program.

It is a silo because it is a group of people who do not need to talk to outside people every day. They are autonomous in moving the program forward. You do not engage with the team outside of what the team produces.

This is a « good » silo, because what comes out of this team is valuable content that gives nice value increment to the program. It does not need anything else to create its own value. It is also called a vertical silo, because you organize it towards vertical features, rather than horizontal technical layers

They are often called Cross Functional Teams. These teams have all the technical and functional expertise needed to move forward. They do not rely on external integration teams and do not require anything but their team members to move forward.

The complete program value is then usually created by the sum of these independent cross functional teams’ outputs

Cross functional teams are silos, but they provide value by themselves, qualifying them as « good silos ». What you need, is a number of such silos

Good silos are still silos

Being good silos is nice, but in organizations where agile is not yet commonplace, they are still seen as silos. And silos have bad reputation.

People tend to see agile teams as teams working on their own, not sharing their status, leaving central management in the blur, because no one clearly knows what they are doing

It takes the management’s and coaches’ energy to explain the value of such teams. Putting the emphasis on team demos is crucial to show the organization that these teams are indeed producing value by themselves, hence removing the worries of centralized management.

Creating and managing good silos

Creating teams that have the right technical expertise and well-defined scope is hard.

This is usually a joint effort between the product managers and architects within the program to create a system where teams are autonomous and yet constantly producing value.

Finding the sweet spot for teams to produce value independently is a trial and error game, mixed with a clear understanding of where the program’s value is located.

Having short iterations with demos showing progress is very important to help steer the scope of each team over time.

What about outsourced resources?

Outsourced teams can be seen as a challenge but actually follow the same rules.

Usually, there are two possibilities while outsourcing development:

• The outsourced company can produce value on its own (for example if the outsourced company develops a complete app on their own). In this case, you can be result oriented and ask for the complete product to the outsourced company. This means the outsourced company is a silo by itself, a « cross functional team »
• The outsourced company will not produce value because it lacks many other pieces (like legacy backend development for example). In this case, you are effort oriented and will ask the outsourced company to participate in one of several cross-functional teams that is part of your program

Conclusion

Silos are not a bad organization. In fact, silos are key if you want to scale up your systems or developments.

Having the classic « let’s put everybody together » mindset can work with small programs or small organizations, but this is definitely a bad idea once you start to scale up to more than a few dozen developers.

Mature organizations have already understood that, that is why the agile mindset, with small, cross functional teams has had tremendous success this last decade.

Of course, it does not mean such team will never interact with anyone. Yet it does it based on what it produces, between sprints, at well defined times.

What about your organizations? Does it have horizontal silos ? Or is it constantly monitoring teams’ outputs to find the right silos that fit their needs?