The necessity for organizations to transform digitally has increased enormously. According to a BCG survey, this urgency has almost doubled. The feeling of this urgency is noticeable and grows. Read more about a cloud-native mentality and how this contributes to a digital transformation.
Many organizations struggle with digital transformations, but the feeling of the necessity keeps growing. Outdated systems and ways of thinking about data architecture are often the culprits. These challenges can be overcome by a cloud-native way of thinking (this is not the same as going to the cloud) and by naturally letting data flow through the organization, making the data available for all employees.
Cloud-Native way of working and data democracy as solutions
These two solutions are both too extensive to cover in one article. However, before we talk about what data democracy is, it’s important to understand the effects of a transition to cloud-native working.
First of all, data-driven working is not about data itself. Data is the fuel to increase the rate of learning, as well as the scale and speed of decision-making. If an organization starts working data-driven, the chances of becoming successful increase. McKinsey says that it is 23 times more likely that you recruit new customers with active use of data, you have six times more likelihood to keep these customers, and it is 19 times more likely that you are lucrative in this process. These numbers are also supported by a Forrester report which says that data-driven organizations grow by 30% per year.
The question is not ‘if’ you have to change, but ‘when.’
This all sounds very promising, but why do 80% of the organizations fail at this?
As is often the case, the most important factors are the strategy and the understanding of data concepts by the senior management (link). If one of the first questions is: “What is the ROI of data?”, then you know that they missed the point and urgency to change their organization. The question is not if you have to change, but when. Your organization cannot stay behind while:
Competition further develops
The expectations of customers frequently change
The demand constantly adapts
Technological innovations within your primary business processes force you to change, and right now. Centralization is always a chain reaction when an organization is focused on operational excellence.
Apply the right data-architecture
Many organizations have lots of data available, but access is limited. The crucial question is: do you apply the correct data architecture to let data flow freely throughout the organization, making it available for everyone? The current, most recurring architecture is based on the centralization of the data from core systems, managed and controlled by a central organization. In other words: not everyone has access to the right insights to learn or create predictions. From the operational excellence perspective, this is a good strategy since consistency and predictability are important. This is a ‘safe’ approach. However, this is not the best approach when looking at learning, speed, and scalability, since the central organization will become the bottleneck for innovation. The solution? That is right in the middle, between chaos and control.
Solving symptoms is only helpful in the short term. The side effect? You ignore the bigger problem, which causes a vicious cycle.
IT solutions are often built in a way they can function independently. This is a good strategy from the supplier’s perspective, but this also introduces another problem: the data is saved in silos. All the different IT solutions save data in their own way, which causes these big data silos.
Keeping core IT systems and data apart
A possible way to solve this problem is by centralization, namely by adding an even bigger silo (data warehouse). This may seem like the best solution, but this won’t help you achieve your goals in the long term. To make sure that data can flow freely through the organization, you have to keep core IT systems and data apart from each other. The trend that is a result of this is building a microservices architecture.
Microservices architecture enables you to build individual pieces with their functionality, that can independently be developed, implemented, and managed. Functionalities, like a log-in service, a service for making an account, or a payment service, can easily be reused and combined to subsequently create new solutions.
To improve the data flow, setting up new data patterns and data usage is crucial. In an ideal situation, the departments of the organization own the services that they know best, and the service includes all data of a producer-consumer relationship. Every data product is under the responsibility of the team that has the most knowledge about specifically this data. The data production will be distributed and decentralized, to prevent closed silos. This is called ‘data democratization’. However, the question remains: where do you need to start? In this blog, our colleague Hayo van Loon explains how you can realize this.
To make sure that the service can offer a so-called “publisher-subscriber relationship”, you have to give everyone that has access to that service also direct access to the data. Without the intervention of a central data team. This also makes it possible for the user to apply the data to every innovation objective.
This approach generates, according to BCG, two times more value, two times faster time-to-value, and is 50% less expensive. We’re sorry if you just invested eight million euros in a “data lake”. But believe us: there is a better way ;)
