Service-oriented architecture. Digital copy of the business

SOA, or service-oriented architecture. A fundamental element of building complex computer systems, without a proper understanding of which it is impossible to build any systems except monolithic ones.

It doesn't matter what architecture the end systems were designed for. If a company does not understand what a service is, all systems become complex, and the settings for any solution become confusing. The lines of responsibility of programs under the supervision of departments, as well as the boundaries of the departments and departments within the company themselves, are being blurred.

As a result, numerous conflicts arise, and the introduction of any innovation is like moving through a minefield: not everyone will arrive slowly and dangerously.

Are microservices better?

When many IT people hear about SOA, they can meaningfully say something like “I know what it is, but we have a microservice architecture”.

With few exceptions, this phrase is a sure sign of a misunderstanding of SOA. And an IT guy who doesn't understand SOA also confuses business users.

As a result, managers perceive service-oriented architecture as some kind of incomprehensible principle. It seems useful, but “microservices are better”.

In the process of advising KT.team clients, we have never found completely error-free services. But mistakes in building services can cost a company half of its IT budget!

This is such an important topic that as part of our course for project managers and strategic consultants, we singled it out separately in the organizational structure of the enterprise. We teach you not to separate IT services with organizational schemes and business process coordination.

Company growth problems

Imagine your organization just came into being. There are no posts and departments yet, there are no regulations and agreements on the division of powers, and indeed you are all sitting in a small office. In engineering terms, you can be called a monolith.

So far, there are few functions and your processes are as simple as possible, due to the wise head of a manager, adding a new duty or task to the company is organizationally “inexpensive”. You cheerfully overcome problem after problem — and “the car goes”. However, the company is in manual mode.

Your company is growing and processes are growing with it. You no longer live in the same closet, you occupy the entire building. However, you don't yet have an organizational structure, and without it, there can be no service architecture.

Processes have more and more nuances, and the likelihood of each innovation being implemented is decreasing. You cannot fire Masha for not complying with paragraph 7 of your contract review instructions, because she has a dozen important functions.

Masha tells you: “It's my fault, I won't do it again.” Next time, she may comply with paragraph No. 7, but she will probably violate paragraph No. 9 and paragraph No. 3.

The powers of departments and individual employees are still mixed.

You don't need at least as many people as features yet. You divided the company as it seemed logical — for example, you divided it into floors where departments sit.

This example is deliberately far-fetched. Of course, you divided the departments into conditional accounting and purchasing, for example. But can you now demand everything from the accounting department? Are there any clear metrics there? Are there any cases when the accounting department will tell you: “We know about this problem, but the IT department is to blame for not making the right feature”? Are there any cases when the material purchase department will tell you that it purchased exactly according to instructions, but the data in a general program is of poor quality: it contains duplicates or inaccuracies?

Such situations signal that departments don't have the power to make decisions to change their work, so you don't have the power to demand results from departments.

Over time, without a correct delineation of areas and powers, without delegating full responsibility along with limiting the managed context, the organization becomes unmanageable. The manager is at work from morning to night; he doesn't leave his laptop and phone either on vacation or on weekends just to “keep everything working fine”. If it is possible to measure accounting and purchasing, someone else is always responsible for this result.

From the beautiful picture, you have the organizational chart of the enterprise. In fact, you still have a monolith.

A monolith is a state of an organization or program in which each new feature extension costs more than the previous one. Each subsequent change requires more service actions. It comes to the point that your number of people is growing, but there is no profit.

The monolithic IT architecture is exactly the same. It may look like different programs and services, but in fact everything is so closely linked that adding new functionality always causes more overhead costs than before.

The company needs superheroes

Of course, your company with poorly divided departments (this is called strong connectivity in IT) can somehow live. The problem is that departments must have superhumans who can solve any problem.

And these superheroes will work for you until they burn out.

In IT, the same thing happens in a highly connected environment. Each team understands logic to some extent. But the further you go, the harder it is to understand what exactly needs to be done. You'll need an increasingly cool techie who will solve any problems — until he burns out. Something needs to be done, right? The reactions of many managers and engineers in this situation are very similar.

Executives read Harvard Business Review, listen to fashion conferences and Gref, and learn about Agile, turquoise, or some super-duper solution that will systematize your business and do all the work for them. “Oh my god, there's some magic pill that will solve everything!”

Developers start to read trendy articles frantically and learn about microservice architecture or some other super-duper system, language, or approach. Inspired, they come to the manager and talk about big plans for change.

At this point, the biggest mistake is to believe in magical thinking. The fact that with the help of one technology, it is possible to solve organizational problems in one fell swoop. That you will improve the quality of your data by implementing a new system. That your development will accelerate by switching to a new approach...

A system solution that will improve the monolithic organizational structure is to separate some part of the functionality into a separate block. But you can't single out on a whim, both in business and in IT. Moreover, IT should be divided exactly along the same boundaries as business.

Service and valuable end product

And now we've come to the point why we need to understand the service.

Service is not some magic spell that seems to be better than a monolith, but worse than a microservice. A service architecture is a map of your systems' functions divided into business blocks. Service is defined in the paradigms of a valuable end product or in the definitions of process output in process management. As a result, it should be very clear and transparent what exactly another company (or another department within your company) is “buying” from this service.

For example, let's look at Hubbard's organizational chart. According to this scheme, the company has a communications department, which, in turn, has a recruiting and induction department. Let's call this the HR context.

This department produces its valuable end product by “hired, hired and well-produced employees”.

‍

‍

This department will be responsible for its final product if and only if it has full authority to execute the CCP. The HR department must work in its own isolated IT system just to independently manage the necessary changes.

It is important that the HR department only provides your organization with what is its central office. In the real world, they're employees, and in the IT world, they're a digital copy of an employee. At the same time, the HR department should issue a digital copy of employees in a format for which it could be responsible, but its work would not depend on what solutions other departments use.

This is called weak connectivity.

In a non-digital form, such communication is provided as follows: the HR department provides the employee's file to a common place where other departments can come and view these dossiers — but not change them on their own.

And digital communication between different departments should always be ensured with the help of special solutions called tires. They allow each department to establish its own rules for providing the information that other departments need.

If the HR department changes its 1C program to a cloud service tomorrow, the only thing it should do is change the bus's rule on how to access an employee's profile.

This is part of the HR department's powers along with recruiting and hiring employees. After all, a digital copy of an employee is the same employee, only in digital form.

The dangers of magical thinking

Now imagine that an incredibly cool IT guy comes to you and says, “I'll implement a super architecture that will make all departments better.”

Will each individual department have the authority to use the button it needs? If so, how much time will it take to implement each implementation? If the two changes compete for resources, how will the conflict be resolved?

You can build IT services without taking into account targeted business processes and services, focusing only on new technologies. But this will result in the digital copy of your enterprise and your business becoming more and more separated.

The HR department should have the authority to say, “I don't want your fancy stuff, I want a different solution.” The responsibility of the IT department is not to micromanage how other departments work, but to provide a common infrastructure so that each department can provide a digital copy of its valuable end product to a common space.

This in no way means that the board of directors is no longer responsible for developing product manufacturing technology or is no longer involved in determining rules and procedures. This means that such changes should be implemented and implemented in the digital world in exactly the same way as “in real life”.

When your IT department brings you an innovation, you should not consider it as a magical solution to problems, but shift it to processes and make sure that the autonomy and flexibility of departments are not lost.

Should the architecture be the same throughout the enterprise?

There are no microservice architectures across the enterprise, just as there is no one-size department.

Due to the specifics of your business, your HR department can employ three highly specialized people, and they will need one cloud-based program to work. At the same time, accounting may consist of hundreds of employees, each of whom will specialize very narrowly. Accounting employees will be united only by common terms and entities (in microservice architecture this is called “connected context”) and they will work in microservices.

And some PR department will work in monolithic programs. And this is normal, because the digital copy of departments cannot differ from reality.

This principle was described 60 years ago by Mel Convey, whose publication the Harvard Business Review first rejected because it did not provide convincing evidence for its findings. After 40 years, HBR has finally released all of Convey's findings. The era of digitalization has clearly shown what is happening to companies that ignore these laws.

Convey Act

Organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations

Any organization that develops a system (in a broad sense) is forced to create projects whose structures are copies of the organization's relationship structure.

This is also described by Eric Evans in his book Domain-Oriented Programming. He clarifies that not only the digital “version” of departments, but even digital copies of business processes and entity names cannot differ from what they are in real life. The book is written for developers, and if you don't have a technical background, you'll be bored reading the second chapter. But the book's beginnings and conclusions closely link the realities of IT and management.

We recommend this literature following Dave Farley, Martin Fowler, and Thomas Earle. Their contribution to the development of computer engineering makes them listen to their recommendations.

The principle of building an IT architecture

Before deciding on an IT architecture, make sure that your technical team is interested in creating a digital copy of your company, rather than introducing trendy technologies. This situation is common and understandable: IT specialists are interested in trying something new and fashionable that can be “put into a portfolio”.

To avoid mistakes, first create an organizational structure at the enterprise with departments and subdivisions. Then draw a section map using approval-level business processes. Depending on the complexity of the processes, the exit of the entire department or each section will become a service. And whether it will be micro or ordinary is not determined by IT, but by how your business works.

It is clear that while your business is small, several services will be run by the same people, and in the case of programs, the same programs. As the organization grows and, accordingly, the load on these sections increases, they will be fragmented. In the end, businesses will have a lot of very simple functions that are responsible for very narrow tasks. If you initially plan your departments in the right way, the growth of people, business processes and software functionality will grow simultaneously.

The beauty of services is that when properly defined and properly separated, they do not separate technology from business, but subordinate technology to business.

But in many companies, we see that subordination is structured in a different order: the IT department, without controlling the strategy and technology of work and product production, determines the technology for creating digital copies. IT does not live by business rules, but other departments live by IT rules.

When services are not properly divided into services, conflicts arise when one task seems to have to be solved in one place, but partially solved in another. In the same way, an incorrect division of departments leads to conflicts between departments and opaque decision-making.

There are a lot of technical examples. For example, a corporate website, being only an interface to the procurement department program, generates an invoice on its own instead of requesting it.

Or a program that is not under the control of that department miraculously becomes responsible for the quality of the procurement department's data.

This topic is not easy to understand. When training managers, we analyze it and ask them to draw up service maps, teach them to model in the context of the organization's business processes and correctly find conflicts between services.