Microservices: Don’t build problems
Portuguese version: https://medium.com/m4u-tech/microservices-n%C3%A3o-crie-problemas-1982c31c750
Microservices has already become a reality in the development market. In times when everything is in the cloud and platforms like AWS, GCP and Azure (not to mention that I forgot about Microsoft) have been gaining more and more market with their solutions. Consequently, the use of microservices is gaining more form and more maturity. Companies like Netflix, Amazon and Nubank have shown themselves ahead of their time and have positioned themselves in the market as influencers in this matter.
However, Microservices is not the solution to all problems. We are coming from years and years of monolithic projects (although they were not called that at the time) and we have brought all the habits and patterns of architecture in our DNA. We also often need to deal with the coexistence of more than one architectural model and this coexistence requires great care, especially with regard to the non-use of habits and some legacy standards when we are going to work with microservices. For this reason, I decided to raise some necessary precautions here to prevent the use of microservices from becoming a problem (fond memories of my eternal Tech Leader Mauricio Pelissari).
Business Responsibility:
There is no way to talk about microservices without starting the subject by talking about responsibilities, after all, what most facilitates good use and development with microservices is exactly the responsibility of each application.
This topic is what defines the cutoff point that will tell you how “micro” a microservice should be. Incredibly, this is the most difficult point to establish: “How far does the responsibility of one microservice go to start that of another”.
Contrary to what many people think, microservice is not separating a part of your system, however much in the end it appears to be doing so. The misuse of these concepts can transform your application into a micro-monolith or to be so micro as to make the maintenance of architecture impossible. See the example below that I met from a lecture by Eduardo Souza:
Who am I to criticize the architecture of both companies above ?! I am bringing this image in order to show how a structure of microservices can become complex in the long run, which makes it extremely important to become aware of the granularity of its services, facilitating communication and avoiding clutter and chaos.
Practical example:
We assume that we will develop a payment gateway, where we will have communication with several companies that will make the actual collection with the card operators. However, I need to control which company I will use based on some registered rules. In this case, we would have the following scenario:
Imagine that we have a microservice for each company, one for rules management and one for customer registration. In this game alone we would have to build seven microservices for a single solution. Now raise the cost of supporting and improving this solution while maintaining N nodes for each microservice. It is a fact that depending on the scenario this is necessary, but should I use microservices even for all scenarios? Do I really need to impose this level of complexity on my solution?
Recently, Sam Newman brought us in a presentation of QCon 2020 (Source InfoQ) some questions in this regard. In this case, he reported on the aspect of decomposing the monolith but also brought us questions about the real need to use monolith for everything, and as pointed out by Edilson Azevedo on his Linkedin on this subject “Newman advised people to focus on the result and not on technology”.
Use of Connectors:
A very insightful solution to this type of problem is to use connectors that are nothing more than libraries. Yes, it seems like a very old idea but it makes perfect sense when we use something that has total relationship with a third party and/or partner. With Maven for example, we can create a communication library for each company. This library is concerned with just communicating with the third party, so we have several or just one connector (depending on the scenario) that can be imported by the configuration pom file itself. In this way, we will build only one microservice that will receive the requests and decide which connector to use according to the rules that this microservice is responsible for.
Here is a link to the official website with an explanation of multi modules within an application: https://maven.apache.org/guides/mini/guide-multiple-modules.html
Exclusive Database:
One of the most controversial definitions about microservices is the exclusive use of a database. Some turn up their noses at this type of solution because they have the very strong concept of communication via API as a problem for future consultations. Well, this is one of the main reasons that you should respect an exclusive database via microservice. When we plug reports and other data sources into the bank, we make room for overloading that bank depending on the scenario. It also does not resolve to put different databases in the same database as this can also cause a lot of damage and cause other types of problems. The ideal is to separate and let the microservice work independently.
But then, how do I do with the data for my reports?
Currently the main platforms (GCP, AWS, etc.) already have a solution for migrating database data. I recently had the pleasure of working with AWS DMS and was very happy with the result. In this case, we replicate all data almost in real time, from one bank to another. In addition, there are other solutions as well, such as Debezium, cited by Bruno Santos in his article here on the medium. That way, you can create a bank just for that purpose, and if possible, already transform the data so that the destination bank already has the formatted and distributed data, avoiding an excess of JOINs in gigantic tables and making the data query more performance.
Ok, and how do I query a lot of data via API, it will not impact my performance?
The fact is, if you need to search a very wide range of data for a given domain, you certainly have problems with the first topic of this article. If you need to handle a lot of information from the source (real time) it is a clear sign that business responsibilities are not well distributed and possibly you will need to migrate part of the code into the responsible microservice. But, in case a large range of queries is needed to generate processes, I also suggest the previous solution where you will have the data almost in real time in a bank for consultations. Take this as a rule avoiding impacts on the main flow of responsibility of the microservice.
API versioning:
Versioning API is a very important habit, however, it must be used consciously and clearly. Questioning what drives the use of versioning an API is the key to preventing your microservice from becoming a micro-monolith with different versions to do the same thing.
The API versioning must be used, with it you can keep versions running in parallel and avoid impacts to the consumer of this API. But it is always important to remember a few tips:
Always agree on a term for using the replaced API.
Every change must be documented and the use of the version can help consumers to have a migration time, but this migration must have a pre-established period, even if it is long. Keeping two versions running can cost more in evolutionary terms as you will always need to take into account for new developments contemplating all possible scenarios.
If the contract is totally different, create a new microservice
Yes, it may seem strange, but if you are going to change the communication pattern where the transformations will be incompatible with the one you have or that will generate a lot of unnecessary code, it is better to create a new service. For example, if you are migrating from a communication where you received XML via SOAP and now you are going to do a new communication via REST, do the new development in a new microservice for this because possibly the transformations and data manipulation are totally different and use totally different libraries. For this specific case, two microservices can access the same database but taking into account that the question of the old microservice’s validity contract is applied in the short term.
In asynchronous communications use Queues instead of API calls:
The use of queues is very interesting when it comes to asynchronous communications. This is because you guarantee that the message generated in the queue will be processed and that there will be a control of the flow rate that this queue will process. This is beneficial in several aspects, including controlling your service’s memory consumption when it comes to processing large amounts of information.
Study the Kubernetes Controller types:
Yes, it seems basic that understanding how something works before using it is healthy, but that’s what I mean. Kubernetes has several types of Controllers and each has its benefit. I say this because I have already seen entire ecosystems in Clusters where everything was Deployment and I have already seen some speakers who mention this as insignificant, but know that with the maturity of their applications this can become a problem. A classic example is the use of Controller Job for extremely punctual executions, after all if you need to run something only at specific times there is no need to leave a machine on with a high degree of availability knowing that it will not be used.
More about Controllers: https://kubernetes.io/docs/concepts/workloads/controllers/
Conclusion:
Of course, on all the points mentioned above, I based on facts that I have already experienced and that are part of my learning and I know that each one can be the subject of another article if we go into the details, but the fact is that, some of them only were identified with the real pain of the problem and I thought it was important to raise this issue. Microservices is something fantastic and that solves a lot, but it is necessary to be careful that the solution does not cause new problems, or if they do, that are not bigger than what your ecosystem already has.
