Migrating to an event-driven architecture (EDA) can be a difficult task. But it can be a good move for organizations. It can reduce coupling, improve scalability, and increase the resilience of a backend.
I participated in two migrations to EDA. In the first one, the architecture had several coupling issues. We had a number of microservices with a lot of REST calls between them.
In the second one, the product was under construction. The team was growing fast. We thought that it was a good move to introduce EDA in the project.
Both migrations went well. We have introduced the EDA and taken advantage of its benefits.
If you plan to introduce EDA in your architecture, you will probably focus on various technical aspects :
- Events broker/messaging platform.
- Integration libraries.
- Infrastructure.
But many teams overlook the wider considerations that can make or break the migration’s success.
In this blog post, I’ll summarize some of the lessons I’ve learned during these two migrations.
Event semantic
One critical aspect of EDA is event semantics. Before producing and consuming events, it’s essential to identify the type of events needed. What type of events do you need in your system? Which events make sense in the system’s business domain? Can you identify some groups of events? Can you categorize them?
When you think about your events semantics, keep in mind :
- Events should be atomic.
- Events should be autonomous.
- Producers must not know the consumers (crucial to reduce coupling).
Naming convention
With a set of events, you can begin your EDA journey. I encourage you to define a naming convention: Events name, Events type, Queues/topics names. It is essential if you want to keep control over your EDA architecture. Moreover, it will ease the discussions between the teams.
I like this topic naming convention: https://dev.to/devshawn/apache-kafka-topic-naming-conventions-3do6
Produce as many events as you can
Producing enough events is crucial to taking full advantage of EDA’s capabilities.
In the first months, it will be painful. Even more, if you migrate a legacy project. Every time you want to use events in your business transactions, you will find that you lack some events. It will take time to implement them. It will be hard to think of possible errors. All this will slow down the developments in the beginning.
This is the classic way. A new event is a step forward, even if it’s not yet in use.
In EDA, the lack of events in the system is an anti-pattern.
Team evangelism
Team evangelism is also vital to the success of EDA. As I said before, it will slow down the developments in the beginning. Stakeholders may perceive the development time as too long. Bugs can also occur. It’s important to make stakeholders aware of the benefits of EDA and to explain the impacts on future development cycles.
In the end, if the implementation of EDA is a success and the teams have fully assimilated the principle, product development will be more efficient.
Share code
It might be a good idea to share code. For example the broker’s integration code. It allows the teams to go faster while keeping technical control of the project.
I did not share code in my first migration. In the second one, we had a shared library system in place. So we took advantage of that by creating a dedicated library for event production/consumption. And also for testing. It eased the teams’ adoption. Integration bug-solving is simpler and versioning is less complex.
(Chapter 8 - Reuse Patterns of the book Software Architecture: The Hard Parts is particularly interesting).
Documentation
Documentation is an essential aspect of EDA. I recommend setting up documentation to avoid duplicates and sum up events’ semantics. Documenting a posteriori is always painful and less efficient. Documenting at the beginning of the migration will also force developers to think about the event’s motivation. It can be a good point to iterate and adjust your practices.
In EDA, you will probably need to execute distributed transactions in your system (SAGA Pattern). I encourage you to document your distributed transactions as well, especially at the beginning.
(Chapters 11 and 12 of Software Architecture: The Hard Parts are particularly interesting on this subject).
Conclusion
Migrating to an event-driven architecture (EDA) is a difficult task that takes time. But it can be a good move for organizations. It’s crucial to think carefully before implementing this pattern. Every organization needs to weigh up the pros and cons. From what I have experienced, it is a game changer. If it is well implemented you can drastically reduce coupling, and improve the reusability and extensibility of your system.
References
Software architecture: The Hard Parts. - Mark Richards, Neal Ford, Pramod J. Sadalage, Zhamak Dehghani.