Handling Transactions in Microservices

Shared Database pattern (Anti pattern)

It is called Anti Pattern because we are losing the benefit of modularity. The services are tightly coupled to the database. If the database schema changes, all the services need to be updated. Also, the services are not scalable. If one service is getting a lot of traffic, it can slow down the other services.

• Let's say we have two services: payment service and order service. Both of these services are using the same database. If the payment service is getting a lot of traffic, it can slow down the order service.

• Deadlocks in the database can happen. If two services are trying to update the same row, the database can lock the row. This can cause the services to hang.

What should we do instead?

Database per service pattern

Each service has its own database. This way the services are independent of each other. If the schema of one service changes, the other services are not affected. Also, the services are scalable. If one service is getting a lot of traffic, it does not affect the other services.

• This pattern provides isolation between services. If one service is down, the other services are not affected.
• Modularity is also increased. If we want to change the schema of one service, we only need to update that service.

This is better but how do we deal with transactions that span multiple services?

Let's say we have another service called shipping service. When an order is placed, the order service needs to update the order table and the payment service needs to update the payment table. If the payment service fails, the order service should rollback the transaction.

How do we ensure this?

2 Phase commit

So we can use 2 phase commit. In the first phase, the services prepare the transaction. If all the services are ready, the transaction is committed. If one service fails, the transaction is rolled back.

But this is not scalable. If one service is down, the whole system is down. Also, the services are tightly coupled to each other.

What should we do instead?

Saga pattern

Let's create some microservices: order service, payment service, shipping service, each of which have its own database.

We also have a message broker like Kafka, RabbitMQ. When an order is placed, the order service raises an event. The event is sent to the message broker. The payment service listens to the event. If the payment is successful, the payment service raises an event. The shipping service listens to the event. If the shipping is successful, the shipping service raises an event.

If any of them fails, the services can compensate. For example, if the payment service fails, the order service can cancel the order. So any updates to the previous state can be reverted.

Should we query and write to the same database?

No, we should not. We should have separate databases for read and write. This way the read database is not affected by the write database. Also, the read database can be scaled independently of the write database.

This is also called CQRS pattern. Command Query Responsibility Segregation.

Event Sourcing System (Event Log)

Event Sourcing System is a system that stores all the events that happen in the system. Instead of storing the current state of the system, we store the events that lead to the current state. This way we can see the history of the system. We can also replay the events to see how the system got to the current state.

This is useful for debugging. If something goes wrong, we can replay the events to see what happened.

This is also useful for scaling. We can have multiple services that listen to the events. If one service is down, the events are stored in the event log. The service can replay the events when it is back up.

But what happens if we want to read the data?

We can have a read database that listens to the events. When an event is raised, the read database updates itself. This way the read database is always in sync with the write database. (We can use triggers in MySQL to update the read database when an event is raised.)