Data replication with microservices using a messaging broker

Question

I have a microservice based application like so:

User.Microservice - stores , user information.

Product.Microservice - stores products that user created.

Order.Microservices - stores product orders

Stack:

RabbitMQ - for Event based communication

MongoDB - for Data Storage

Each service has a public HTTP REST API.

Each service can publish and consume messages from Rabbit MQ.

Here is how everything communicates: current stack

Client makes a request to API gateway:

lets suppose client makes a POST request to product service -> the product has user reference and other details about a product.

Well now if I want to populate product item with user details such as user's username, the API gateway should make a GET request to user service.

Here comes the data replication part i have to add the required information about user to product service.

-

How can i replicate user details to product service after a product is created ? And return the response to POST request along with user's username ? if it is not present in product service...

-

I could use the current API gateway method where the gateway calls user service , but if the user service goes down i will not have the required user details...

Somehow i need to run bus events before the response from POST request is returned...

Answer 1

The idea of a message bus is to run things asynchronously. Message exchange is not supposed to take place while a client is waiting for your response. It should happen in the background, so for example if your ProductService needs some data about users, it should listen to events from UserService about user changes, and apply them to its own local copy of the data as the events arrive. When it comes time to answer a synchronous user request, ProductService already has the data it needs in its local storage and can just fetch it. Note that there may be some delay between changes in one service and their visibility in the other, and that there will be extra storage needs. And that duplicated data will at some point get out of sync.

If on the other hand (due to freshness requirements, for example) you need to pull data in real time from one service to another, use a standard, synchronous REST call.

• Dorin Musteața: Yeah , i get it , but REST calls are expensive, it will add big latency. It's easier to have a replicated data. And the idea here is that ProductService should have the only required user details , i mean if only 100 users posted 100 products , the ProductService should contain only 100 replicated users
• Nima Boobard: I compeletly agree with @Michal Kosmulski answer. Do not stick to replication tools, IT SUCKS. The eventual costistency would a better option. You shoudn't create a copy in all of the events, You just need to update your User storage if the newer one has come. Either way, you should update User changes in its storage. You're not going to query the User everytime. You've already had it in most cases. So you can be sure of the stability of your services.