How to get analytical information from a microservices system?

Question

In an event-driven microservice architecture, imagine we have three services:

• Customer service
• Order service
• Item service

Each service raises an event with data when something happens and stores only its related transactions.

For example, we want to know the customers that placed an order in the 30 min after they registered, and the items it contains.

In a monolith with one database it would be trivial, but I'm thinking what could be the best option in this scenario.

Could it be feasible to have an "ancillary" service that listens for all the events and stores that event data in a single relational database so then it can be queried without bothering the services itself? If not, what could be a solution?

• Ben Cottrell: Some analytics tools will automatically do this out-the-box with minimal configuration (For example Splunk already has many different listening/forwarding capabilities which can be configured to consume from any number of sources into its own analytics database). I believe DataDog has similar capabilities, along with many other similar tools -- it would be worth investigating these in case any of them meet your needs
• John Wu: In a monolith with one database it would be trivial So make one, and load it with data from your microservices on a regular basis. This is called a data warehouse
• lewislbr: Thanks both, I explored the options and Google Cloud's BigQuery seems to be a good fit for my use case.

Answer 1

Could it be feasible to have an "ancillary" service that listens for all the events and stores that event data in a single relational database so then it can be queried without bothering the services itself

You answered your own question. :-) Correct. That's exactly how I do it. Create an "Analytics" service that collates all the events/data it needs from the whole system and then produce specific views of that data from your analytics service.

• lewislbr: Nice! Glad to hear