How to develop using a microservices architecture knowing services should not share data?

Question

I currently have a monolithic web application. Some API calls need a lot of processing resources and I would like to take that part out of my "backend" monolith and put it in its own service.

I use Docker Compose to develop my application. My database is a MongoDB replica set, to simulate what I have in production, which is a 3-nodes replica set on MongoDB Atlas. My containers in development look like this:

• frontend
• backend
• mongo0
• mongo1
• mongo2
• redis
• ...

I've read about microservices and it seems to be highly recommended that distinct services should not share data. I've never worked with a microservices architecture before, but the arguments for not sharing data made sense, so I intend to follow this advice.

So this would mean that if I split my "backend" in 2 parts, let's call them "backend" and "backend-heavy", each one would need its own database.

In production, on MongoDB Atlas, I will probably spin up another database for "backend-heavy", or even possibly another cluster. Then, I will run my "backend-heavy" service separately from "backend". But now, I'm wondering how my development setup would look like. Would I need "mongo-heavy0", "mongo-heavy1" and "mongo-heavy2" for my "backend-heavy" service, like this?

• frontend
• backend
• mongo0
• mongo1
• mongo2
• backend-heavy
• mongo-heavy0
• mongo-heavy1
• mongo-heavy2
• redis
• ...

What if I eventually want to take another part of my backend and make it its own service? Would I need another 3 containers for this third service as well? It doesn't really make sense to me. By the way, the reason I need a replica set in development is because some MongoDB commands, such as transactions, are only available with a replica set. Also, I like having my development setup as close to my production setup as possible.

Thank you!

• Robert Harvey: I've never worked with a microservices architecture before, but the arguments for not sharing data made sense -- Which arguments made sense to you, and why?
• Robert Harvey: If your architecture is already working well, except for those API calls that require additional resources, my suggestion would be to focus on a solution that solves your specific problem with those API calls, and leave the rest of the architecture untouched for now. Only split the databases if you can demonstrate that the database is the bottleneck.
• samdouble: @RobertHarvey Thank you for your answers. To your first question, I think it's easier to really decouple backend-heavy from backend if they don't use the same data. If one service needs schema changes at some point, I won't need to take into account that the new schema also needs to be compliant for the other service.
• samdouble: @RobertHarvey To your second comment, you are right, but the database does seem to be the bottleneck. Since December, we have had to upgrade our production cluster tier a few times and it's beginning to be costly. I would like my backend-heavy service to be able to create collections that pre-aggregate the data so that requests to the DB are faster and less demanding to process.
• Stack Exchange Broke The Law: @samdouble yeah but do they actually use different data?

Answer 1

But now, I'm wondering how my development setup would look like. Would I need "mongo-heavy0", "mongo-heavy1" and "mongo-heavy2" for my "backend-heavy" service, like this?

The essence of the microservices approach is that each service can be developed, tested, and deployed independently - potentially each one can be owned by a separate team.

So you would have an independent development setup for each service, and you would only have that one database as part of that setup.

Think if it like if you have an ecommerce application that uses stripe or paypal payments - you don't have your own development copy of stripe or paypal inside your development environment. You either connect it to the real the product, or you make some very simple fake version that's just enough to test your own system.

I can't tell you for sure whether traditional monolith, distributed monolith (i.e. components that are run on separate servers but share one dev setup and have to have matching versions tested together), or microservices is the best approach for you. Look carefully at the pros and cons of each, and take care not to confuse a distributed monolith with microservices.

• samdouble: Thank you, I've never heard of "distributed monolith". I will have to look more into it to be sure, but it seems as though it's more of an anti-pattern. Like some people think they have a microservice architecture, but they really have a distributed monolith where services are not really decoupled from each other.
• bdsl: @samdouble, yep, that's the context it's generally discussed in. I would say that if you need to have the databases for all the services running at the same time in the dev environment then you have a distributed monolith, not separate microservices.

Answer 2

Microservices require that you dont share databases.. You can still use the same 3 mongo instances to hold multiple databases

• samdouble: That's right, but I was wondering if was a good practice to use the same 3 MongoDB instances. I guess it wouldn't be that bad, as long as each service keeps to their own database(s)
• Robert Perry: Downvote from a fly by night down arrow clicker... Grand. I love StackExchange.
• samdouble: I just upvoted you, thank you for your comment! :)