DB design while designing the system

Question

Recently we had a discussion if rough DB schema (high level tables and columns) design should be part of system design phase or not.

We have two confronting approaches in the company. Let's assume System Design consists of designing Queues, Lambdas, integration with other microservices, etc.

Approach 1) In system design we should also include (at least in rough idea) what tables should be in DB, with relations, and normalization.

Approach 2) During system design we should just state that DB is there, and that's it. Later DB schema will naturally evolve after logic is implemented and needs to be persisted.

What do you think?

Answer 1

It is never wrong to have a rough idea of what your goals are, at any stage of design, as long as it is a rough idea, a concept on a piece of paper, and nothing engraved in stone. You could also envision certain tables and attributes if it helps you to get a clearer picture. But make sure everyone in the team takes this for what it is, and especially for what it is not: it is not a blueprint of what must be built afterwards.

When it comes to table design with attributes, relations, and normalization, it is a good idea to only implement exactly the pieces one needs for the current iteration in the database, for supporting the persistence required for the use case or the user story being planned for the next release, and nothing more, and nothing just because it was in the former concept.

Often it turns out that the former "vision" was roughly correct, but the details are quite different, so the team will be happy that it won't become necessary to fix too many of those differences at the implementation level.

So, in short: the two alternatives presented in your question don't need to be in conflict when you take the right point of view.

• IMSoP: A counterpoint to the "only build enough for the current iteration" thinking is that migrating data to a different structure later may be tricky, particularly for frequently referenced tables, or those which will have a high number of rows. It might save a lot of work later to anticipate, say, a many-to-many relationship. Of course, as with any such prediction there is a risk of the design changing to no longer use that flexibility, so there is a lot of truth to what you say; I'd just add a pinch of pragmatic salt.

Answer 2

I think a data model is an essential part of most designs. This means you have expressed the major entities and their relationships in the system you are designing. How is a real world problem and solution mapped to a computer program? You'll quickly fail when working in a team if you go into coding without having reached clarity on how you are going to solve the problem.

When I read database tables, columns, these are implementation artifacts. There is no need to write this all out. You note down there is a persistence layer needed.

There are couple exceptions. Sometimes the data structures are dictated by existing designs. Sometimes they are generic, like log files. In an exploratory design, the may not be enough information available. But they are never 'obvious' - any two developers will write vastly different implementations when it is 'obvious' ;-)