I am using the RAFT consensus theorem through the Apache Ratis library. In my use case, lets say for a particular command my application performs multiple actions (logical calculation, updating new values in local variables, persisting the value in a file/DB). In such case what would happen if the leader node crashes in the between the actions?
- While implementing this use case through Ratis, I killed the leader node explicitly in between the actions and the rest of the actions were left un-performed for the particular command. However, one of the follower does reply back to client (still as a follower, and not a leader).
- As per my logic, only the LEADER was writing the value to the file (replicating the case where there would be only 1 instance of a DB).
So, does anyone know how does RAFT handles this case? Whether RATIS provides any support for such a use case ? or any other library which provides support for this?
What I have already tried ? I tried another logic where I would fire individual commands (for each action) through the LEADER against each command for the client, so that each action is tracked in the logs and can be picked up by the next leader. But this failed, as neither, the original and the internal commands were completed and got stuck in sort of a deadlock.
-
Ideally state machine should be deterministic. Your state machine should not rely on any other systems. What would happen if your DB is running very slow or down? Ideal mechanism would be your state machine can emit events which can be applied to other systems. and the dependent system should be idempotentArjun Vachhani– Arjun Vachhani2024年09月06日 17:48:11 +00:00Commented Sep 6, 2024 at 17:48
-
In your case only leader writes to DB, I think this is a bad design. you will end up in situation where some logs are applied to DB twice and some logs are not at all applied. Suppose a log is replicated to all node but not committed and leader failed. Suppose a log is applied on leader but before sending commit to followers it crashed.Arjun Vachhani– Arjun Vachhani2024年09月06日 18:04:55 +00:00Commented Sep 6, 2024 at 18:04