-
Notifications
You must be signed in to change notification settings - Fork 10
Assume our cluster starts in a consistent state. All the hosts in the cluster has exactly the same partitioning configuration to start with. From here, we want to make changes to the partition. These includes,
- Same N hosts but different partitions;
- Add one or more hosts, with different partitions;
- Delete one or more hosts, with same/different partitions;
- Add and delete hosts, with same/different partitions;
Here's what each hosts do when new config is detected:
void handle_partition_config_change()
{
load:
new_config = load_partition_config();
if (new_config is the same as existing one)
return; // nothing to do
for (each host "H" in the new_config)
{
if (H does not have a new config)
sleep TIME_OUT secs;
if (H still does not have a new config)
goto load;
if (H's new_config is different than mine)
{
sleep 10 secs;
goto load;
}
}
// now the new_config is confirmed;
// first, collect all missing data;
missing_data = my_data(new_config) - my_data(existing_config);
collect_data(missing_data, existing_config);
// now we can response to "finished collecting missing data"
// make sure all hosts have completed collecting missing data
for (each host "H" in the new_config)
{
// also make sure they are still on the agreed config
wait_for_host_to_finish_collecting_missing_data("H");
if (timed out) goto load; // abort
}
// this is the point of no return!
// secoond, purge data that's no longer needed
redundant_data = my_data(existing_config) - my_data(new_config);
purge_data(redundant_data);
existing_config = new_config; // update config
return; // done
}
All reactions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment