Skip to content

Navigation Menu

Sign in
Sign up

Partition and Replication - Dealing with Changes #14

ytyou started this conversation in Ideas
Discussion options

Assume our cluster starts in a consistent state. All the hosts in the cluster has exactly the same partitioning configuration to start with. From here, we want to make changes to the partition. These includes,

- Same N hosts but different partitions;
- Add one or more hosts, with different partitions;
- Delete one or more hosts, with same/different partitions;
- Add and delete hosts, with same/different partitions;

Here's what each hosts do when new config is detected:

void handle_partition_config_change()
{
load:
 new_config = load_partition_config();
 if (new_config is the same as existing one)
 return; // nothing to do
 for (each host "H" in the new_config)
 {
 if (H does not have a new config)
 sleep TIME_OUT secs;
 if (H still does not have a new config)
 goto load;
 if (H's new_config is different than mine)
 {
 sleep 10 secs;
 goto load;
 }
 }
 // now the new_config is confirmed;
 // first, collect all missing data;
 missing_data = my_data(new_config) - my_data(existing_config);
 collect_data(missing_data, existing_config);
 // now we can response to "finished collecting missing data"
 // make sure all hosts have completed collecting missing data
 for (each host "H" in the new_config)
 {
 // also make sure they are still on the agreed config
 wait_for_host_to_finish_collecting_missing_data("H");
 if (timed out) goto load; // abort
 }
 // this is the point of no return!
 // secoond, purge data that's no longer needed
 redundant_data = my_data(existing_config) - my_data(new_config);
 purge_data(redundant_data);
 existing_config = new_config; // update config
 return; // done
}
You must be logged in to vote

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Ideas
Labels
None yet
1 participant

AltStyle によって変換されたページ (->オリジナル) /