relinker: Parallelize per disk

Code Issues Proposed changes

Add a new option, workers, that works more or less like the same option
from background daemons. Disks will be distributed across N worker
sub-processes so we can make the best use of the I/O available.
While we're at it, log final stats at warning if there were errors.
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: I039d2b8861f69a64bd9d2cdf68f1f534c236b2ba

This commit is contained in:

Tim Burke

2021年01月06日 16:18:04 -08:00

parent b79ca57fa9

commit abfa6bee72

3 changed files with 357 additions and 127 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

9
etc/object-server.conf-sample

View File

@@ -610,7 +610,12 @@ use = egg:swift#xprofile

# log_level = INFO

# log_address = /dev/log

#

# Target this many relinks/cleanups per second, to reduce the

# Start up to this many sub-processes to process disks in parallel. Each disk

# will be handled by at most one child process. By default, one process is

# spawned per disk.

# workers = auto

#

# Target this many relinks/cleanups per second for each worker, to reduce the

# likelihood that the added I/O from a partition-power increase impacts

# client traffic. Use zero for unlimited.

# files_per_second = 0.0

@@ -632,4 +637,4 @@ use = egg:swift#xprofile

# useful if previous partition power increases have failed to cleanup

# tombstones from their old locations, causing duplicate tombstones with

# different inodes to be relinked to the next partition power location.

# link_check_limit = 2

relinker: Parallelize per disk

9 etc/object-server.conf-sample Unescape Escape View File

9
etc/object-server.conf-sample

View File