Add a new URL parameter to allow for async cleanup of SLO segments

Code Issues Proposed changes

Add a new config option to SLO, allow_async_delete, to allow operators
to opt-in to this new behavior. If their expirer queues get out of hand,
they can always turn it back off.
If the option is disabled, handle the delete inline; this matches the
behavior of old Swift.
Only allow an async delete if all segments are in the same container and
none are nested SLOs, that way we only have two auth checks to make.
Have s3api try to use this new mode if the data seems to have been
uploaded via S3 (since it should be safe to assume that the above
criteria are met).
Drive-by: Allow the expirer queue and swift-container-deleter to use
high-precision timestamps.
Change-Id: I0bbe1ccd06776ef3e23438b40d8fb9a7c2de8921

This commit is contained in:

Tim Burke

2020年06月02日 16:14:46 -07:00

committed by Tim Burke

parent f5e03ee973

commit e78377624a

14 changed files with 635 additions and 46 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

7
etc/proxy-server.conf-sample

View File

@@ -1022,6 +1022,13 @@ use = egg:swift#slo

# clients may request that Swift send whitespace ahead of the final response

# body. This whitespace will be yielded at most every yield_frequency seconds.

# yield_frequency = 10

#

# Since SLOs may have thousands of segments, clients may request that the

# object-expirer handle the deletion of segments using query params like

# `?multipart-manifest=delete&async=on`. You may want to keep this off if it

# negatively impacts your expirers; in that case, the deletes will still

# be done as part of the client request.

# allow_async_delete = false

# Note: Put after auth and staticweb in the pipeline.

# If you don't put it in the pipeline, it will be inserted for you.

Add a new URL parameter to allow for async cleanup of SLO segments

7 etc/proxy-server.conf-sample Unescape Escape View File

7
etc/proxy-server.conf-sample

View File