proxy: Add a chance to skip memcache for get_*_info calls

Code Issues Proposed changes

If you've got thousands of requests per second for objects in a single
container, you basically NEVER want that container's info to ever fall
out of memcache. If it *does*, all those clients are almost certainly
going to overload the container.
Avoid this by allowing some small fraction of requests to bypass and
refresh the cache, pushing out the TTL as long as there continue to be
requests to the container. The likelihood of skipping the cache is
configurable, similar to what we did for shard range sets.
Change-Id: If9249a42b30e2a2e7c4b0b91f947f24bf891b86f
Closes-Bug: #1883324

This commit is contained in:

Tim Burke

2022年01月06日 12:09:58 -08:00

committed by Matthew Oliver

parent 24acc6e56b

commit 5c6407bf59

11 changed files with 327 additions and 223 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

2
etc/proxy-server.conf-sample

View File

@@ -153,8 +153,10 @@ use = egg:swift#proxy

# data is present in memcache, we can periodically refresh the data in memcache

# without causing a thundering herd. Values around 0.0 - 0.1 (i.e., one in

# every thousand requests skips cache, or fewer) are recommended.

# container_existence_skip_cache_pct = 0.0

# container_updating_shard_ranges_skip_cache_pct = 0.0

# container_listing_shard_ranges_skip_cache_pct = 0.0

# account_existence_skip_cache_pct = 0.0

#

# object_chunk_size = 65536

# client_chunk_size = 65536

proxy: Add a chance to skip memcache for get_*_info calls

2 etc/proxy-server.conf-sample Unescape Escape View File

2
etc/proxy-server.conf-sample

View File