openstack/swift - swift - OpenDev: Free Software Needs Free Tools

Add file to the reno documentation build to show release notes for
stable/2024.1.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2024.1.
Sem-Ver: feature
Change-Id: Ic940ff424aef9cc402bf54ebe5e5fc16330fc25c

Pin selenium to 3.x for now, until we can run down the issues with 4.x
Change-Id: I596415d17f77f48a6e8a63a61b734a8ca0865847

Change-Id: I5ab762dfe0f85e346c4868ec4540884ba5f0a7f4

Change-Id: I16ec414f87ac1a5e1e87e7560290c5ef0ca4f7cf

Change-Id: I6b0a074fca4d6bd3b8b95888ae8721a62f3a6df0

The last time I really looked at this was probably Yoga, when we were
targetting 3.6 through 3.9 (and left 3.7 and 3.8 as experimental jobs).
Now, though, OpenStack is targetting 3.8 through 3.11; as before, we
can assume that if tests pass on those two versions, they should pass
on the versions in-between, too. (But still have them as experimental,
on-demand jobs).
See https://governance.openstack.org/tc/reference/runtimes/2024.1.html
Keep 2.7 and 3.6 testing as our own self-imposed minimums.
Change-Id: I7700aa3c93df311644655e7ebaf0b67aa692ee80

Change-Id: I936d9074ab60e34b379fb207d63f10bd5c3a4312

Add support for config options such as:
 head_requests_per_device_per_second = 100
Change-Id: I2936f799b6112155ff01dcd8e1f985849a1af178

Change-Id: I89050cead3ef2d5f8ebfc9cb58f736f33b1c44fe

Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Closes-Bug: #1735284
Change-Id: Ib396309c706fbc6bc419377fe23fcf5603a89f45

This change allows individual SLO segments to be downloaded by adding
an extra 'part-number' query parameter to the GET request. You can
also retrieve the Content-Length of an individual segment with a HEAD
request.
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Change-Id: I7af0dc9898ca35f042b52dd5db000072f2c7512e

Update the xena release notes configuration to build from
unmaintained/xena.
Change-Id: I72576af46cc13fac0a7a69461b673931c8b8496e

Update the wallaby release notes configuration to build from
unmaintained/wallaby.
Change-Id: Ieb2d4542d395bb1f4498beab0d9ec146fad7ba84

Update the victoria release notes configuration to build from
unmaintained/victoria.
Change-Id: I5a420de74a2ef5096135511dfea2489419414856

Add support for a backend_ratelimit_conf_path option in the
[filter:backend_ratelimit] config. If specified then the middleware
will give precedence to config options from that file over config
options from the [filter:backend_ratelimit] section.
The path defaults to /etc/swift/backend-ratelimit.conf.
The config file is periodically reloaded and any changed options are
applied. The middleware will log a warning the first time it fails to
load a config file that had previously been successfully loaded. The
middleware also logs at info level when it first successfully loads a
config file that had previously failed to be loaded. Otherwise, the
middleware will log when a config file is loaded that results in the
config being changed.
Change-Id: I6554e37c6ab5b0a260f99b54169cb90ab5718f81

Change-Id: Ie3c64646373580b70557f2720a13a5a0c5ef7097

Change-Id: I2aa13e2b23bda86c51ef6aaa69ea3fd0075bb9ad

Currently when the memcachering `_get_conns` method runs out of memcached
servers to try and so fails to yield anything we log a:
 All memcached servers error-limited
However, this error message isn't entirely accurate. It can also fail
because it failed to connect all it's memcached servers not just because
they're error limited.
You can disable error-limiting of memcached servers. So in this case
this error message is a red-herring.
Downstream we use a mcrouter client on each node which itself talks to a
bunch of memcache servers. Therefore in swift's memcachering client we
only configure the 1 mcrouter client as a single server in the ring.
Because of this we disable memcached error-limiting.
If the node gets too overloaded we've had timeouts talking to the local
mcrouter client. This fires off error-limitted log messages which can
confuse things.
Because it's possible to turn off error-limiting, the log line isn't
quite adequate anymore. So this patch changes it to:
 No more memcached servers to try
Change-Id: I97fb4f3ee2ac45831aae14a782b2c6dc73e82d85

CentOS 7 will go EOL later this year, and infra wants to drop the nodes
soon-ish -- don't make them wait on our account.
The only major loss is py2 probe tests, but officially, yoga was the
last release we pledged to support py2.
Change-Id: I8f6c247c21f16aa4717569cc69308f846c6a0245

train and ussuri are now EOL; drop them.
yoga has moved from stable to unmaintained.
Change-Id: I3516823fdacbe8fd3c2434c0de9dedd1d82980fe

Their 2.9.0 release is known-broken for py27-py35.
Change-Id: I40c1724fa673ac252f5052ac85006788ba69d5c7

Change-Id: I7c8adb129b8770eee501748a378f3adc42c8cd39

Since we fake out all the greenthread stuff to run in the main thread,
we can (sometimes?) find that a transaction ID has already been set,
leading to failures in test_bad_request_app_logging like
 AssertionError: b'X-Trans-Id: test-trans-id' not found
 in b'X-Trans-Id: tx...'
By resetting the logger's txn_id, we're assured that our mock will be
run and the expected transaction ID will be used.
Change-Id: I465eed5372a2a5e591f80a09676f4b7f091cd444

Currently, when object-server serves GET request and DiskFile
reader iterate over disk file chunks, there is no explicit
eventlet sleep called. When network outpace the slow disk IO,
it's possible one large and slow GET request could cause
eventlet hub not to schedule any other green threads for a
long period of time. To improve this, this patch add a
configurable sleep parameter into DiskFile reader, which
is 'cooperative_period' with a default value of 0 (disabled).
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: I80b04bad0601b6cd6caef35498f89d4ba70a4fd4

The existing test fails on macOS because the value of errno.ENODATA is
platform dependent. On macOS ENODATA is 96:
% man 2 intro|grep ENODATA
 96 ENODATA No message available.
Change-Id: Ibc760e641d4351ed771f2321dba27dc4e5b367c1

Object GET requests with a truthy X-Newest header are not resumed if a
backend request times out. The GetOrHeadHandler therefore uses the
regular node_timeout when waiting for a backend connection response,
rather than the possibly shorter recoverable_node_timeout. However,
previously while reading data from a backend response the
recoverable_node_timeout would still be used with X-Newest requests.
This patch simplifies GetOrHeadHandler to never use
recoverable_node_timeout when X-Newest is truthy.
Change-Id: I326278ecb21465f519b281c9f6c2dedbcbb5ff14

Both GetOrHeadHandler (used for replicated policy GETs) and
ECFragGetter (used for EC policy GETs) have _get_next_response_part
methods that are very similar. This patch replaces them with a single
method in the common GetterBase superclass.
Both classes are modified to use *only* the Request instance passed to
their constructors. Previously their entry methods
(GetOrHeadHandler.get_working_response and
ECFragGetter.response_parts_iter) accepted a Request instance as an
arg and the class then variably referred to that or the Request
instance passed to the constructor. Both instances must be the same
and it is therefore safer to only allow the Request to be passed to
the constructor.
The 'newest' keyword arg is dropped from the GetOrHeadHandler
constructor because it is never used.
This refactoring patch makes no intentional behavioral changes, apart
from the text of some error log messages which have been changed to
differentiate replicated object GETs from EC fragment GETs.
Change-Id: I148e158ab046929d188289796abfbbce97dc8d90

Change-Id: Ib790be26a2b990f313484f9ebdc99b8dc14613c9

... in document_iters_to_http_response_body.
We seemed to be relying a little too heavily upon prompt garbage
collection to log client disconnects, leading to failures in
test_base.py::TestGetOrHeadHandler::test_disconnected_logging
under python 3.12.
Closes-Bug: #2046352
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Change-Id: I4479d2690f708312270eb92759789ddce7f7f930