openstack/swift - swift - OpenDev: Free Software Needs Free Tools

Container servers will store an etag like
 <MD5 of manifest on disk>; slo_etag=<MD5 on concatenated ETags>
which the SLO middleware will break out into separate
 "hash": "<MD5 of manifest on disk",
 "slo_etag": "\"<MD5 of concatenated ETags\"",
keys for JSON listings. Text and XML listings are unaffected.
If a middleware left of SLO already specified a container update
override, the slo_etag parameter will be appended. If the base header
value was blank, the MD5 of the manifest will be inserted.
SLOs that were created on previous versions of Swift will continue to
just have the MD5 of the manifest in container listings.
Closes-Bug: 1618573
Change-Id: I67478923619b00ec1a37d56b6fec6a218453dafc

If swift-ring-builder is erroneously given a composite builder
file, which it will fail to load, it will now print a hint
that the file is a composite builder file.
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: If4517f3b61977a7f6ca3e08ed5deb182aa87a366

This patch adds an additional optional parameter to tempurl
which restricts the ip's from which a temp url can be used from.
Change-Id: I23fe998a980960d4a32df042b3f6a21f096c36af

Yes, this means you can now start a proxy-server process on py3! And it
will give you something useful on /info!
Apparently you can even get auth tokens from tempauth, although the
tempauth unit tests don't pass yet.
Change-Id: I86ead2989b5934a7584cdd75719ce239826e01ec

Instead, require that callers provide an encoding.
Related-Change: I31408f525ba9836f634a35581d4aee6fa2c9428f
Change-Id: I3e5ed9e4401eea76c375bb43ad4afc58b1d8006a

There's already a nice way to reap a process if it's exited while not
waiting, and it doesn't require timeouts.
Change-Id: Ie327fecb6a3055ff146a94e1171ec0ec68d7179f
Related-Change: If6dc7b003e18ab4e8a5ed687c965025ebd417dfa

If we're going to drop the quotes during initialization, we should
probably drop them when making comparisons, too.
Change-Id: I90c6e8a7c6faf8fafb0cd64fabbc9629d6c4c48a

Change-Id: Iee1bab5775b243c318aa22ee4a548d793e6684ca

The domain_remap middleware didn't need to change, only its tests.
Change-Id: I5edf05b609ce263708f851219c38c1bd1b645a6f

Typed "b" in a half-dozen places and fixed a typo in a test name.
Change-Id: Idcb8b2eeef5b545eb0944cc22b7c7961e8c2f8f0

Change-Id: I83a35f066d69edf39c74fd22c7fb248b5320d21f

The ignored_secret variable wasn't being used.
Change-Id: Ia659f3672941f9bb86054ce6cafbbe6fe7ccafbc
Related-Change: I1cd9a97c4e8d87d7c065866e7ad3a9e748ff19ab

Otherwise, a sharded container AUTH_test/sharded will have its stats
included in the totals for both AUTH_test *and* .shards_AUTH_test
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Change-Id: I7fa74e13347601c5f44fd7e6cf65656cc3ebc2c5

Today's diff is brought to you by the letter "b".
Change-Id: I074c1ff3f1232a899c101f2856dff4a881a6fe77

Only the tests needed to change, and it was nearly all just prepending
"b" to a bunch of string literals.
Change-Id: Id5317e471c04a05627438c281bb0d6a97cf6a8a4

Change-Id: Ibdb9e2bbec1c962d930a3f69fc95a8c562ac13b7

...which helps us differentiate between a drive that's not mounted vs.
not a dir better in log messages. We were already doing that a bit in
diskfile.py, and it seems like a useful distinction; let's do it more.
While we're at it, remove some log translations.
Related-Change: I941ffbc568ebfa5964d49964dc20c382a5e2ec2a
Related-Change: I3362a6ebff423016bb367b4b6b322bb41ae08764
Change-Id: Ife0d34f9482adb4524d1ab1fe6c335c6b287c2fd
Partial-Bug: 1674543

Change-Id: I31408f525ba9836f634a35581d4aee6fa2c9428f

Alternate approach to https://review.openstack.org/#/c/559902
Change-Id: I1e244c231753b8f4b6f1cf95cb0ae4c3c959ae0f

Resolve outstanding TODO's. One TODO is removed because there isn't an
easy way to arrange for an async pending to be targeted at a shard
container.
Change-Id: I0b003904f73461ddb995b2e6a01e92f14283278d

eventlet<=0.17.4 LoggerFileWrapper does not have an error() method
so don't try to call it. Instead, use info() which even in newer
eventlet versions does exactly the same as error().
This bug only manifests in unit tests once the Related-Bug has been fixed
(see Related-Change).
Closes-Bug: #1777836
Related-Bug: #1777701
Related-Change: I68fd9d0263a6720aaf0d90b8ea6da1158105ac01
Change-Id: I46190567549826c811ffa51e9a71a38a20d9ce97

Older versions of eventlet expect getsocketname() to return a tuple,
and newer versions are happy to receive a tuple, so fix the mock to
return a tuple.
Change-Id: I68fd9d0263a6720aaf0d90b8ea6da1158105ac01
Closes-Bug: #1777701 

If a WSGI application produces the header "Content-Length: <N>" but
does not produce exactly N bytes of response, then that is an error
and an exception should be thrown so that the WSGI server can take the
correct action (close the TCP connection for HTTP <= 1.1, something
else for HTTP 2.0).
As part of this, I also fixed a bug in DLOs where a HEAD response
might have a body. The way it works is this:
 * user makes HEAD request for DLO manifest
 * DLO middleware makes GET request for container
 * authorize callback (e.g. from tempurl) replies 401 for container
 GET; response has a nonempty body (it's a GET response; that's
 fine)
 * DLO notes that response is non-2xx, returns it as-is
 * client gets response with nonempty body to a HEAD request
The fix there was simple; if the original request method was HEAD,
clear out the response body.
Change-Id: I74d8c13eba2a4917b5a116875b51a781b33a7abf
Related-Bug: 1568650

Every other middleware has files swift/common/middleware/<mw>.py and
test/unit/common/middleware/test_<mw>.py, while catch_errors has
test_except.py, which makes its tests harder to find. Let's follow the
same naming pattern.
Change-Id: I534ff57ac2fe8e12a3f84d821da04d679f1ced3a

Change-Id: Ifc3f2f58f51c71f4e652567b4f9c82deb98bbb5b

... so we can use storage_url for actual, complete URLs.
Also, expose storage_scheme for easier comparisons.
Change-Id: Iffcfc327bc674681ac51fe75c1d9911fa3a60d54