openstack/swift - swift - OpenDev: Free Software Needs Free Tools

Instead, require that callers provide an encoding.
Related-Change: I31408f525ba9836f634a35581d4aee6fa2c9428f
Change-Id: I3e5ed9e4401eea76c375bb43ad4afc58b1d8006a

Change-Id: Iee1bab5775b243c318aa22ee4a548d793e6684ca

The domain_remap middleware didn't need to change, only its tests.
Change-Id: I5edf05b609ce263708f851219c38c1bd1b645a6f

Change-Id: I83a35f066d69edf39c74fd22c7fb248b5320d21f

Typed "b" in a half-dozen places and fixed a typo in a test name.
Change-Id: Idcb8b2eeef5b545eb0944cc22b7c7961e8c2f8f0

The ignored_secret variable wasn't being used.
Change-Id: Ia659f3672941f9bb86054ce6cafbbe6fe7ccafbc
Related-Change: I1cd9a97c4e8d87d7c065866e7ad3a9e748ff19ab

Otherwise, a sharded container AUTH_test/sharded will have its stats
included in the totals for both AUTH_test *and* .shards_AUTH_test
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Change-Id: I7fa74e13347601c5f44fd7e6cf65656cc3ebc2c5

Today's diff is brought to you by the letter "b".
Change-Id: I074c1ff3f1232a899c101f2856dff4a881a6fe77

Only the tests needed to change, and it was nearly all just prepending
"b" to a bunch of string literals.
Change-Id: Id5317e471c04a05627438c281bb0d6a97cf6a8a4

Change-Id: Ibdb9e2bbec1c962d930a3f69fc95a8c562ac13b7

...which helps us differentiate between a drive that's not mounted vs.
not a dir better in log messages. We were already doing that a bit in
diskfile.py, and it seems like a useful distinction; let's do it more.
While we're at it, remove some log translations.
Related-Change: I941ffbc568ebfa5964d49964dc20c382a5e2ec2a
Related-Change: I3362a6ebff423016bb367b4b6b322bb41ae08764
Change-Id: Ife0d34f9482adb4524d1ab1fe6c335c6b287c2fd
Partial-Bug: 1674543

Change-Id: I31408f525ba9836f634a35581d4aee6fa2c9428f

Alternate approach to https://review.openstack.org/#/c/559902
Change-Id: I1e244c231753b8f4b6f1cf95cb0ae4c3c959ae0f

Resolve outstanding TODO's. One TODO is removed because there isn't an
easy way to arrange for an async pending to be targeted at a shard
container.
Change-Id: I0b003904f73461ddb995b2e6a01e92f14283278d

eventlet<=0.17.4 LoggerFileWrapper does not have an error() method
so don't try to call it. Instead, use info() which even in newer
eventlet versions does exactly the same as error().
This bug only manifests in unit tests once the Related-Bug has been fixed
(see Related-Change).
Closes-Bug: #1777836
Related-Bug: #1777701
Related-Change: I68fd9d0263a6720aaf0d90b8ea6da1158105ac01
Change-Id: I46190567549826c811ffa51e9a71a38a20d9ce97

Older versions of eventlet expect getsocketname() to return a tuple,
and newer versions are happy to receive a tuple, so fix the mock to
return a tuple.
Change-Id: I68fd9d0263a6720aaf0d90b8ea6da1158105ac01
Closes-Bug: #1777701 

If a WSGI application produces the header "Content-Length: <N>" but
does not produce exactly N bytes of response, then that is an error
and an exception should be thrown so that the WSGI server can take the
correct action (close the TCP connection for HTTP <= 1.1, something
else for HTTP 2.0).
As part of this, I also fixed a bug in DLOs where a HEAD response
might have a body. The way it works is this:
 * user makes HEAD request for DLO manifest
 * DLO middleware makes GET request for container
 * authorize callback (e.g. from tempurl) replies 401 for container
 GET; response has a nonempty body (it's a GET response; that's
 fine)
 * DLO notes that response is non-2xx, returns it as-is
 * client gets response with nonempty body to a HEAD request
The fix there was simple; if the original request method was HEAD,
clear out the response body.
Change-Id: I74d8c13eba2a4917b5a116875b51a781b33a7abf
Related-Bug: 1568650

Every other middleware has files swift/common/middleware/<mw>.py and
test/unit/common/middleware/test_<mw>.py, while catch_errors has
test_except.py, which makes its tests harder to find. Let's follow the
same naming pattern.
Change-Id: I534ff57ac2fe8e12a3f84d821da04d679f1ced3a

Change-Id: Ifc3f2f58f51c71f4e652567b4f9c82deb98bbb5b

... so we can use storage_url for actual, complete URLs.
Also, expose storage_scheme for easier comparisons.
Change-Id: Iffcfc327bc674681ac51fe75c1d9911fa3a60d54

Change-Id: I7231e8af310e268484f2075f0194b7783cf1c3ea

Provides a simple, experimental, CLI tool to generate a
composite ring from a list of component builder files.
For example:
 swift-ring-composer <composite-file> compose \
 <builder-file> <builder-file> --output <ring-file>
Commands available:
- compose: compose a list of builder file to a composite ring
- show: show the metadata for a composite ring
Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Co-Authored-By: Matthew Oliver <matt@oliver.net.au>
Change-Id: I25a79e71c13af352e19e4358f60545265b51584f

The object updater now supports two configuration settings:
"concurrency" and "updater_workers". The latter controls how many
worker processes are spawned, while the former controls how many
concurrent container updates are performed by each worker
process. This should speed the processing of async_pendings.
There is a change to the semantics of the configuration
options. Previously, "concurrency" controlled the number of worker
processes spawned, and "updater_workers" did not exist. I switched the
meanings for consistency with other configuration options. In the
object reconstructor, object replicator, object server, object
expirer, container replicator, container server, account replicator,
account server, and account reaper, "concurrency" refers to the number
of concurrent tasks performed within one process (for reference, the
container updater and object auditor use "concurrency" to mean number
of processes).
On upgrade, a node configured with concurrency=N will still handle
async updates N-at-a-time, but will do so using only one process
instead of N.
UpgradeImpact:
If you have a config file like this:
 [object-updater]
 concurrency = <N>
and you want to take advantage of faster updates, then do this:
 [object-updater]
 concurrency = 8 # the default; you can omit this line
 updater_workers = <N>
If you want updates to be processed exactly as before, do this:
 [object-updater]
 concurrency = 1
 updater_workers = <N>
Change-Id: I17e18088e61f664e1b9942d66423666d0cae1689

Change-Id: I63428132283986bda9e5c082ffe85741449b71ba

ContainerBroker adds the shard_max_row item to the
get_replication_info result by querying the db for the shard ranges
table max rowid. However, the wrong table name was being used in the
db query such that the value was always -1. This bug was benign
because the value of shard_max_row is not currently used.
Noticed while reviewing [1] which does make use of the shard_max_row
*key* in replication info.
[1] Related-Change: I7231e8af310e268484f2075f0194b7783cf1c3ea
Change-Id: I9e733e301894f1ffff4a1092926cc0df8419c5b5