openstack/swift - swift - OpenDev: Free Software Needs Free Tools

This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.
Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.
Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are
normalized.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: Ib8b04669a50a84345f2ad8456a73473aec2df2f4
Story: #2002586
Task: #24337 

For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html
Change-Id: I10424a57f36b0a5e83456bb07d86f6a7444a304b

I've at least tried it out with a py3 proxy, and it seems to work out
OK. I haven't tried killing the socket and verifying that it's actualy
dead, but getting a hold of _real_close *seems like* what we want?
At least the three (!!) tests pass.
Change-Id: Ic08c26185d63a36a5422793d81f621e0698fa572

Seen locally:
 Warning, treated as error:
 .../swift/doc/source/overview_policies.rst:555:more than one target
 found for cross-reference u'get_data_dir':
 swift.obj.reconstructor.get_data_dir, swift.obj.replicator.get_data_dir,
 swift.obj.diskfile.get_data_dir
Not sure why it hasn't been seen in the gate...
The whole sentence is suspect, though; the Diskfile class doesn't define
a get_data_dir method, though it uses the module-level get_data_dir...
Change-Id: I6855c82315e1c71596ecce25b66b54133c239377

Change-Id: Ida9c204f644e41ae05dbf6f68083410e143d3183

Change-Id: I76af04899b696066b009aa753ce71091f3cf874c

... so we can more easily expand their irrelevant-files.
Change-Id: Id365128f524aac0200c611307415feea83c40178

Change-Id: Ia3a8d065e849ec9fed780a6927674e27cbed522b

Change-Id: Ifefe01cc28517a7040596885b54acf3faa75be85

Change-Id: I9d83dc7823cf35a94d7e54c161c40e79911aa48f

This is comparable to what AWS returns, and should greatly simplify
debugging when diagnosing 403s.
Change-Id: Iabfcbaae919598e22f39b2dfddac36b75653fc10

Now that the trivial keymaster supports multiple keys, let's do
something similar for the KMIP keymaster. Additional keys are
configured as:
 key_id_<secret_id> = <KMIP unique identifier>
While it might be tempting to use the unique identifier directly as the
secret_id, the added indirection allows operators to move keys between
different backends, which may cause different identifiers to be issued.
As with the trivial keymaster, the key to use for PUTs and POSTs is
specified with:
 active_root_secret_id = <secret_id>
Change-Id: Ie52508e47d15ec5c4e96902d3c9f5f282d275683

For some use cases operators would like to periodically introduce a
new encryption root secret that would be used when new object data is
written. However, existing encrypted data does not need to be
re-encrypted with keys derived from the new root secret. Older root
secret(s) would still be used as necessary to decrypt older object
data.
This patch modifies the KeyMaster class to support multiple root
secrets indexed via unique secret_id's, and to store the id of the
root secret used for an encryption operation in the crypto meta. The
decrypter is modified to fetch appropriate keys based on the secret id
in retrieved crypto meta.
The changes are backwards compatible with previous crypto middleware
configurations and existing encrypted object data.
Change-Id: I40307acf39b6c1cc9921f711a8da55d03924d232

Test B109 was removed from bandit in release 1.5.0[1]
[1] - d93eed5492
Change-Id: I57ea610b924c2140b7572762c44cbc6510d0f549

The previous locking method would leave the lock dir lying around
if the process died unexpectedly, preventing others swift-recon-cron
process from running sucessfuly and requiring a manual clean.
Change-Id: Icb328b2766057a2a4d126f63e2d6dfa5163dd223

Otherwise, users can create buckets in accounts they don't own.
Change-Id: I13d557c32b12529ef1087c52f7af302a33d33acb

On vanilla Swift, deleting an object that doesn't exist will 404.
On AWS, deleting a key that doesn't exist will either 404 if the bucket
doesn't exist (with a NoSuchBucket code) or 204 (because yep, that's not
accessible).
Change-Id: Ied2a78b56522316bb374f23961621641af3adc83
Related-Change: I6e154594dfda6c3065774c23b24f728625a842bc

It was actually testing the invalid-JSON handling before...
Change-Id: Ia8b5eaeb42fea5136525c80e67e8d33548c2a8df

Added healthcheck middleware to account, container, object servers
Added the s3api, keymaster, encryption config to the proxy config
file to make it easy to enable it.
Change-Id: I96f120c5bc416e9aba388cbfa6c30b648d6ade2f

For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html
Change-Id: I04d4c26b762ba3e1c540b0de95d293e77ad9add6

Change-Id: I3bce1c4efeb3a3a7319020de76ba7f06015a5a36

We don't support it yet, so return 501 Not Implemented.
Change-Id: Ie2f4bd1bfdb1bcbdf1a0f0db9d542b6057e9d2ec

...instead of just check jobs. While we're at it, drop the voting line;
it's voting by default.
Change-Id: I478a82bbac7ba19ed81aae1f5225ffc4e10fb2cb

We don't support it yet, so return 501 Not Implemented. Previously, we'd
store the aws-chunked content (!) and most clients would see it as data
corruption.
See https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
for more information.
Change-Id: I697962039667980ef89212bc480f8b1d3fbd718c

These files are imported (and very lightly edited) from the old
ocata user-guide. It has a few other swift-related docs that seemed
more duplacative of what we already have, but these seem to fill
existing gaps in our docs.
Change-Id: Ib00bf6992327f15f271120dc5dbc86a4a235baec

Change-Id: Ia420ac3f84284eab9d0a371e4d56be6e45994fae

I don't see why running `python setup.py sdist` should leave me
with a dirty checkout.
Change-Id: Id932780c2e555b77a0066c70641684c0ddfdd79e

This used to be necessary on older eventlet, or you'd get TypeErrors
when you went to reraise. Following eventlet 0.13.0, however, it's just
extra code.
For the original eventlet issue, see
https://web.archive.org/web/20140823005223/https://bitbucket.org/eventlet/eventlet/issue/149/yield-in-except-clause-with-wilcard-raise
Change-Id: I19ad0968a82827bdd4ef75fde9ed51f193627d6e
Related-Bug: 1181146

Change-Id: Iddc0f333861b6c1f81e181f006cd592b5eb6ea17

... instead of having KmipKeyMaster instantiate its own logger
inside of _get_root_secret.
Bonus: KmsKeyMaster uses its own log route now.
Bonus bonus: get rid of KmsKeyMaster's pointlessly-overriden
__init__ method.
Change-Id: Idb4b832e5ca0e3d749fe2c0b7ba283447a4dc69e