openstack/swift - swift - OpenDev: Free Software Needs Free Tools

When we fixed bulk response heartbeating in https://review.openstack.org/#/c/510715/,
code review raised the issue of moving the xml header down to after the
early-exit clauses. At the time, it didn't seem to break anything, so
it was left in place. However, that insight was correct.
The purpose of the earlier patch was to force eventlet to use chunked
transfer encoding on the response in order to prevent eventlet from
buffering the whole response, thus defeating the purpose of the
heartbeat responses.
Moving the first line of the body lower (ie after the early exit
checks), allows other headers in a chunked transfer encoding response
to be appropriately processed before sending the headers. Sending the
xml declaration early causes it to get intermingled in the 100-continue
protocol, thus breaking the chunked transfer encoding semantics.
Closes-Bug: #1819252
Change-Id: I072f4dab21cd7cdb81b9e41072eb504131411dc8

We really only need to have one way to cleanup empty suffix dirs, and
that's normally during suffix hashing which only happens when invalid
suffixes get rehashed.
When we iterate a suffix tree using yield hashes, we may discover an
expired or otherwise reapable hashdir - when this happens we will now
simply invalidate the suffix so that the next rehash can clean it up.
This simplification removes an mis-behavior in the handling between the
normal suffix rehashing cleanup and what was implemented in ssync.
Change-Id: I5629de9f2e9b2331ed3f455d253efc69d030df72
Related-Change-Id: I2849a757519a30684646f3a6f4467c21e9281707
Closes-Bug: 1816501

You could *try* doing something similar to what we were doing
there over in email.message for py3, but you would end up
breaking pkg_resources (and therefor entrypoints) in the
process.
Drive-by: have mem_diskfile implement more of the diskfile API.
Change-Id: I1ece4b4500ce37408799ee634ed6d7832fb7b721

Besides being easier to reason about, this also lets us
run more unit tests under py37 which complains about a
a generator raising StopIteration
Change-Id: Ia6b945afef51bcc8ed20a7069fc60d5b8f9c9c0b

Change-Id: I83f8f59023eabc97386481c18ed8bbf8fab64fa8

Looks like some base templates got moved from xenial to bionic, which
doesn't have py35. Explicitly say that this job needs xenial.
Change-Id: I44df8736d0c33fc2c58c9be6b5b8023932f14a83

Change-Id: I51708cb2f0deca61b147589e062b520ac7a1807e

Change-Id: I46bdb6da8f778a6c86e0f8e883b52fc31e9fd44e
Partial-Bug: 1774238
Closes-Bug: 1678022
Closes-Bug: 1598093
Closes-Bug: 1762997

`dir` is a reserved instruction term in python, so this patch
avoiding to assing a value to it.
Change-Id: If780c4ffb72808b834e25a396665f17bd8383870

Replace 'o' with 'to'.
Change-Id: I0a9b1547016b2662002c050e8388591d7d91ef97

cls should be used in classmethd, instead of self.
Change-Id: I149b18935ce909ef978f2b7147b109e11c22f923

There's some code duplication we can drop, and some tests scenarios we
can expand on. I don't believe there's any behavior change here.
Change-Id: I2271d1cb757c989c4b0bfe228cd26c8620a151db

Change-Id: I2610cbdc9b7bc2b4d614eaedb4f3369d7a424ab3

Drive-by: use six.moves in s3api; fix "unexpected indent" warning when building
docs on py3
Change-Id: I2a354e2624c763a68fcea7a6404e9c2fde30d631

Change-Id: I5a0d2108d01395aedb4ac18464ed2e38ad13ce84

- ConfigParser.set() requires that the value be a string
- The stdlib HTTP client responses don't have a body property
- We might raise a ResponseError with response=None
- Bodies should be bytes
- Headers should be strings
- Make containers()/files() return native strings
- file() isn't a thing on py3
- format should be a parm, not a header
- Switch sorted() to use key instead of cmp
- Use integer division explicitly
Change-Id: I99d3eebc9d7ec4e8b295352294b831492135c568

Change-Id: Ib1cd0659d5085ff9f8555bc08bd17a8f02f2efc0

Turns out, unquote()ing WSGI strings is a great way to mangle them.
Change-Id: I42a08d84aa22a1a7ee7ccab97aaec55d845264f9

Tracebacks are ugly.
Change-Id: I09b907608127e4c633b554be2926245b35402dbf

Change-Id: Ic00273e5d3120f7daba1d3cbf084159f1e7d8bbb

Change-Id: I8f3d4d5f6976ef5b63facd9b5723aac894066b74

Change-Id: Ia2ba8f7a998bc55ba3fba3ce37f1d2974646494f

Change-Id: I82503b13e4541196ad056e861221e9429c7f2c1c

That way we don't traceback when you just provide a DB file.
At some point, py3 switched to having optional subparsers. In py37,
they added a kwarg to say whether a subparser is optional or required,
but for the sake of earlier versions, we have to check whether it was
set manually and cannot rely on argparse doing it for us. See also:
- https://bugs.python.org/issue9253
- https://github.com/python/cpython/commit/aaf6fc0
- https://bugs.python.org/issue33109
- https://github.com/python/cpython/commit/8ebf5ce
Change-Id: Iac8048ad0e23ffb28b914fe880c4b6538bc41c86

- Drop the (partial) logging translation
- Save our log concatenations until the end
- Stop encoding object names; direct_client is happy to take Unicode
- Remove a couple loop breaks that were only used by tests
Change-Id: I4a4f301a7a6cb0f217ca0bf8712ee0291bbc14a3
Partial-Bug: #1674543 

* name_check
 * quotas
 * ratelimit
 * read_only
 * recon
Note that the middlewares themselves seem to be fine.
Fix proxy-logging for PEP-0479.
Change-Id: If47d09e8e8c09b9f50451f93d5b0803aa58743a7

Change-Id: Id74a93f10bc5c641d62141af33bef68e503f7e04

This started with ShardRanges and its CLI. The sharder is at the
bottom of the dependency chain. Even container backend needs it.
Once we started tinkering with the sharder, it all snowballed to
include the rest of the container services.
Beware, this does affect some of Python 2 code. Mostly it's trivial
and obviously correct, but needs checking by reviewers.
About killing the stray "from __future__ import unicode_literals":
we do not do it in general. The specific problem it caused was
a failure of functional tests because unicode leaked into a field
that was supposed to be encoded. It is just too hard to track the
types when rules change from file to file, so off with its head.
Change-Id: Iba4e65d0e46d8c1f5a91feb96c2c07f99ca7c666

* We just caught it to raise another one
* On newer versions of python, we wouldn't actually catch
 anything, because HTTPPreconditionFailed is not an instance
 of BaseException -- it's a partial function application
 which when called will return an exception.
Change-Id: Ia44e832e4198e75f3337884c4612f4e9fe3b9e0b

Change-Id: I1095262563eff92d6d0a399eb0d5f9d3045becf3

Change-Id: Ida7c776201a068d44572d1e94472c975c4bc8e36

Change-Id: I2701358f9f01f368c892a010a543ecc1f17684a3