openstack/swift - swift - OpenDev: Free Software Needs Free Tools

This patch removed the separate s3api, staticweb functional tests
gate jobs and added them across all other functional test jobs.
Change-Id: Ie1c606132a054defc2b3cc14a66031090e7b8449

New flake8 came out with new & improved rules. Ignore E741; it would be
too much churn. Fix the rest.
Change-Id: I9125c8c53423232309a75cbcc5b695b378864c1b

The global upper constraints repo will no longer track py2 packages,
meaning upper constraints for the py2.7 runtime will need to be handled
locally.
This adds an in-repo file for upper constraints enforcement using the
last py27 constraints from openstack/requirements to validate
functionality.
Change-Id: I624f914c6155178cbe474f1eae2871e2aef721eb
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>

Change-Id: Ia968ec7375ab346a2155769a46e74ce694a57fc2

The repo is Python using both Python 2 and 3 now, so update hacking to
version 2.0 which supports Python 2 and 3. Note that latest hacking
release 3.0 only supports version 3.
Fix problems found.
Remove hacking and friends from lower-constraints, they are not needed
for installation.
Change-Id: I9bd913ee1b32ba1566c420973723296766d1812f

Swift servers can now be seamlessly reloaded by sending them a SIGUSR1
(instead of a SIGHUP). The server forks off a synchronized child to
wait to close the old listen socket(s) until the new server has started
up and bound its listen socket(s). The new server is exec'ed from the
old one so its PID doesn't change. This makes Systemd happier, so a
ReloadExec= stanza can now be used.
The seamless part means that incoming connections will alwyas get
accepted either by the old server or the new one. This eliminates
client-perceived "downtime" during server reloads, while allowing the
server to fully reload, re-reading configuration, becoming a fresh
Python interpreter instance, etc. The SO_REUSEPORT socket option has
already been getting used, so nothing had to change there.
This patch also includes a non-invasive fix for a current eventlet bug;
see https://github.com/eventlet/eventlet/pull/590
That bug prevents a SIGHUP "reload" from properly servicing existing
requests before old worker processes close sockets and exit. The
existing probtests missed this, but the new ones, in this patch, caught
it.
New probe tests cover both old SIGHUP "reload" behavior as well as the
new SIGUSR1 seamless reload behavior.
Change-Id: I3e5229d2fb04be67e53533ff65b0870038accbb7

...because otherwise, I think they might be running under py3 right now.
Change-Id: I96ad9e5ed87a7e08d06e889233730cec476d3af7

Change-Id: I5c856012a04ef8f910c28b0248ea7bcda47e4816

This patch adds a `pdf-docs` tox target that will build
PDF versions of our docs. As per the Train community goal:
 https://governance.openstack.org/tc/goals/selected/train/pdf-doc-generation.html
Add sphinxcontrib-svg2pdfconverter to doc/requirements.txt
to convert our SVGs.
Story: 2006122
Task: 35515
Change-Id: I26cefda80d3234df68d7152b404e0a71da74ab90

We were (indirectly) importing swiftclient (and therefore requests and
urllib3) before doing our eventlet monkey-patching. This would lead
boto3 (which digs an SSLContext out of urllib3) to trip RecursionErrors
on py3 similar to
 >>> from ssl import SSLContext, PROTOCOL_SSLv23
 >>> import eventlet
 >>> eventlet.monkey_patch(socket=True)
 >>> SSLContext(PROTOCOL_SSLv23).options |= 0
 Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/usr/lib/python3.6/ssl.py", line 465, in options
 super(SSLContext, SSLContext).options.__set__(self, value)
 File "/usr/lib/python3.6/ssl.py", line 465, in options
 super(SSLContext, SSLContext).options.__set__(self, value)
 File "/usr/lib/python3.6/ssl.py", line 465, in options
 super(SSLContext, SSLContext).options.__set__(self, value)
 [Previous line repeated 330 more times]
 RecursionError: maximum recursion depth exceeded while calling a Python object
Change-Id: I4bb59edd87336597791416c4f2a096efe0e72fe3

Change-Id: I9a754f1eb06debbe16800a05fc8e969af2f89043

Change-Id: I8c2517599a6d1cdf68050dc7acb039f09875fd53

test_bucket.py is proving somewhat problematic.
Change-Id: I5b337ef66a23fc989762801dd6a5ba1ed903f57b

Drive-by: fix an issue where tf.config wasn't initialized when we decide
whether to skip test_dlo_referer_on_segment_container.
Change-Id: Iad8b3ab40dccb23847cde5c09f55627da0963fe0

Change-Id: Ia9ae0fc226dfc9b40157faebac100c10a9180c62

Yes, it really is just the import.
Change-Id: Id7e2891b6600af102176eaa2e87f81b528adde8d

The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.
Note that the Swift API lives at /object-store and not /object-storage.
Note that redirects will be set up as well but let's point now to the
new location.
For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html
Change-Id: Ie38357e4c278335c35d186708573bb6bdabaa012

Drive-by: remove some py2/py3 code branches where encoding/decoding
should always succeed.
Change-Id: Iabfa157f2b20e6fd650a441e9c0f6163478ce147

Change-Id: I2929ce1f8e08ae5b4059605d09189c8521852423

Drive-by: Tighten domain_remap assertions on listings, which required
that we fix proxy pipeline placement. Add a note about it to the sample
config.
Change-Id: I41835148051294088a2c0fb4ed4e7a7b61273e5f

For more detail, see http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html
Change-Id: I95114c4aa670c07491d5a15db2341f65cb0d1344

This patch ports the tempurl middleware over to PY3. We use
an "all-native" string model, where we convert WSGI strings
as soon as we see them. It helps to deal with HMAC.
Aaand, we fix formpost along the way. It _clearly_ was
doing a wrong thing, encoding the same body several times
if we had several keys. On py2 it wasn't noticeable, but
on py3 this breaks, because the bytes type cannot be encoded
again.
Change-Id: I69974cc8a39731c980b54137b799a36b2e63a44a

We needed to change the sample translation a little, because
Python 3 requires encoding metadata.
We also remove workarounds for a specific bug in the subprocess
module, which were needed on Python 2.6 only.
Change-Id: Ie7791297cab16a649880088fa9eae6b4f95855f7

0.25.0 is now included in upper-constraints.txt
Related-Change: I04be480829eaf67ce328b2e7f7d7e2f70aaed90e
Change-Id: I5b8ecd82d6915d0e1cb216c423bbcc75c50a6f3d

All of the swift changes we needed for this were already done elsewhere.
Change-Id: Ib2c26fdf7bd36ed1cccd5dbd1fa208f912f4d8d5

Change-Id: I63a502be13f5dcda2a457d38f2fc5f1ca469d562

- Have the unit tests use WSGI strings, like a real system.
- Port the func tests.
Change-Id: I3a6f409208de45ebf9f55f7f59e4fe6ac6fbe163

This patch ports the symlink middleware to py3.
The middleware itself seems to be mostly fine and most changes
are in the symlink unit tests.
Change-Id: I973c2e1bb8969cf6bffece8ce68881c393efbaef

Change-Id: Ie73f9df460f9014014168d6e60767e66b24d30b0

This adds wsgi_to_str(self.path_info) everywhere we forgot it,
not only in the slo module itself.
Dropping the body=''.join(body) after call_slo() is obvious:
the latter only returns strings of bytes, not lists of such.
Change-Id: I6b4d87e4cda4945bc128dbc9c1edd39e736a59d2

Change-Id: Ib5aeda94fe9b165f54ad7412d68d71b62e427a40

Related-Change: I3603247e0c36299de2107aa8e494a3f87647696f
Change-Id: Iaf14c1ab3e236e6774492ed711fccef066a16fca

This finishes the proxy unit tests.
Change-Id: I2452e4e4dabef071bfbf376d03ec63211f7c2906

Change-Id: I8287db75b4f19581203360c646e72f64fe45f170

Note that you need a pretty recent eventlet to pick up
https://github.com/eventlet/eventlet/commit/f0bc79e
Change-Id: I6b006b972e7431c406039f4e0f6890a8f74a4432

Change-Id: I378705a8d9ad51dbfa85287d44e6313aa7680c23