openstack/swift - swift - OpenDev: Free Software Needs Free Tools

Update to rocky release
Change-Id: Id35a41d689a24bf68d17647879780f5daaecdf37

Previously, we'd install development versions of Swift as root, causing
later tox runs as zuul to fail on a permissions error because the
generated egg-info (at least) was locked down.
Change-Id: Ia688790f8b23ed1cf76947b5809c208df5dee8bb

...and drop openstack-cover-jobs since it is no longer necessary.
Change-Id: Iad12e64f41bbdd8b8e005b0fb14a7e2108fa3bcf

Note that eventlet 0.22.0+ closes connections between requests when
it stops accepting connections.
Partial-Bug: #1792615
Change-Id: Ia8d9ab95e2aad40e8d797acc3423a917e809ffdb

This new test job adds multinode testing to swift.
It currently configures 5 nodes: test-runner, proxy,
account, container and object. It starts by checkout out
the latest release and then 'upgrades' the storage nodes
to the current patch under test. That means that it will
the test-runner and the proxy are still running the latest
release.
Future work would include a new set of tests where it will
add objects to the cluster prior to upgrading and then try
to read those objects after the upgrade. It should also be
noted that the 'upgrade' currently only means a new
'git checkout' the configuration files are not being updated.
A future patch should fix that too.
Change-Id: If8e09a082fc024257a98cf332de9a36a18d2adc5

Previously we'd use two users, one admin and one unprivileged.
Ceph's s3-tests, however, assume that both users should have access to
create buckets. Further, there are different errors that may be returned
depending on whether you are the *bucket* owner or not when using
s3_acl. So now we've got:
 test:tester1 (admin)
 test:tester2 (also admin)
 test:tester3 (unprivileged)
Change-Id: I0b67c53de3bcadc2c656d86131fca5f2c3114f14

This patch added new non-voting gate job to check the s3api compatibility
via swiftstack/s3compat tool that shows the ratio of compatible S3 APIs
in the gate result for each patch. This is very useful to check the possibility
if the new incoming patch breaks S3 API compatibility unexpectedly.
Originally swift3 has this kind of the gate job but we missed the staff
while migrating from swift3 into swift upstream repo so this is the porting of that.
Note that currently the job is against to only tempauth because we don't have
custom gate jobs using keystone environment other than dsvm.
Change-Id: I6f30f74678ad35479da237361bee48c46c0ecc49

Note that there's no swift-dsvm-functional-identity-v3-only because
as of the Queens release, that's the *only* way Keystone gets deployed.
Change-Id: I49d963b98f3df21fea0db24c83553ef873ad73c8

Latest version of CentOS should not need this anymore
Change-Id: Ief83c39420cfb4509f16d4e357d704025b116134

Change-Id: Ib08c2c4c685eb30a4c804ece0b667575c04d2ca0

This patch includes some generic playbooks to configure a SAIO[0]
plus the playbook to run the probe tests.
The current SAIO playbooks have only been tests on CentOS, they
would need additional changes to run on Ubuntu.
[0] - https://docs.openstack.org/swift/latest/development_saio.html
Change-Id: Ib72c77021aeed20f2070d42cff0ea9b671a4317a

Since I747c2b8754effbc6ec82af3bf7543fd9599a6c14 we do not install
the RDO package repository anymore and thus liberasurecode-devel
cannot be installed.
For CentOS 7, remove liberasurecode-devel from bindep.txt and install it
from test-setup.sh instead after enabling the RDO package repositories.
Update python dependencies: CentOS 7 does not have python3. Fix the
SUSE tags.
Change-Id: I72aa6b5455dfb025f54e83334983ac280f04afb2

Convert the legacy tox jobs to Zuul v3 native and use the
tools/test_setup.sh script to setup a XFS file like it's done in the
legacy job.
Needed-By: Id2b5cff998ac3a825a8f515c7bae3b433f30d272
Change-Id: I34ed9e1c4b822f700e42fb07937df7be72cbaf4e

Rename tools/pip-requires to requirements.txt and tools/test-requires
to test-requirements.txt. These are standard files, and tools in the
general world are growing intelligence about them.
Change-Id: Ib3e50a811868e2969923d978ee00c4f92682aa1c
Fixes: bug #1179008 

cname_lookup cannot resolve IP addresses as CNAME records, and therefore
should not attempt to resolve the host in that case. The middleware is
skipped when the host is an IP address.
Change-Id: I6961ec205e771116ace1ebcb8c088f3116eb38f0
Fixes: bug #1172289 

Change-Id: I3d4a31111c3044da06611405ce80f208ef8a0ce3

Change-Id: Ic1c67ced0169658c9aa7f78ed09bf24863cc9fb6

Using install_requires makes sure that if some code is using Swift internal
and wants to require Swift, all its dependencies have been already pulled.
This is really useful on test automation for example, otherwise, code might
be failing because on of the module listed in pip-requires is not instaled.
Also, we change the pip-requires files to use >= rather than ==, so the
requirements are easily fulfiled with distribution provided packages.
Change-Id: I65814bcd8ce798da21a5c17b4d5916a23f59e962
Signed-off-by: Julien Danjou <julien@danjou.info>

This required a bunch of whitespace-poking of the scripts in bin, but
that's all. Now every file in swift/ and bin/ is pep8-1.3.3-compliant,
so hopefully we can be done with this pep8 stuff for a good long time.
Change-Id: I44fdb41d219c57400a4c396ab7eb0ffa9dcd8db8

Using github as a package source is less reliable in the Jenkins test
framework than using pypi. This is because the Jenkins test framework
has a local pypi mirror.
Change-Id: Ia904d39ce70f5c2bd7d6edb9172f8226131074e2

One of the tests uses mock.call, which is only available in version
0.8.0 and up.
Fixes bug 1066274.
Change-Id: I77e4de0908f93326992fc4d4af4a7e54e60b5585

Change-Id: Ie566e712ba0d3dca543e99efbdb576b19623e73c

This change replaces WebOb with a mostly compatible local library,
swift.common.swob. Subtle changes to WebOb's API over the years have been a
huge headache. Swift doesn't even run on the current version.
There are a few incompatibilities to simplify the implementation/interface:
 * It only implements the header properties we use. More can be easily added.
 * Casts header values to str on assignment.
 * Response classes ("HTTPNotFound") are no longer subclasses, but partials
 on Response, so things like isinstance no longer work on them.
 * Unlike newer webob versions, will never return unicode objects.
Change-Id: I76617a0903ee2286b25a821b3c935c86ff95233f

This moves search_devs into RingBuilder to make it accessible to other utils
that need to search the builder. Along the same lines this also adds a
load() call to get a RingBuilder instance when working with the builder files.
- This adds python-mock >= 0.7 as a dependency for unittests. On Ubuntu
 10.04 you'll have to pip install it, on 12.04 you can apt-get install
 it. Fedora 17+ should be able to yum install it.
- new pep8 compliance
- Fixed a small issue (undefined var) in swift-ring-builder when remove was
called but failed to find a match.
Change-Id: I2e02684235aa2f4e901a00858ae037091594c545

Adding nosehtmloutput as a test dependency allows nose to output its
results to an html file. This will be used by Jenkins to save logs on
a different server.
Change-Id: I6bd4736e8cc4132db65bfe8cdfe1f3a388369fa8

Based on PatchSet 3 of https://review.openstack.org/#/c/7569/ , make them to pass all funcional tests with both webob 1.x and 1.2.
The additional following compatibility issues were addressed:
 - Until patch for range header issue is merged into official webob release, testRangedGetsWithLWSinHeader() should skip test against webob 1.2
(49c175aec2)
 - common.constraints.check_utf8() can accept both utf8 str and unicode.
 - To convert unicode to utf-8 str if necessary.
 - Making proxy_logging can handle invalid utf-8 str
bug 888371
bug 959881
blueprint webob-support
Change-Id: I00e5fd04cd1653259606a4ffdd4926db3c84c496

- It has been to its own gerrit project.
- direct_client should follow next.
- Implements blueprint clientbindings.
Change-Id: I3bb50c95eba81302bfec71cb7ce5288b85a41dc0

Need sphinx in test-requires so that we can build docs in
the venv (wanted because module introspection needs build
depends installed)
Change-Id: If3caf817381aea8e9cd8fb50e7e35c647f088b60

* Adds tox config
 - based on the config from python-quantumclient and updated for
 test, pep8 and coverage execution as per nova's run_tests.sh.
 * Adds nosetests defaults in setup.cfg
 * Adds runtime dependencies in tools/pip-requires
 - dependencies were gathered by referencing the packages used in
 creation of a Swift All In One. Versions were determined by
 checking the swift-core/trunk ppa or, failing that, the version
 available in lucid.
 * Adds test dependencies in tools/test-requires
 * Updates swift/common/middleware/formpost.py for pep8 compliance
 * Adds instructions for executing the tests with Tox to the
 developer_guidelines
 * Adds instructions for installing openstack.nose_plugin to
 developer_saio
 * Fixes bug 909177
Change-Id: I5407924d2181e9ab335aaf76bf30c8d40deccbb4

Change-Id: I297c3801991cbed2c360b3324d005ed003a1108d

Change-Id: If2af58cd4724b440d47385c10f202c9441cdf4bb