openstack/swift - swift - OpenDev: Free Software Needs Free Tools

Change-Id: I0dab4ff013f300396cff7313bc27b9d5644fe8a7

Note that the user DB is stored in-memory as native strings, so we do
some crazy-decoding to make comparisons for auth decisions. Seems to
keep the config handling mostly sane, though.
I maybe need to look harder at container ACLs?
Change-Id: Ia58698c9b30d2211eeee8ecb3bbdd1c26fa4034d

Change-Id: I4e69beac30be1d8726849aacb603af5e2aeecaf8

I wanna see how far I can get *without* mucking around in swob and
request_helpers. Maybe eventually we can get some helpers out there to
make working with UTF-8-pretending-to-be-Latin-1 strings better, but for
the time being, I feel more at ease *embracing* the crazy.
Change-Id: I0b9983a182daedd9dbec483b805d263238fcfac7
Co-Authored-By: Pete Zaitcev <zaitcev@kotori.zaitcev.us>

Change-Id: I4e2712a4a38b5cab3e31cc75bb162e73b4f5dafb

also a drive-by fix to the bindep job--it doesn't need to install
Change-Id: Ic9b68bc60bfbf21b45a1b7f9d7ff9998e01ddd26

Change-Id: I6be1a1c618e4b4fa03b34dad96f378aca01e8e08

Apparently this was already passing? Not a whole lot there...
Change-Id: I029fe2e7f4162765eee0a390793402523d0975fa

Change-Id: Ic6d0532f9f42b4d57a12aafeec5d34baffab6e23

I've at least tried it out with a py3 proxy, and it seems to work out
OK. I haven't tried killing the socket and verifying that it's actualy
dead, but getting a hold of _real_close *seems like* what we want?
At least the three (!!) tests pass.
Change-Id: Ic08c26185d63a36a5422793d81f621e0698fa572

Create a tox environment for running the unit tests against the lower
bounds of the dependencies.
Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.
Add openstack-tox-lower-constraints job to the zuul configuration.
See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.
Change-Id: Iac4a202a1d723822d10f4d34a397eff9de3f2108
add the job in project-config:
Depends-On: https://review.openstack.org/555034
remove branch optimiazation in check and add swift exclusions (series):
Depends-On: https://review.openstack.org/560109
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Co-Authored-By: Nguyen Hai <nguyentrihai93@gmail.com>

Yes, this means you can now start a proxy-server process on py3! And it
will give you something useful on /info!
Apparently you can even get auth tokens from tempauth, although the
tempauth unit tests don't pass yet.
Change-Id: I86ead2989b5934a7584cdd75719ce239826e01ec

The domain_remap middleware didn't need to change, only its tests.
Change-Id: I5edf05b609ce263708f851219c38c1bd1b645a6f

Change-Id: I83a35f066d69edf39c74fd22c7fb248b5320d21f

Typed "b" in a half-dozen places and fixed a typo in a test name.
Change-Id: Idcb8b2eeef5b545eb0944cc22b7c7961e8c2f8f0

Today's diff is brought to you by the letter "b".
Change-Id: I074c1ff3f1232a899c101f2856dff4a881a6fe77

Only the tests needed to change, and it was nearly all just prepending
"b" to a bunch of string literals.
Change-Id: Id5317e471c04a05627438c281bb0d6a97cf6a8a4

Change-Id: I31408f525ba9836f634a35581d4aee6fa2c9428f

Alternate approach to https://review.openstack.org/#/c/559902
Change-Id: I1e244c231753b8f4b6f1cf95cb0ae4c3c959ae0f

Ubuntu 18.04 ships 3.6.5; may as well have a testenv handy for it.
Change-Id: Ibea05bc879685fab148cde1b19510e83f0639f5d

See https://review.openstack.org/#/c/571007/ and
https://review.openstack.org/#/c/570999/ for context.
Change-Id: Ib6045de3cb9097ed7e6d48646109f35612cbbc74

This attempts to import openstack/swift3 package into swift upstream
repository, namespace. This is almost simple porting except following items.
1. Rename swift3 namespace to swift.common.middleware.s3api
1.1 Rename also some conflicted class names (e.g. Request/Response)
2. Port unittests to test/unit/s3api dir to be able to run on the gate.
3. Port functests to test/functional/s3api and setup in-process testing
4. Port docs to doc dir, then address the namespace change.
5. Use get_logger() instead of global logger instance
6. Avoid global conf instance
Ex. fix various minor issue on those steps (e.g. packages, dependencies,
 deprecated things)
The details and patch references in the work on feature/s3api are listed
at https://trello.com/b/ZloaZ23t/s3api (completed board)
Note that, because this is just a porting, no new feature is developed since
the last swift3 release, and in the future work, Swift upstream may continue
to work on remaining items for further improvements and the best compatibility
of Amazon S3. Please read the new docs for your deployment and keep track to
know what would be changed in the future releases.
Change-Id: Ib803ea89cfee9a53c429606149159dd136c036fd
Co-Authored-By: Thiago da Silva <thiago@redhat.com>
Co-Authored-By: Tim Burke <tim.burke@gmail.com>

There were a couple of cleanups in swob as part of this. First,
status lines are always native str objects (as PEP 3333 wants), rather
than being encoded to bytes under py3. Second, _resp_body_property
now works (only) with bytestrings from the app iter.
In gatekeeper, we now deal with dict.items() returning an object of
type "dict_items" in py3, not a list. Also fixed a NameError caused by
py2's list comprehensions leaking variables to function scope where
py3's don't.
Change-Id: I6da8eceb91edb2b47aa345d61b825c7199a5569b

Also a drive by HASH_PATH_PREFIX/SUFFIX bytesting fix which was
breaking ring and relinker tests.
Change-Id: I1db8f102361770cfed2b12f689789d1e8c21c3e4

This lets you run something like
 $ tox -e py35 -- -s test.unit.common.test_swob
and just run the swob tests. This is handy if you've got one or two
test failures to debug or if you're trying to get another test file
passing under Python 3.
Also removed the old py34 environment. Nothing uses it.
Change-Id: I244b8903ef0a445abd028004bf02d6c581a8afe2

For compliance with the Project Testing Interface as described in:
https://governance.openstack.org/tc/reference/project-testing-interface.html
For more details information, please refer to:
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html
http://lists.openstack.org/pipermail/openstack-dev/2018-March/128594.html
Co-Authored-By: Nguyen Hai <nguyentrihai93@gmail.com>
Change-Id: I26dc41c7df57bf79db531c6e67e148e01c17e992

This patch ports the form_signature cli `swift-form-signature`
and it's corrisponding tests to py3.
In essence in Py3 the HMAC function expects binary strings.
Change-Id: I5dded4ceb80f0cc595403775e8f9c17873e1e37b

Change-Id: I6bc70039c1b99b8a5bf5a30e9c6877d4f0aa638d

Bring under test
 - test/unit/cli/test_dispersion_report.py
 - test/unit/cli/test_info.py and
 - test/unit/cli/test_relinker.py
I've verified that swift-*-info (at least) behave reasonably under
py3, even swift-object-info when there's non-utf8 metadata on the
data/meta file.
Change-Id: Ifed4b8059337c395e56f5e9f8d939c34fe4ff8dd

Note that we're punting on configuring socket buffer sizes (for now)
Change-Id: I285a9b521fd0af381a227e0e824bc391817547f4

Change-Id: I7f04b3977971f0581b04180e5372686d8186346f

Change-Id: I7030280a8495628df9ed8edcc8abc31f901da72e

I can't imagine us *not* having a py3 proxy server at some point, and
that proxy server is going to need a ring.
While we're at it (and since they were so close anyway), port
* cli/ringbuilder.py and
* common/linkat.py
* common/daemon.py
Change-Id: Iec8d97e0ce925614a86b516c4c6ed82809d0ba9b

Since commit 1e79f828, we don't need to test with post_as_copy=True
any more since we haven't got post_as_copy at all.
Change-Id: I9c96ce0b812d877bbe11bdb50eb160d6ffa5933d

Change-Id: I966f849804cd5eeeae566bc7e732c7bf56bc9eaf

Add support for a 2+1 EC policy to be optionally used as default
policy when running in process functional tests.
The EC policy may be selected by setting the env var:
 SWIFT_TEST_IN_PROCESS_CONF_LOADER=ec tox
when running .functests, or by using the new tox test env:
 tox -e func-ec
Change-Id: I02e3553a74a024efdab91dcd609ac1cf4e4f3208

Following up on patch 417209.
Fast post is now the default mode, so there's
no more need for a fast-post specific testenv.
Change-Id: I6faf8fcfa0a1d96aaf0f5e0ad2106b2b416da22f
Signed-off-by: Thiago da Silva <thiago@redhat.com>

Additionally, emit deprecation warnings when running POST-as-COPY
Change-Id: I11324e711057f7332577fd38f9bff82bdc6aac90

Currently swift ignores a lot of the Hacking style guide. This patch
enables the H401 and H403 checks and fixes any violations. With this
we can get a little closer to following the OpenStack style guidelines.
Change-Id: I5109a052f2ceb2e6a9a174cded62f4231156d39b

The bandit gate is passing, let's make it voting.
Since bandit is a security linter, add it to the linter target - called
pep8 - and run it there. This will save the CI systems spawning an extra
test machine for this.
Needed-By: I90e5401ee8e46a2a153adb62d0f99a8a88a686b8
Change-Id: I75eeefa2441466da60b9b206769a2e64e9c1e7a9

Change default py3x to py35 in tox.ini to support python3.5,
and fix the error when run tox test for Swfit.
Closes-Bug: #1649773
Change-Id: I8e26183ffe270518503da8b5aa394e99ae4cd8c6

Fixes: NameError: name 'file' is not defined
file() is removed in python3.x using open()
 will solve it.
Fixes: AttributeError: 'generator' object has no
attribute 'next'
Python3 it is __next__()
Change-Id: Ib2cbdfbbedb86f6e01239d2ee991677727b0548b