openstack/swift - swift - OpenDev: Free Software Needs Free Tools

RFC2616 implicitly allows transferring utf-8 in
headers (vals) but not in header names (keys)
Swift functional tests includes testing for utf-8
header names (keys) which is non aligned with the standard.
This seem to be the case when testing file metadata.
File metadata keys are than transferred as part of
the http header keys (header names) as
 X-Object-Meta-<metadata_key>=metadata_name.
As a result metadata_keys in swift must be composed from
a subset of ASCII characters - alpha numerics,
some punctuation, not a lot else* and cannot be utf-8.
On branch bp/wsgi-application-interface-4
modified: test/functional/tests.py
Change-Id: I1932e9d4ead83089e8a31a06d28a54f5625efb9c
Fixes: bug 1068026
Implements: blueprint wsgi-application-interface

The directory tests TestContainerPathsEnv creates directorys during SetUp while indicating duplicate application internet media type (MIME) via the Content-Type header.
E.g the output on the line using 'nosetests -v test.functional.tests:TestContainerPaths.testContainerListing' includes:
Host: 127.0.0.1:8080
Accept-Encoding: identity
Content-Type: application/octet-stream
Content-Length: 0
content-type: application/directory
X-Auth-Token: AUTH_tkd40f5550289d4d60af75bb1b6c6067be
On branch bp/wsgi-application-interface-3
 modified: test/functional/tests.py
Fixes: Bug #1068499
Implements Blueprint 	wsgi-application-interface
Change-Id: I49c5d1f73fff3c06d4be18b20c7209eff4e1326c

test_client sends a wrong size indication when testing
chunked data.
For example instead of sending:
 5\r\nabcde\r\n (which is inline with the standard)
The client would send:
 0x5\r\nabcde\r\n
On branch bp/wsgi-application-interface-2
 modified: test/functional/swift_test_client.py
Change-Id: I7597dcd24adade009aead66f67674f361f3fe4fa
Fixes: Bug #1070036
Implements: Blueprint wsgi-application-interface

This patch merely fixes a selection of files to the point where
pep8 1.3.3 is happy. Most of the errors are indentation related to
continued lines (E126, E127, E128), bracket positions (E124) and the
use of backslash (E502).
Patch 2 fixes David's comments regarding backslash and an odd comment
 - thanks David!
Change-Id: I4fbd77ecf5395743cb96acb95fa946c322c16560

This fixes swob to handle "Range: bytes=-X" where X > len(content); ex.
"Range: bytes=-17" when the object has 10 bytes. Based on the RFC, the
range is satisfiable and all bytes should be returned.
It also fixes "Range: bytes=-0" to be, correctly, not satisfiable. In
addition, this case's response has Content-Length: 0 and has a zero-byte
body.
It also fixes an existing regression in swob for the case
"Range: bytes=100-" for a body of length < 100 (Content-Length was
negative and the body was returned).
The relevant RFC is 2616, section 14.35.1.
Change-Id: Ib3dc672e083173eb970c10801283813623f26e0e

This change replaces WebOb with a mostly compatible local library,
swift.common.swob. Subtle changes to WebOb's API over the years have been a
huge headache. Swift doesn't even run on the current version.
There are a few incompatibilities to simplify the implementation/interface:
 * It only implements the header properties we use. More can be easily added.
 * Casts header values to str on assignment.
 * Response classes ("HTTPNotFound") are no longer subclasses, but partials
 on Response, so things like isinstance no longer work on them.
 * Unlike newer webob versions, will never return unicode objects.
Change-Id: I76617a0903ee2286b25a821b3c935c86ff95233f

Change previously hard-coded constants into config variables. This
allows deployers to tune their cluster more specifically based on
their needs. For example, a deployment that uses direct swift access
for public content may need to set a larger header value constraint to
allow for the full object name to be represented in the Content-
Disposition header (which browsers check to determine the name of a
downloaded object).
The new settings are set in the [swift-constraints] section of
/etc/swift/swift.conf. Comments were also added to this config file.
Cleaned up swift/common/constraints.py to pass pep8 1.3.3
Funtional tests now require constraints to be defined in /etc/test.conf or in
/etc/swift/swift.conf (in the case of running the functional tests against a
local swift cluster). To have any hope of tests passing, the defined
constraints must match the constraints on the tested cluster.
Removed a ton of "magic numbers" in both unit and functional tests.
Change-Id: Ie4588e052fd158314ddca6cd8fca9bc793311465

fix some format according to pep8
Change-Id: I8ceb266d92c8997f131fa4096aaa6cfd4d585aaa

fir some format according to pep8
Change-Id: Iec8f67ea2166c955732bd598661140d4cfce6d40

Based on PatchSet 3 of https://review.openstack.org/#/c/7569/ , make them to pass all funcional tests with both webob 1.x and 1.2.
The additional following compatibility issues were addressed:
 - Until patch for range header issue is merged into official webob release, testRangedGetsWithLWSinHeader() should skip test against webob 1.2
(49c175aec2)
 - common.constraints.check_utf8() can accept both utf8 str and unicode.
 - To convert unicode to utf-8 str if necessary.
 - Making proxy_logging can handle invalid utf-8 str
bug 888371
bug 959881
blueprint webob-support
Change-Id: I00e5fd04cd1653259606a4ffdd4926db3c84c496

bug 701248
bug 819303
This change makes syslog optional, by providing a unit test
config with the ability to cause test/unit/__init__.py to
replace SysLogHandler with a fake logging class. The
default behavior is unchanged.
FakeLogger now inherits directly from Handler and mocks out
its API - this was a backward-compatibility issue.
Change-Id: I653dec148c4f6cf81759de03964c6a3290c1a290

Change-Id: I78dbb390cf0afe03d4bcf1aa9ffc9497392579dd

Change-Id: Ifd909d3561c2647770a7e0caa3cd91acd1b4f298

The None, True, and False values are singletons.
All variable *comparisons* to singletons should use 'is' or 'is not'.
All variable *evaluations* to boolean should use 'if' or 'if not'.
All Object type comparisons should use isinstance()
instead of comparing types directly.
Change-Id: I47863c4862791022670619f19b8bc15d8a93fd81

fixes lp bug 857673
also fixes similar bug in post-as-copy code
adds test
Change-Id: I7a1984506bba28afe2b7ba46998d7306c058fd5c

SWIFT_TEST_CONFIG_FILE can redirect both functional test suites to any conf
functional tests that don't require a account:username can now work with only
username for auth systems that don't support the account/namespace concept