openstack/swift - swift - OpenDev: Free Software Needs Free Tools

It's not sufficient to just look at swift.object-updater.successes to
see the async_pending unlink rate. There are two different spots where
unlinks happen: one when an async_pending has been successfully
processed, and another when the updater notices multiple
async_pendings for the same object. Both events are now tracked under
the same name: swift.object-updater.unlinks.
FakeLogger has now sprouted a couple of convenience methods for
testing logged metrics.
Fixed pep8 1.3.3's complaints in the files this diff touches.
Also: bonus speling and, grammar fixes in the admin guide.
Change-Id: I8c1493784adbe24ba2b5512615e87669b3d94505

The directory tests TestContainerPathsEnv creates directorys during SetUp while indicating duplicate application internet media type (MIME) via the Content-Type header.
E.g the output on the line using 'nosetests -v test.functional.tests:TestContainerPaths.testContainerListing' includes:
Host: 127.0.0.1:8080
Accept-Encoding: identity
Content-Type: application/octet-stream
Content-Length: 0
content-type: application/directory
X-Auth-Token: AUTH_tkd40f5550289d4d60af75bb1b6c6067be
On branch bp/wsgi-application-interface-3
 modified: test/functional/tests.py
Fixes: Bug #1068499
Implements Blueprint 	wsgi-application-interface
Change-Id: I49c5d1f73fff3c06d4be18b20c7209eff4e1326c

Add trove classifier to have swift listed among the
other OpenStack-related projets on PyPI to reserve
the name, even though we won't release packages there.
Change-Id: I2c6cd2e37ea55d2bc2ddbba813ee2a190b1c90a9
Signed-off-by: Doug Hellmann <doug.hellmann@dreamhost.com>

test_client sends a wrong size indication when testing
chunked data.
For example instead of sending:
 5\r\nabcde\r\n (which is inline with the standard)
The client would send:
 0x5\r\nabcde\r\n
On branch bp/wsgi-application-interface-2
 modified: test/functional/swift_test_client.py
Change-Id: I7597dcd24adade009aead66f67674f361f3fe4fa
Fixes: Bug #1070036
Implements: Blueprint wsgi-application-interface

Add path_qs property to swob.Request. First of all this property
has webob.Request, also this property can be used in swift3 middleware
for generate canonical string, if webob will be replaced in swift3 with
swob (2b36fbd477).
Change-Id: Idf58096baaf7830dd0d624ea6c72eda1eb91ff0d

Turns out this prioritization causes stampedes right after capacity
additions (especially on smaller clusters), which means the things it
prioritizes don't get done for several passes anyway, which means the
prioritization isn't solving the problem it was designed to do, and
it causes non-productive load spikes instead. :/
Getting rid of the prioritization doesn't make things go any slower
overall, could make things actually go faster since there are less
collisions, and evens out the load.
Change-Id: I4938bcd12d9a21fad635cfe772897e4c151f8a95

Change-Id: Ia430637f24d883ee166cb6529dde03049cc06120

In general, Linux does not care about what a mount point is named as
long as it is a vaild directory name (no / or null characters).
However, that is too relaxed for swift, which will pass that mount
point name around as part of url path construction all over the
place. To make sure that the mount point name was sane from that POV,
Swift was using isalnum to verify that the mount point name was sane,
which is overly restrictive.
This patch replaces that test with a test that verifies that the name
has no characters that need to be URL encoded.
The specific use case this enables is allowing mount points to be
named according to the UUID of the filesystem that is being mounted,
which will make Swift more robust in the face of device name instability.
Change-Id: I4d49b21c1783e97c16d3f394c2171f1f80eea058

There have been a bunch of Jenkins failures lately where the StatsD
tests fail because they can't bind to their desired port. There's
nothing special about the particular port they're using, so now we let
the kernel pick an available one for us.
This also lets us get rid of a sleep() in the test that looked like an
attempt to alleviate EADDRINUSE errors, so now in the happy case, the
tests are a few fractions of a second faster.
Change-Id: Idee11349254107a59643539b1566f3588eee7ef4

Change-Id: I38e38607ba9c5cf331c216e4c81582d23c58eeca

One of the tests uses mock.call, which is only available in version
0.8.0 and up.
Fixes bug 1066274.
Change-Id: I77e4de0908f93326992fc4d4af4a7e54e60b5585

This patch merely fixes a selection of files to the point where
pep8 1.3.3 is happy. Most of the errors are indentation related to
continued lines (E126, E127, E128), bracket positions (E124) and the
use of backslash (E502).
Patch 2 fixes David's comments regarding backslash and an odd comment
 - thanks David!
Change-Id: I4fbd77ecf5395743cb96acb95fa946c322c16560

It's there to let administrators turn down the barrage of stats data
that StatsD must cope with, but it wasn't actually honored. Worse, if
the sample rate was set to e.g. 0.2, the stats would all be multiplied
by its inverse, e.g. 2. This patch actually drops packets when
sample_rate < 1, so you get correct measurements.
Fortunately, the default sample rate is 1 (i.e. drop nothing), and
multiplying by 1/1 doesn't change anything, so stats with the default
sample rate of 1.0 are, and have been, just fine.
Fixes bug 1065643.
Also, make the two touched files compliant with pep8 v1.3.3.
Change-Id: I66663144009ae4c9ee96f6a111745d8f5d2f5ca3

Change-Id: I2c1246b9bbd1d3ab22c2a035b735d937dd90da11

Change-Id: Ifd609f305ee878c39ea4e6840ed8fa9369595cfa

_get_metadata() should only look for metadata on 200 family http status codes
in the path_parts() in the internal client test file should return distinct unicode strings
with spaces for account, container, and obj
Change-Id: I74e335d3b73a59092a0eec06539f4931420061dd

fixes bug 1049017
As per the bug report, the -remove- form of deletion only works with
 meta headers and not with x-container-read and x-container-write.
This patch by Yukihiro KAWADA detects the container acls and sends
them through to the backend empty like the other metadata values.
patch2 extends metadata-helper in ContainerController tests so that
the new functionality can be tested.
patch3 changes the k.lower().startswith() commands for read/write to
a single k.lower - thanks David :)
patch4 fixes PEP8
patch5 fixes more than two hundred pep8 errors. Except one,
 where the pep8 tool complains of overindent, or underindent
depending on the position on the line. No idea how to fix that one.
patch6 fixes the remaining pep8 error - thanks Darrell
Change-Id: I36c2dd72b4636136c5ce7db414bf2b61501090ad

Change-Id: Ic263f4a77a0aa0eb40078772a567eb41a60e40f7

Fixes bug 1064249
sys.path should be extended not appended to
Change-Id: I3404a50b16b843b47f5082676029a4dd3b03f812

Change-Id: I5efb3712faa7d159be2d86913fa5fbdf09b7f98f

Change-Id: Ie566e712ba0d3dca543e99efbdb576b19623e73c

This fixes swob to handle "Range: bytes=-X" where X > len(content); ex.
"Range: bytes=-17" when the object has 10 bytes. Based on the RFC, the
range is satisfiable and all bytes should be returned.
It also fixes "Range: bytes=-0" to be, correctly, not satisfiable. In
addition, this case's response has Content-Length: 0 and has a zero-byte
body.
It also fixes an existing regression in swob for the case
"Range: bytes=100-" for a body of length < 100 (Content-Length was
negative and the body was returned).
The relevant RFC is 2616, section 14.35.1.
Change-Id: Ib3dc672e083173eb970c10801283813623f26e0e

Add a hook to get_logger to run custom functions to add custom log
handlers or the like.
Change-Id: Ib04b12939dcac7e4ad6453dea9795682044c6ae0

made internal client handle unicode path parts by adding make_path method
fixed pep8 problems in internal client and its test
moved internal client unit test file to correct directory
Change-Id: Id1c81c9cb0db05342e4e8a8393db93552fda4647

in swift.obj.server.HEAD
Change-Id: Ie7cf853319589db7c48836dd5d7fd5e68cb009cd

Change-Id: I84dd7bc2fc5ebd1190a7184f42fc5ecf5fd7b4d3

Have swift-init warn when the running user doesn't have permissions to
signal processes. Fixes bug 1017494.
Change-Id: Icb9048ab36f1ca73bb93b11c9c2aed882d99dfa7

Change-Id: I70ee55d12b1ca91fdfe4891c3f82ab0f832f15d3

This patch introduces a new user format to tempauth which starts with
"user64_" and in which the account and user are base64 encoded (sans
padding equal signs):
user64_<account_b64>_<user_b64> = <key> [group] [group] [...] [storage_url]
This allows unambiguous parsing and adds the ability to have a user or
account name containing underscores.
The handling of tempauth configuration is fully backward compatible and
this only adds new, optional functionality.
Also beefed up unit tests to get full coverage of the user parsing code
in tempauth's __init__().
Change-Id: Id06304934ab8055a28921f2e1aa3c58a2036d8f8