openstack/swift - swift - OpenDev: Free Software Needs Free Tools

Change-Id: Iaf8d289c5ddb6ac778243a5e1fc27600bade6feb

Change-Id: I07278bc3314c26426ff0557fe2f5c69c02d6550c

doc.openstack.org -> docs.openstack.org
Change-Id: Iacadb8f70cb94adc002c562a1068ff38be19a9fc

The connection timeout to a memcache server is performed by using the
"with Timeout()" construct over the sock.connect() call in the
.create() method. In addition, the same construct was being applied to
the Pool.get() call in ._get_conns().
If the maximum number of connections was already created, and the
Pool.get() called took longer than the connect timeout, then the error
handling path would add a place holder to the connection
pool. Eventlet's Pool class allows for additional items to be added to
the pool, over and above the max_size setting. This additional place
holder will eventually be pulled and a new connection created to take
its place.
The fix is to remove the timeout construct in the _get_conns() method.
In addition, we also apply the unit test patch mentioned in the review
comments for Patch Set 6 of https://review.openstack.org/45134,
located at http://paste.openstack.org/show/47288/.
Fixes bug 1235027
Change-Id: I786cabefe3e8ddf7d92feb7ebc7cfb613d60a1da
Signed-off-by: Peter Portante <peter.portante@redhat.com>

This reverts commit 7760f41c3c
Change-Id: I95e57a2563784a8cd5e995cc826afeac0eadbe62
Signed-off-by: Peter Portante <peter.portante@redhat.com>

Mea culpa: these two scripts were missed in commit:
 https://review.openstack.org/46956
Fixes bug 1235441
Change-Id: I4303bc808448a79bddbb991526b0cca26150b392
Signed-off-by: Peter Portante <peter.portante@redhat.com>

Change-Id: I80e9a4c40ff99ec09a8eeef935447c6393ea78ec
Signed-off-by: Peter Portante <peter.portante@redhat.com>

Change-Id: Ic34bbd9cc65d96ea9b8434be7b54e5bcfae28b63
Signed-off-by: Peter Portante <peter.portante@redhat.com>

If a client were in the middle of an object GET request and then
disconnected, the proxy would wait a while (default 60s) and then time
out the connection. As part of the teardown for this, the proxy would
attempt to close the connection to the object server, then drain any
associated buffers. However, this didn't work particularly well,
resulting in the proxy reading the entire remainder of the object for
no gain.
Now, the proxy closes the connection hard, by calling .close() on the
underlying socket._socket object. This is different from calling
.close() on a socket._socketobject object, which is what you get back
from socket.socket() and similar methods. Calling .close() on a
socket._socketobject simply decrements a reference counter on the
socket._socket, which has been observed in the past to result in
socket leaks when something holds onto a reference. However, calling
.close() on a socket._socket actually closes the socket regardless of
who else has a reference to it.
I had to delete a test assertion that said the object server never got
SIGPIPE after a GET w/X-Newest. Well, you get a SIGPIPE when you write
to a closed socket, and now the proxy is actually closing the sockets
early, so now you *do* get a SIGPIPE.
closes-bug: 1174660
Note that this will cause a regression on bug 1037337; unfortunately,
the cure is worse than the disease, so out it goes.
Change-Id: I9c7a2e7fdb8b4232e53ea96f86b50e8d34c27221

The very same functionality for fewer keystrokes.
Change-Id: I8c9513412f398132db5636fdfb7bf8301e8443cf

The SAIO is purpously cut into two parts, so that you don't have to switch
back and forth between root and your unprivledged user. Add some "note" box
callouts to highlight this changeover.
Change-Id: I8b1a8f0539eac60d4121bdd4dab01df75ecca207

This has a couple benefits.
First, it means Ubuntu Precise users can just install python-dnspython
from packages instead of having to pull one in from source. This
should also fix an install error with new SAIO boxes where running
"python setup.py develop" fails unless a satisfactory dnspython is
already installed.
Second, it matches the dnspython dependency in the global
requirements. This means that the gates are already running with this
dnspython dependency, and it means our requirements.txt is one step
closer to being a subset of the global requirements.txt, which is
important to some people.
Change-Id: I5d58f488e1e4c4139c9fb20d89f386cab1537e98

This creates a pool to each memcache server so that connections will not
grow without bound. This also adds a proxy config
"max_memcache_connections" which can control how many connections are
available in the pool.
A side effect of the change is that we had to change the memcache calls
that used noreply, and instead wait for the result of the request.
Leaving with noreply could cause a race condition (specifically in
account auto create), due to one request calling `memcache.del(key)` and
then `memcache.get(key)` with a different pooled connection. If the
delete didn't complete fast enough, the get would return the old value
before it was deleted, and thus believe that the account was not
autocreated.
ClaysMindExploded
DocImpact
Change-Id: I350720b7bba29e1453894d3d4105ac1ea232595b

Change-Id: I95138852e45aa7632218a7107e0e7ba1f6ef373c
Signed-off-by: Peter Portante <peter.portante@redhat.com>

When reporting errors for SLO PUT requests, use the object path from the
manifest without the '/vrs/account' prefix. This is a continuation of
the same changes made for SLO/bulk delete requests in 6768d5b.
Change-Id: I40c90cccc1b7b5303d9f2b084dccb3be4f4448d8

The new_req Request object created for fetching
nested sub_slo manifests does not have its
path_info set to the sub_slo path before being
passed to GETorHEAD_base.
The sub manifest GET works fine because
GETorHEAD_base uses its path argument
(not req.path) to set up its connection,
BUT if an exception is thrown,
GETorHEAD_base uses req.path to construct
the log message - so the exception log would
have the wrong path.
Fixes bug 1231872
Change-Id: I0f0f969958f814d8a7053440d9de4e50796169e4

This a very small change which just tell users request url of bulk
delete request. In original docstrings, it just states the request
parameters, request body and request method but not request url.
Change-Id: I0bbc302a0e072910bb58e4814614d7f761433b10

Change-Id: Ia48224a1a187a8ed6b0c9a3c72cac06f084a6fc8

In 1.8.0 (Grizzly), your proxy logs would indicate which middleware
was responsible for an internal request, e.g. TU for tempurl or BD for
bulk delete. At some point, those all turned into GET_INFO, which does
not give you any idea which specific middleware was responsible, only
that it came from a get_account_info/get_container_info call.
This commit puts it back to how it was in 1.8.0. Also, the
new-since-1.8.0 function get_object_info() got swift_source plumbing
added to it, so source tracking for the quota middlewares'
get_object_info() calls will happen now too.
Note that due to the new-since-1.8.0 in-environment caching of
account/container info, you may not see as many lines in the proxy log
as you would with 1.8.0. This is because there are actually fewer
internal requests being made.
Change-Id: I2b2ff7823c612dc7ed7f268da979c4500bbbe911

If you don't, then newer versions of xattr won't install, and since
our xattr requirement is simply ">= 0.4" in requirements.txt, this
affects anyone setting up a new SAIO.
This happened with xattr 0.7, which was released on 2013年07月19日.
Change-Id: Iaf335fa25a2908953d1fd218158ebedf5d01cc27

Change-Id: I0a034ca1420761cbf4e35dcea1d9cd18a92f90bd

While doing some work around swiftclient Python 3 support, httplib
imports were changed to use six.moves in order to access httplib in Py2
and http.client in Py3. Tests related to those changes are failing
because these functional tests are depending on HTTPException being
a name in swiftclient.client. These tests should acces that name from
its canonical location in httplib (or, eventually, a Py3 friendly
location like six.moves).
Change-Id: I6335d465574045daedab47e5fe23c415d171a83a

Fixes object versioning when object name and/or version
container name contain multibyte utf-8 characters.
When object names containing non-ASCII characters
are PUT multiple times into a container with an
x-versions-location set, subsequent DELETE of the
object results in a 500 response status.
When the versions container name contains
non-ASCII characters the first delete of an object
succeeds but fails to restore previous version of
object, so second delete incorrectly returns 404.
Fixes bug 1229142
Change-Id: I425440f76b8328f8e119d390bfa4c7022181e89e

Change-Id: I2573be1ac14f65b8008611edf940363b31c8d86e
Signed-off-by: Peter Portante <peter.portante@redhat.com>

First: even if a user has exceeded their account quota, they should be
able to make object POST requests. Updating an object's metadata isn't
going to make them any more over quota, so should be allowed.
Second: don't bother with the reseller_admin check for container or
object requests. If I send the header X-Account-Meta-Quota-Bytes: 100
on e.g. an object PUT request, the proxy will (rightly) ignore it. Now
account-quotas does too.
Change-Id: I970a76349659acdd8229a324bd33bfe7fe7261a4