openstack/swift - swift - OpenDev: Free Software Needs Free Tools

Give users a sample rsyslog.conf to manage their logs easily based on
LOCAL0 swift used. In this patch I offer some choices to output logs
via commnet lines or uncomment.
Change-Id: I2fe150a6e3d164a989c3520c0b7f032897a71f18

This will allow users to more easily match the failed objects in Errors
for bulk delete requests to the object names they provided in the request.
For extract Errors, it reports the failed path from the tar archive.
DocImpact
Change-Id: I084057810fc4fb7fdac05494cc6fec2cbf81bb9d

Fixes bug 1180680.
Change-Id: I1c0eaa6f934f73853e832a8e468e1b8369a21117

The proxy can now be configured to prefer local object servers for PUT
requests, where "local" is governed by the "write_affinity". The
"write_affinity_node_count" setting controls how many local object
servers to try before giving up and going on to remote ones.
I chose to simply re-order the object servers instead of filtering out
nonlocal ones so that, if all of the local ones are down, clients can
still get successful responses (just slower).
The goal is to trade availability for throughput. By writing to local
object servers across fast LAN links, clients get better throughput
than if the object servers were far away over slow WAN links. The
downside, of course, is that data availability (not durability) may
suffer when drives fail.
The default configuration has no write affinity in it, so the default
behavior is unchanged.
Added some words about these settings to the admin guide.
DocImpact
Change-Id: I09a0bd00524544ff627a3bccdcdc48f40720a86e

- Added simple unit tests for direct client
 - Implemented chunked put in direct_put_object
Change-Id: Icec1019b5157fae33e2fd36a7fc3b9c0cc15afec

As of e499b91, these config values and functions are unused in StaticWeb.
As of 2e155e5, the comment in TempURL is false.
Change-Id: I75b631ece8a9a6075c406765361629c549c449f6

This patch is needed in 1.9 as a dependency for adding swift/apache
to devstack.
zhang-hare is working in parallel on a devstack patch that depend
on this patch.
As part of this patch we add examples dir.
In this dir we add template config files that can be used either by
a script (as will be done by devstack) or serve a an example to the
admin.
Apache2 vhost files were added under examples/apache2
Wsgi files were added under examples/wsgi
Fixes: Bug #1191388
Change-Id: If90e38a0ba36f6b59016d4a1374f4100f0bbee7b

Ignore build/ directory, which is auto-created when running `python
setup.py bdist` like dist/ directory.
Change-Id: Ie58936320d995e4e81f8929128475e538665bde8

Update staticweb to use get_container_info instead of its own memcache entry.
Change-Id: I59bc0d94b9e45f3d5776a5462f53b82ab78101d4

Change-Id: Ia0a1b36ae90ed895dcc9876c016aaf261ad56674

So, there's two cases here. In one case, we have memcache enabled;
account info gets cached in the usual place, and tempurl keys get
cached separately. In the other case, we have no cache. In neither
case is anything gained by having TempURL cache keys separately from
the account info.
This commit removes TempURL's custom caching logic and makes it rely
on get_account_info() instead.
Benefits include:
 * immediate visibility of new keys on account POST
 * less data in the cache per account --> more stuff fits in cache
 * less code for bugs to hide in
Change-Id: Idb0b6c165da14196b4c79149c546f0159b54edcb

A really simple version of this was in container sync already, and I
needed a more complete version for work I'm doing, and I noticed
https://review.openstack.org/#/c/33405/ was also making use of it.
So, here's a more full version.
If https://review.openstack.org/#/c/33405/ lands before this, I'll
update it accordingly.
Change-Id: Iba66b6a97f65e312e04fdba273e8f4ad1d3e1594

1. Add explanation of MAX_HEADER_SIZE into swift.conf-sample as same as
other settings in swift.conf. Especially point out the default size of
header line in eventlet is 8192 which is the main reason why we set 8192
for MAX_HEADER_SIZE in swift.
2. Add some unit tests to check valid settings in swift.conf. Test cases
in test_constraints use /etc/swift/swift.conf if exists, and if any
wrong settings are in it (MAX_META_VALE > MAX_META_OVERALL_SIZE), swift's
unit test must fail. These new unit tests is used in this case.
Change-Id: I7bb21951d46050163c1b7bceac8d49302b9209f7

Change-Id: Ieceed3837466c45bc2790b09cd56ae95f43dba2e

Allows to run unit tests on BSD-based platforms. The commit
da99c339 that solved the problem doesn't contain fixes of unit
tests.
Change-Id: Iff1771c1cbb37c30a888cd1da4fca5636e5797e7

subdir elements now have their names quoteattr
Change-Id: I37faf164e339deacb6555db94196135debe22a6c

Change-Id: I91709c3fc6e937fdb31b418ac98fe5a67265cea7

Commit 8f9b135 fixed a bug where an XML attribute could have arbitrary
characters jammed into it, resulting in a document with arbitrary
tags... and it did remove the ability to get an arbitrary XML document
out of the object server. However, it left a couple of ways to get a
malformed XML document, one example of which is an account named ".
This fixes up the remaining ways and ensures you always get a
well-formed XML document in the account-listing response. Also, it
adds a unit test for the escaping of the container name; this was
already working, just untested.
If you look in the discussion for bug 1183884, you'll see that the
review comments there are basically "seems good, but could use a unit
test". (The astute reader will note that I am one of the guilty
parties in that review.)
I found this bug while writing the missing unit test.
The moral of this story is left as an exercise for the reader.
Fixes bug 1183884 harder.
Change-Id: I84b24dd930ba1bb6c4f674f8d3996639dedbce15

Lucid won't EOL until May of 2014; but I stopped trusting that ppa a long time
ago. Besides with the requires for dnspython and mock where they're at you
almost can't install swift from source on any stock distro and expect tests to
pass with system packages - so we're looking at pypi for depends regardless.
While I'm in there:
* more explanation of <your-user-name> and a helpful find/sed for configs
* group the "setup ~/.bashrc" stuff with the "setup ~/bin" stuff
* some updates/fixes from my experience installing on CentOS
* remove region warnings from remakerings
Change-Id: Ie2e6b06959ab699d853e07e5b7e8cda7036a44fe

Fixes bug 1183884.
* swift/account/utils.py: Escape account name in XML listings.
Change-Id: I2392d012ddeec05a267c3dcf14748112316096f3

Added logging of the x-delete-at value for object PUTs and POSTs and
made the additional log field generic for other similar use cases.
Logging the x-delete-at is useful because otherwise you might never
know why an object "disappeared" just by looking at saved proxy logs.
You'd see a successful PUT of the object followed some time by a GET
that 404s. If you still have the other logs, you could track down
what happened; but, at least with our clusters, we keep proxy logs
forever but only keep the other logs for a short while.
Change-Id: Ida559abcf8dc5666e2a7eea250a67d1c4b28b1be

Below three bug reports talk about one thing.
Current keystoneauth ACL supports as:
tenant_name:user_id ok
tenant_name:user_name no
tenant_name:* no
tenant_id:user_id ok
tenant_id:user_name no
tenant_id:* no
*:user_id ok
*:user_name no
*:* no
This patch will make all of above work fine.
Applying (tenant/user)name could let user put or get their data in a
more readable way. The tenant_name:* and *:user_name is suitable for
many usage.
note: to keep compatibility here add a new keystone.identity just for
authorize() itself and leave env['keystone.identity'] to other
middlerwares.
Fixes: bug #1020709
Fixes: bug #1075362
Fixes: bug #1155389
Change-Id: I9354dedaad875117f6a9072c67e9ecf69bfca77e

improving points:
1. Remove yum install swift in Fedora; Use installing from source for
both Ubuntu and Fedora.
2. Explain you could use all users including root, your own guest. An
d the points developer have to care.
Change-Id: Id6d683441bd790a21734624e29eb7c98bb40de85
Fixes: bug #1126389