openstack/swift - swift - OpenDev: Free Software Needs Free Tools

Multiple Server Swift Installation (Ubuntu) instruction does not
indicate that the directory /var/run/swift needs to be created.
That directory actually needs to be created and the ownship needs
to be changed to the user/group which swift service runs under.
This patch will fix the document and gives the steps how to create
the directory and set the ownership right. It also gives instruction
on how the script can be added so that swift services can be resarted
after system reboots.
Change-Id: Id61aa67cc0d6f66d749701e6ea824b1ff3b2c478
Fixes: bug #1156631 

This prevents people from creating bogus ring builder files.
Example: "swift-ring-builder object.builder create 33 0.9 -4".
Fixes bug 924577.
Change-Id: I7bfc04f7fa5f55f70a4eaae96c414f6b2872e283

swift-ring-builder uses outdated unbounded class method to handle
not supported command. It worked for python 2.6 or older but not
for python 2.7 or newer. This patch fixes the problem.
Change-Id: I7dbc681ef6be44f6d79ff93189ccca13c51eab74
Fixes: bug #1154882 

Apparently the use of port 6030 was causing some confusion.
Fixes bug 1095474.
Change-Id: I0cc71f4733ad91694e015a9b75c3eda080aca6fb

Using root logger will make public module log failed by unknow keyword.
Just change the logger name to module itself could dismiss this impact.
And disable logger.propagate could prevent double outputing.
Change-Id: I18696d124ebac9ca970d502558972e51de759097
Fixes: bug #1105133 

Change-Id: Icef35175682c11a406ef85accc6c6e1acbf89f4d

If you run list_parts against a builder that has never been
rebalanced, you'd get a crash. Now you don't.
To reproduce:
$ swift-ring-builder foo.builder create 8 3 1
$ swift-ring-builder foo.builder add r1z1-1.2.3.4:6000/sda 100
$ swift-ring-builder foo.builder list_parts z1
Change-Id: Ic3edffab0c5c2e9551a2f89ddb881153f0b07db7

The only place in the spec that I could see the Origin being checked was
during the pre-flight OPTIONS request. If it gets to the actual request
let auth decide. Please correct me if this is wrong.
Change-Id: Ic31b71746ec056091c7778ebff3db7becc32bd9c

I broke it in 7548cb9 when a ring's replica count changed from an int
to a float.
Change-Id: I49fa7e2961a09daad0c96e19cf1e39fec248d998

The region is one level above the zone; it is intended to represent a
chunk of machines that is distant from others with respect to
bandwidth and latency.
Old rings will default to having all their devices in region 1. Since
everything is in the same region by default, the ring builder will
simply distribute across zones as it did before, so your partition
assignment won't move because of this change. If you start adding
devices in other regions, of course, the assignment will change to
take that into account.
swift-ring-builder still accepts the same syntax as before, but will
default added devices to region 1 if no region is specified.
Examples:
$ swift-ring-builder foo.builder add r2z1-1.2.3.4:555/sda
$ swift-ring-builder foo.builder add r1z3-1.2.3.4:555/sda
$ swift-ring-builder foo.builder add z3-1.2.3.4:555/sda
Also, some updates to ring-overview doc.
Change-Id: Ifefbb839cdcf033e6c9201fadca95224c7303a29

Fixes MANIFEST.in to include missing CONTRIBUTING.md into source
code tarballs.
Change-Id: I81876adad14e0f1b2211d39ec6471e35f5464120

There was a bug where account_info wasn't converting the
container_count value to an int. Causes max container count cap to
get hit pretty quick since '0' > 0.
Change-Id: Ibfc6eebbff5a00aaebb47e9731dd053b60e3caa4

While poking at such problems with authentication other than swauth
it was useful to allow stock functional tests running back-to-back
at two clusters, with tempauth and Keystone.
When running the legacy tests, this version packs account into user
the way it was down previously. This way we do not need to repack
them before every call to get_auth. The downside is additional work
to be done when setting ACLs.
Change-Id: Ieb1d9227cb88977ecd2c39825039bc4be8afae0b

The recent account_quotas (https://review.openstack.org/23434)
patch added a new setting request.environ[reseller_request].
This patch adds tests for tempauth and keystoneauth as well as
an updated overview_auth.rst.
Change-Id: Icdb7ec9948ae7424b0721fc51a143782b2fdc5a6

Hard coding of '/srv/node' maybe drive to some errors. But it seems
unfixable (please see talks in bug report). To avoid erros as we can,
showing a friendly notification to let user set their own device
location works better than hard coding directly.
Change-Id: I45d749dd52af8d3b051dfe98dbb9d71a5a374f97
Fixes: bug #907908 

container-sync now skips faulty objects in the first and second rounds.
All replicas try in the second round.
No server will give up until the faulty object suceeds
Fixes: bug #1068423
Change-Id: I0defc174b2ce3796a6acf410a2d2eae138e8193d

Sometimes there is no date at the beginning of a line in kern.log.
Although it does not happen often, there should be a check ensuring
the program doesn't crash in case it happens.
Added try-except block surrounding parsing string to date format.
Change-Id: I44a101266582eea2199189a006afa1037a9bd4ea
Fixes: bug #1152658 

Add a new middleware implementing account quotas.
This middleware blocks write requests (PUT, POST) if a given quota (in bytes)
is exceeded while DELETE requests are still allowed.
Quotas are stored in the x-account-meta-quota-bytes metadata entry.
Write requests to this metadata setting are only allowed for resellers.
Change-Id: I57fd7c6209f34cc79d4bab72d500d43ba2a62083

I thought it was readable back when I wrote it; turns out it's
not. Oops.
Change-Id: I1e1020935356522d6e07409aa867a5ffc8919787
Bonus: remove unused import.

Replace all that map(operator) nonsense.
It changes the error raised on invalid hashes, but we don't handle that
anywhere, and it shouldn't ever happen in real life.
Change-Id: Ib8cb549fac05e0b2725b4ea295326ac0c5e1f035

Add support for functional tests that work with Apache web front end
Change-Id: I72358a12016eeccc842d834461dbebaa188aa117
Implements: blueprint wsgi-application-interface

Implements blueprint list-endpoints.
DocImpact: new middleware list_endpoints.
Change-Id: I0c4911ff726abd4cb8ce2b6245c99786ad46b410

Change-Id: Ied6ed3cad16e9797df73a05f3df3ac9cc64299e0

Before, you'd get your 3* primary nodes in 3 different zones, and then
get_more_nodes would give you everything it could from a non-primary
zone, and then finish up with stuff from the primary zones. It would
sort of look like this:
P: device in a primary node's zone
N: device not in a primary node's zone
PPPNNNNNNNNNNNNNNNNNNN...NNNNNNNNNPPP...PPPPPP
(The first three Ps are the primary nodes; they don't actually come
out of get_more_nodes(), but they're included for clarity.)
Now, the first few handoffs from get_more_nodes are in non-primary
zones, but only one per zone, and then the rest of the handoffs ignore
zones. It's still sampling the ring, so it's still taking weights into
consideration, but the zone distribution is more even early in the
handoff chain. It looks like this, assuming 10 zones:
P: device in a primary node's zone
N: device not in a primary node's zone
D: zone doesn't matter
PPPNNNNNNNDDDDDDDDDDD...DDD
* or whatever your replica count is
Change-Id: I31d2a2bc2cd6038386a2df85cd4fa37ccf2f650e

Change-Id: Ic3e9ad50f123eecb4abbe91340b0acff4a2d580c

The handoff nodes will try to be in zones other than the primary
zones, will take into account the device weights, and will usually
keep the same sequences of handoffs even with ring changes.
On a real ring test the old get_more_nodes placed data mostly evenly
across zones, which is a problem for differently weighted zones. But
the real problem was that the extra partitions given to each device
was 0% to 0.77% with only 46.05% of the candidate devices getting
anything. Some of the devices increased in effective weight over 50%
in the test.
The new get_more_nodes placed closer to what the zone weights were
and the extra partitions given to each device was 0% to 0.24% with
90.58% of the candidate devices getting something. The worst off
device only increased in effective weight by 10.71%.
Change-Id: Iffb133a22db69074acaa2b90854cbfa92e4c2b9e

DocImpact
Change-Id: I7edaa5e44208ab451f7f7566b64bb571b8eea1f9

- Prior to this commit, a Content-Disposition header was always set
 on responses to GET requests, with the filename based on the object
 name. Now, the header will only be set for 2xx responses and the
 filename can be overridden with a filename query parameter on the
 request.
- Fixed a bug where all query parameters on the request were being
 passed down the WSGI pipeline. Now, just the query parameters
 useful in log-based debugging are included. This becomes important
 with things like the Bulk middleware that act upon query
 parameters.
- Fixed bug where the Content-Disposition header wasn't following RFC
 spec.
DocImpact
Change-Id: I66ad809321dcdd03444324973c8b76869e3b0c8e

Remove len() check and parenthesis.
Fixes Bug #1136893
Change-Id: Ib47ec4890c2f6a50e316a7fef204ef818c6c4d6e

Fixes bug 1104708
There could be severe performance drop for swift is one disk of one
storage node is problematic due to the tragic state of async disk I/O.
This patch provided PUT timing per kB transfered (ms/kB) monitoring
support for each non-zero-byte request of each disk and report to
statsD for alert.
-adding "object-server.PUT.<device>.timing" metrics for object-server.
DocImpact.
Change-Id: Ie94bddad28e8be52e71683bf6c9db988664abe47

Swift never fsyncs, it only fdatasyncs. That is dumb, we have important
metadata we need to save. Also, the code was weird and had no tests.
Change-Id: I6ec875c14560820b686266a28043a2b7631781e9