openstack/swift - swift - OpenDev: Free Software Needs Free Tools

The trans_id_suffix (default is empty) would be appended to the swift transaction
id allowing one to easily figure out from which cluster that X-Trans-Id
belongs to. This is very useful when one is managing more than one swift
cluster. Also updated sample and manpage to reflect the changes.
Change-Id: Icdf63643e9c1bde36a9ef5e3f41ee9fb20e55f5d

The region is one level above the zone; it is intended to represent a
chunk of machines that is distant from others with respect to
bandwidth and latency.
Old rings will default to having all their devices in region 1. Since
everything is in the same region by default, the ring builder will
simply distribute across zones as it did before, so your partition
assignment won't move because of this change. If you start adding
devices in other regions, of course, the assignment will change to
take that into account.
swift-ring-builder still accepts the same syntax as before, but will
default added devices to region 1 if no region is specified.
Examples:
$ swift-ring-builder foo.builder add r2z1-1.2.3.4:555/sda
$ swift-ring-builder foo.builder add r1z3-1.2.3.4:555/sda
$ swift-ring-builder foo.builder add z3-1.2.3.4:555/sda
Also, some updates to ring-overview doc.
Change-Id: Ifefbb839cdcf033e6c9201fadca95224c7303a29

Add a new middleware implementing account quotas.
This middleware blocks write requests (PUT, POST) if a given quota (in bytes)
is exceeded while DELETE requests are still allowed.
Quotas are stored in the x-account-meta-quota-bytes metadata entry.
Write requests to this metadata setting are only allowed for resellers.
Change-Id: I57fd7c6209f34cc79d4bab72d500d43ba2a62083

Add support for functional tests that work with Apache web front end
Change-Id: I72358a12016eeccc842d834461dbebaa188aa117
Implements: blueprint wsgi-application-interface

Implements blueprint list-endpoints.
DocImpact: new middleware list_endpoints.
Change-Id: I0c4911ff726abd4cb8ce2b6245c99786ad46b410

DocImpact
Change-Id: I7edaa5e44208ab451f7f7566b64bb571b8eea1f9

Fixes bug 1104708
There could be severe performance drop for swift is one disk of one
storage node is problematic due to the tragic state of async disk I/O.
This patch provided PUT timing per kB transfered (ms/kB) monitoring
support for each non-zero-byte request of each disk and report to
statsD for alert.
-adding "object-server.PUT.<device>.timing" metrics for object-server.
DocImpact.
Change-Id: Ie94bddad28e8be52e71683bf6c9db988664abe47

When someone split away python-swiftclient, he left behind the
client manpage.
Apparently all we need now is to delete it from swift and
add it to python-swiftclient. Except that of course the releases
have to be synchronized somehow or else files conflict.
Change-Id: I8b5f7b5557c28818048d8941df36473dacfb1d57

- Things swill go badly with swift if we leave the default to authtoken
 to use its own memcache cache connection based python-memcache c based
 binding.
Change-Id: I293b875acdcb06e5a7a0cfa9a9bb5d7678675da0

Fix bug 1129760
Without speed limit, DB auditor will likely consume high CPU% on
storage node. That will highly impact the cluster's performance.
This patch adds two options for account/container auditor:
 - containers_per_second: Maximum containers audited per second
 - accounts_per_second: Maximum accounts audited per second
DocImpact
Change-Id: I9faa506438185a83ca77db4906969328624d015f

These are mostly cosmetic fixes for irritating imperfections:
- "separated with commas" was duplicated, leave just one
- extra whitespace here and there, man pages are not PEP8, drop
- weird extra commas, drop
- Fedora logs to /var/log/messages
- "drive is has failed", drop "is"
Change-Id: I5ceba2e61b16db4855d76c92cbc83663b9b2a0da

Mostly removed execute on non-executable files
Change-Id: Ibfbe7e0cf0fbeabef602d70b20f75e1dd3bdf9c9

This was an oustanding TODO for StatsD Swift metrics. A new timing
metric is tracked for (only) GET requests for accounts, containers,
and objects:
 proxy-server.<req_type>.GET.<status_int>.first-byte.timing
Also updated StatsD documentation in the Admin Guide to clarify that
timing metrics are sent in units of milliseconds.
Change-Id: I5bb781c06cefcb5280f4fb1112a526c029fe0c20

git ls-files | misspellings -f -
Source: https://github.com/lyda/misspell-check
Change-Id: I4132e6a276e44e2a8985238358533d315ee8d9c4

Replaced GA code for cross-domain tracking.
Patchset addresses reviewer's comments
and follows new guidance from Foundation:
http://wiki.openstack.org/Documentation/Copyright
Adds current year to each Sphinx-built page.
Addresses only the docs copyright attribution, not code files.
Change-Id: Ib90fd1c92c8fafce2db821bc2b17cef1377cfc1e

- Change a bit the formatting of the documention as well.
- Fix WARNING: Title underline too short. in misc.rst.
Change-Id: I2f4e36bcb5e01e984f0af0152bc5b3b9f7e942ce

Reviewed development saio and guidelines while installing
a new dev and test node.
Change-Id: I59238a4d43fa08a89be14f43681ad4693f1261dd

Add a new middleware implementing some basic container quotas.
Quotas are subject to several limitations: eventual consistency, the timeliness
of the cached container_info (60 second ttl by default), and it’s unable to
reject chunked transfer uploads that exceed the quota (though once the quota
is exceeded, new chunked transfers will be refused).
However, they get most of the way to container quotas fairly inexpensively.
Quotas are set by adding meta values to the container, and are validated when
set:
 X-Container-Meta-Quota-Bytes: Maximum size of the container, in bytes.
 X-Container-Meta-Quota-Count: Maximum object count of the container.
DocImpact
Change-Id: I77cfbf6dc231a2e522bd67328e4c082424a93eee

Seems we missed these references when committing
357b12dc2b
DocImpact
Change-Id: Ia226ce1d63e52769bc067d50ec4704cea4e11c5c

Fixes bug 1102319
DocImpact
Change-Id: I8fb0417ab9468e97ed01a6cb1e262630905e7f29

Some systems behave badly when they completely run out of space. To
alleviate this problem, you can set the fallocate_reserve conf value
to a number of bytes to "reserve" on each disk. When the disk free
space falls at or below this amount, fallocate calls will fail, even
if the underlying OS fallocate call would succeed. For example, a
fallocate_reserve of 5368709120 (5G) would make all fallocate calls
fail, even for zero-byte files, when the disk free space falls under
5G.
The default fallocate_reserve is 0, meaning "no reserve", and so the
software behaves exactly as it always has unless you set this conf
value to something non-zero.
Also fixed ring builder's search_devs doc bugs.
Related: To get rsync to do the same, see
https://github.com/rackspace/cloudfiles-rsync
Specifically, see this patch:
https://github.com/rackspace/cloudfiles-rsync/blob/master/debian/patches/limit-fs-fullness.diff
DocImpact
Change-Id: I8db176ae0ca5b41c9bcfeb7cb8abb31c2e614527

If invoked as 'swift-ring-builder-safe' the directory containing the builder
file provided will be locked (via lock_parent_directory()). This provides a
small safe guard against multiple instances of the swift-ring-builder (or
other utilities that observe this lock) from attempting to write to or read
the builder/ring files while operations are in progress.
This is particularly useful in environments where ring management has been
automated (via Chef or custom solutions) but the operator still occasionally
needs to manually interact with the ring.
DocImpact
Change-Id: Ia362744a8151a91bfb586d01da582906726852e6

Fix small problem in ratelimiting middleware.
DocImpact
Change-Id: Ide3e0b9f4887626c30cae0b97eb7e2237b1df3ed

Fix for bug 1095130
* Added a wrapper function around public methods to handle
 CORS actual requests. These requests need to return some
 extra headers to be valid responses to a CORS request.
 Access-Control-Expose-Headers and Access-Control-Allow-Origin.
* Added support for the CORS header Access-Control-Expose-Headers.
* Some refactoring of the OPTIONS method so the
 "is_origin_allowed" logic can be reused.
* Added a little extra detail to the CORS documentation.
DocImpact
Change-Id: I68538e472a900775427f21a8a59e738a83dcc8bc

Removed sidebar with broken (static) links referencing out-of-date docs.
Added an external link to the Swift API docs
fixes bug #1025099
Change-Id: I7f3106175b84b1063f74aa6c5693ab1e422cdb59

- Add ability to specify an alternative run_dir than the default
 /var/run/swift.
- DocImpact
Change-Id: I17677588f2c8da563b7fec2dc4fdc52da87126ed

As Dieter pointed out in bug 1090495
(https://bugs.launchpad.net/swift/+bug/1090495), the volume of metrics
can vary wildly between StatsD metrics.
This patch implements a partial solution by reducing the sample_rate
used for known high-volume metrics (operational experience will need to
inform this over time) and introducing a new tunable,
log_statsd_sample_rate_factor which is multiplied by the sample_rate for
every statsd stat. This tunable can be used to reduce StatsD traffic
proportionally for all metrics and is intended to replace
log_statsd_default_sample_rate, which is left alone for
backward-compatibility, should anyone be using it.
This patch also includes a drive-by fix for log_udp_port which wasn't
being converted to an int (I didn't verify that actually causes trouble
in SysLogHandler(), but it's definitely an improvement regardles).
Change-Id: Id404636e3629f6431cf1c4e64a143959750a3c23

- In howto install multi nodes container-sync was not included in the
 config example.
Change-Id: I2682d47c5cae28ff321ba03d70dae2fea00c276f

Change-Id: I0a1dc9e07f246db3e294f5c62f4ffbd3560b0afd

Changing that complex dd to a functionally equivalent truncate that's easier
to read and modify.
DocImpact
Change-Id: I64404318364608e62a1d80f6a0550271eb4cd03a

Change-Id: I3742c4580d920e09de434e86b75b134c62f1732e

Change-Id: I4d5ba8e58216749c2858dc1c0b8225b3619a7287

You can now give swift-init a -k N (or --kill-wait N) option to
override the default 15 second wait for a process to die after
sending it the die signal. This is useful for boxes that are awfully
slow for whatever reason.
DocImpact
Change-Id: I328ec254f6e0ee1cd423c1d062ba4c5331bd8337