openstack/swift - swift - OpenDev: Free Software Needs Free Tools

DocImpact
Change-Id: I7edaa5e44208ab451f7f7566b64bb571b8eea1f9

- Prior to this commit, a Content-Disposition header was always set
 on responses to GET requests, with the filename based on the object
 name. Now, the header will only be set for 2xx responses and the
 filename can be overridden with a filename query parameter on the
 request.
- Fixed a bug where all query parameters on the request were being
 passed down the WSGI pipeline. Now, just the query parameters
 useful in log-based debugging are included. This becomes important
 with things like the Bulk middleware that act upon query
 parameters.
- Fixed bug where the Content-Disposition header wasn't following RFC
 spec.
DocImpact
Change-Id: I66ad809321dcdd03444324973c8b76869e3b0c8e

Remove len() check and parenthesis.
Fixes Bug #1136893
Change-Id: Ib47ec4890c2f6a50e316a7fef204ef818c6c4d6e

Fixes bug 1104708
There could be severe performance drop for swift is one disk of one
storage node is problematic due to the tragic state of async disk I/O.
This patch provided PUT timing per kB transfered (ms/kB) monitoring
support for each non-zero-byte request of each disk and report to
statsD for alert.
-adding "object-server.PUT.<device>.timing" metrics for object-server.
DocImpact.
Change-Id: Ie94bddad28e8be52e71683bf6c9db988664abe47

Swift never fsyncs, it only fdatasyncs. That is dumb, we have important
metadata we need to save. Also, the code was weird and had no tests.
Change-Id: I6ec875c14560820b686266a28043a2b7631781e9

Change-Id: I72943a9575b873cef1bf193f197b390b544ed6bf

they don't get misused.
Change-Id: Ie0292df9d9e565ccd608b6b61ead57dfff1d3797

Different versions of syslog-ng and probably other syslog services
handle multi line log messages differently and sometimes quite
poorly. This patch collapses multi line log messages into single
lines before sending them on to syslog.
It's just a copy of what was already in Python's logging.Formatter
but altered to replace the newlines with #012. I used #012 since
that's a convention we've already used elsewhere in Swift.
Change-Id: I8d0509b7cf48e45c2cf6480b51c67eec5bc94fe2

Deleted unused container checks. As method
swift::common::db::AccountBroker::get_container_timestamp becomes
unused, it is deleted too, along with the corresponding tests.
Change-Id: I61de4549b0abd7103226d6a13f1d9844abaa92d3

When someone split away python-swiftclient, he left behind the
client manpage.
Apparently all we need now is to delete it from swift and
add it to python-swiftclient. Except that of course the releases
have to be synchronized somehow or else files conflict.
Change-Id: I8b5f7b5557c28818048d8941df36473dacfb1d57

The function authorize in tempauth.py and keysthoneauth.py is very
important and frequently called in swift. But authorize in tempauth
seems hard to debug with nearly no logs here.
Change-Id: I3300a5a8d02743ff76e2ff86c51338ca24ddddcb
Fixes: bug #1129769 

Change-Id: I7aca3c6cafd9391031f7a10cc233f99e81ee0393

unit test for it.
Some mistakes is in original docstring of that method. There's no unit
test for two methods in swift.common.ring.utils.
Fixes: bug #1070621
Change-Id: I6f4f211ea67d7fb8ccfe659f30bb0f5d394aca6b

1.Correct the usage of initializing a Ring object.
2.Fix typo.
3.Align lines of print_status part. (keep origin author mind)
note: Adam Kijak has commited it at early time, but didn't put it
 in Gerrit. I import his commit and send it in right Workflow.
Fixes: bug #1131901
Change-Id: If56316cc424045f7df662d60088382f7fd40e8b1

Example:
$ swift-ring-builder account.builder set_replicas 4
$ swift-ring-builder rebalance
This is a prerequisite for supporting globally-distributed clusters,
as operators of such clusters will probably want at least as many
replicas as they have regions. Therefore, adding a region requires
adding a replica. Similarly, removing a region lets an operator remove
a replica and save some money on disks.
In order to not hose clusters with lots of data, swift-ring-builder
now allows for setting of fractional replicas. Thus, one can gradually
increase the replica count at a rate that does not adversely affect
cluster performance.
Example:
$ swift-ring-builder object.builder set_replicas 3.01
$ swift-ring-builder object.builder rebalance
<distribute rings and wait>
$ swift-ring-builder object.builder set_replicas 3.02
$ swift-ring-builder object.builder rebalance
<distribute rings and wait>...
Obviously, fractional replicas are nonsensical for a single
partition. A fractional replica count is for the whole ring, not for
any individual partition, and indicates the average number of replicas
of each partition. For example, a replica count of 3.2 means that 20%
of partitions have 4 replicas and 80% have 3 replicas.
Changes do not take effect until after the ring is rebalanced. Thus,
if you mean to go from 3 replicas to 3.01 but you accidentally type
2.01, no data is lost.
Additionally, 'swift-ring-builder X.builder create' can now take a
decimal argument for the number of replicas.
DocImpact
Change-Id: I12b34dacf60350a297a46be493d5d171580243ff

- Things swill go badly with swift if we leave the default to authtoken
 to use its own memcache cache connection based python-memcache c based
 binding.
Change-Id: I293b875acdcb06e5a7a0cfa9a9bb5d7678675da0

- Add missing tests to make it coverage to 100%.
Change-Id: Ic182ad379e2c48cc4151b2532c509bd62d44bd00

- When a user as a valid token it would go to authorize but the acl
 check was after the reseller_check and due fail. Check this before
 reseller_check and add a test for it.
- Fixes bug 1020722.
Change-Id: Iaff9f35f5ee690e9b729c36d05fb9adf3368dc79

Fix bug 1129760
Without speed limit, DB auditor will likely consume high CPU% on
storage node. That will highly impact the cluster's performance.
This patch adds two options for account/container auditor:
 - containers_per_second: Maximum containers audited per second
 - accounts_per_second: Maximum accounts audited per second
DocImpact
Change-Id: I9faa506438185a83ca77db4906969328624d015f

Ensures that any QUERY_STRING to FormPost is not passed onward.
Handles a redirect with a query string properly.
Change-Id: If0a7d9b0a17314c6cd3852175362d4633f828d81

This changes the way primary replicas can be sorted on GET requests.
Previously, replicas were shuffled. Now, if configured, the replicas
are sorted based on the most recent connection time data to that node.
This patch adds a config value that changes the sorting method.
get_more_nodes() (ie handoffs) is unaffected by this patch because
sorting by affinity would break the durability provided by the current
as-unique-as-possible handoff selection.
Timing data is collected for each node each time the proxy makes a
connection to that node (IP address). If timing data for a node doesn't
exist, then it is assumed at -1 (ie will sort earlier) so that timing
data can be collected for that node.
Change-Id: I837fa21c3a566b10cce33eb75788665e1d01cd8a

These are mostly cosmetic fixes for irritating imperfections:
- "separated with commas" was duplicated, leave just one
- extra whitespace here and there, man pages are not PEP8, drop
- weird extra commas, drop
- Fedora logs to /var/log/messages
- "drive is has failed", drop "is"
Change-Id: I5ceba2e61b16db4855d76c92cbc83663b9b2a0da

- Fixes bug 1119282.
- Allow middleware accessing metadata of an account without having to
 store it separately in a new memcache namespace.
- Add tests for get_container_info that was previously missed.
- Add get_account_info method based on get_container_info, a function
 for other middleware to query accounts.
- Rename container_info['count'] as container_info['object_count'].
Change-Id: I43787916c7a812cb08d278edf45370521f12c912

Mostly removed execute on non-executable files
Change-Id: Ibfbe7e0cf0fbeabef602d70b20f75e1dd3bdf9c9

This was an oustanding TODO for StatsD Swift metrics. A new timing
metric is tracked for (only) GET requests for accounts, containers,
and objects:
 proxy-server.<req_type>.GET.<status_int>.first-byte.timing
Also updated StatsD documentation in the Admin Guide to clarify that
timing metrics are sent in units of milliseconds.
Change-Id: I5bb781c06cefcb5280f4fb1112a526c029fe0c20

Change-Id: I2a088e6cfe354d18cb2107a42bc626c2e7822e42

This patch fixes the Swift MemcacheRing set and set_multi
interface incompatible problem with python memcache. The fix
added two extra named parameters to both set and set_multi
method. When only time or timeout parameter is present, then one
of the value will be used. When both time and timeout are present,
the time parameter will be used.
Named parameter min_compress_len is added for pure compatibility
purposes. The current implementation ignores this parameter.
To make swift memcached methods all consistent cross the board,
method incr and decr have also been changed to include a new
named parameter time.
In future OpenStack releases, the named parameter timeout will be
removed, keep the named parameter timeout around for now is
to make sure that mismatched releases between client and server
will still work.
From now on, when a call is made to set, set_multi, decr, incr
by using timeout parametner, a warning message will be logged to
indicate the deprecation of the parameter.
Fixes: bug #1095730
Change-Id: I07af784a54d7d79395fc3265e74145f92f38a893