openstack/swift - swift - OpenDev: Free Software Needs Free Tools

In a long-term effort to change the recommended ports for Swift,
the first step is to require the bind_port in config files. Later,
we can change the recommended setting.
Anyone currently explicitly setting the ports will not be affected.
Anyone not setting the ports will need to specify them to match their
rings.
DocImpact
Change-Id: Icca83a263acdd0afc9016424a3e9f8c15e944789

Red Hat's QA noticed that in case of the infamous "xattr>=0.4"
error, swift-init exits with a zero error code, which confuses
startup scripts (not Systemd though -- that one knows exactly
if processes run or not).
The easiest fix is to return the error code like Guido's blog
post suggested.
Change-Id: I7badd8742375a7cb2aa5606277316477b8083f8d
Fixes: rhbz#1020480

For this commit, ssync is just a direct replacement for how
we use rsync. Assuming we switch over to ssync completely
someday and drop rsync, we will then be able to improve the
algorithms even further (removing local objects as we
successfully transfer each one rather than waiting for whole
partitions, using an index.db with hash-trees, etc., etc.)
For easier review, this commit can be thought of in distinct
parts:
1) New global_conf_callback functionality for allowing
 services to perform setup code before workers, etc. are
 launched. (This is then used by ssync in the object
 server to create a cross-worker semaphore to restrict
 concurrent incoming replication.)
2) A bit of shifting of items up from object server and
 replicator to diskfile or DEFAULT conf sections for
 better sharing of the same settings. conn_timeout,
 node_timeout, client_timeout, network_chunk_size,
 disk_chunk_size.
3) Modifications to the object server and replicator to
 optionally use ssync in place of rsync. This is done in
 a generic enough way that switching to FutureSync should
 be easy someday.
4) The biggest part, and (at least for now) completely
 optional part, are the new ssync_sender and
 ssync_receiver files. Nice and isolated for easier
 testing and visibility into test coverage, etc.
All the usual logging, statsd, recon, etc. instrumentation
is still there when using ssync, just as it is when using
rsync.
Beyond the essential error and exceptional condition
logging, I have not added any additional instrumentation at
this time. Unless there is something someone finds super
pressing to have added to the logging, I think such
additions would be better as separate change reviews.
FOR NOW, IT IS NOT RECOMMENDED TO USE SSYNC ON PRODUCTION
CLUSTERS. Some of us will be in a limited fashion to look
for any subtle issues, tuning, etc. but generally ssync is
an experimental feature. In its current implementation it is
probably going to be a bit slower than rsync, but if all
goes according to plan it will end up much faster.
There are no comparisions yet between ssync and rsync other
than some raw virtual machine testing I've done to show it
should compete well enough once we can put it in use in the
real world.
If you Tweet, Google+, or whatever, be sure to indicate it's
experimental. It'd be best to keep it out of deployment
guides, howtos, etc. until we all figure out if we like it,
find it to be stable, etc.
Change-Id: If003dcc6f4109e2d2a42f4873a0779110fff16d6

Change-Id: I7c3df47c31759dbeb3105f8883e2688ada848d58
Closes-bug: #1214176 

Change-Id: Ifd909d3561c2647770a7e0caa3cd91acd1b4f298

server/daemon bin scripts; more standized python/linux daemonization
procedures; fixed lp:666957 "devauth server creates auth.db with the wrong
privileges"; new run_daemon helper based on run_wsgi simplifies daemon
launching/testing; new - all servers/daemons support verbose option when
started interactivlty which will log to the console; fixed lp:667839 "can't
start servers with relative paths to configs"; added tests

as a deb install.