openstack/swift - swift - OpenDev: Free Software Needs Free Tools

If you have 2 swift regions served by the same keystone,
then the client cannot get the correct URL for the swift endpoint
without specifying a region_name.
Closes-Bug: 1587088
Change-Id: Iaab883386e125c3ca6b9554389e63df17267a135

This provides the capability to specify a project_name,
project_domain_name and user_domain_name in /etc/swift/dispersion.conf.
If this values are set in dispersion.conf they get populated to the
swift-client. With this it is possible to have a specific dispersion
project specified, which is not the keystone default domain. Changes
were applied to swift-dispersion-populate and swift-dispersion-report.
Relevant man pages, the example dispersion.conf and the admin guide were
updated accordingly.
DocImpact
Closes-Bug: #1468374
Change-Id: I0e716f8d281b4d0f510bc568bcee4a13fc480ff7

This change is the result of an audit through the config parameters
provided by swift and how/if they are addressed in the swift
documentation. The documentation being the sample config files in
the /etc directory or the documentation.
This change is only concerned with the config files in etc/ next
I will look at the documentation in the doc/ folder.
This change makes the following assumptions:
 - Unless stated otherwise, the commented out parameter in the
 sample configuration is the default for swift.
 - When the default in the code differs from that of the sample
 configuration, the default in the code is correct.
Container reconciler:
 Parameter: interval
 - code: 30
 - config: 300
 Result: config = 30
Object Expirer:
 Parameter: recon_cache_path
 - code: /var/cache/swift
 - config: Parameter missing
 Result: Add parameter
swift-dispersion-populate && swift-dispersion-report
 Parameter: auth_version
 - code: 1.0
 - config: 2.0 (due to being a confusing example of how to setup
 version 2.0).
 Result: Added 'auth_version = 1.0' to the right section (showing
 default and make the sample configuration for auth version
 2.0 easier to understand.
swift-drive-audit:
 Parameter: log_file_pattern
 - code: /var/log/kern.*[!.][!g][!z]
 - config: /var/log/kern*
 Result: config = /var/log/kern.*[!.][!g][!z]
 NOTE: swift-drive-audit uses a parameter called device_dir which
 defaults to '/srv/node'. In all other swift binaries/services
 there is a similar parameter called devices which stores the
 same thing. This is an inconsistency which I haven't fixed
 as this could break existing swift clusters out in the wild.
Proxy Server:
 Parameter: object_chunk_size
 - code: 65536
 - config: 8192
 Result: config = 65536
 Parameter: client_chunk_size
 - code: 65536
 - config: 8192
 Result: config = 65536
 Parameter: strict_cors_mode
 - code: True
 - config: No parameter
 Result: config = True
Account and Container replicator configuration confusion:
 NOTES:
 The account and container replicators have parameters:
 - interval
 - run_pause
 Both of these are loaded into the same variable in code:
 self.interval = int(conf.get('interval') or
 conf.get('run_pause') or 30)
 If a user sets both to different values then interval is used.
 Result: Update the configuration to make this more clear.
DocImpact
Change-Id: Iaadbb1a6284f8b3e0801bc343b29772f70f4bf6e

1. add a comment to hint using a new account for using dispersion tools
2. change sample url for keystone from 'saio' to 'localhost'
Change-Id: I4683f5eb0af534b39112f1b7420f67d569c29b3a

- Makes swift-dispersion-populate a bit faster when using a larger
 dispersion_coverage with a larger part_power.
- Adds option to only run population for container OR objects
- Adds option to let you resume population at given point (useful if you
 need to resume population after a previous run error'd out or the
 like) by specifying which suffix to start at.
The original populate just randomly used uuid4().hex as a suffix on the
container/object names until all the partition's required where covered.
This isn't a big deal if you're only doing 1% coverage on a ring with a
small part power but takes ages if you're doing 100% on a larger ring.
Change-Id: I52f890a774412c1d6179f12db9081aedc58b6bc2

The swift-dispersion-populate and swift-dispersion-report tools now
accept a --insecure option.
Also, dispersion.conf now has a keystone_api_insecure option.
Default is obviously to use the secure path.
DocImpact
Change-Id: I4000352e547d9ce5b08ade54e0c886281caff891

For systems with very large numbers of partitions, 1% dispersion
coverage may simply be too much/take too long. This fix allows <1
values to be used for dispersion_coverage.
DocImpact
Change-Id: I5ed35b69754d55a410e66e658b3854de57c7666b

Fixes bug 1102319
DocImpact
Change-Id: I8fb0417ab9468e97ed01a6cb1e262630905e7f29

- Add two optional flags that let you limit swift-dispersion-report to only
reporting on containers OR objects.
- Also make dispersion.conf and swift-dispersion-report manpages
 current.
DocImpact
Change-Id: Iad56133cad261241db27d0e2103098e3c2f3c245

We're still using saio:11000 in a few spots so a few things
don't work out of the box on the saio. Fixes bug #1024561
Change-Id: I226de54c2785b0d0b681c8d0cc24260adbd3d663

Change-Id: I080d531471d8ea57c69a918d4f6930441f1e69f6
Signed-off-by: Julien Danjou <julien.danjou@enovance.com>

Add's the configuration file option "dump_json" or command line
options [-j|--dump-json] to have swift-dispersion-report output
the report in json format. This allows the dispersion report to
be more easily consumed elsewhere.
There's also a few pep8 fixes and removal of unused imports.
Change-Id: I2374311ccbef43e6bbae24665c9584e60f3da173