openstack/swift - swift - OpenDev: Free Software Needs Free Tools

Add unit tests to verify the precedence of access_log_ and log_
prefixes to options.
Add pointers from proxy_logging sections in other sample config files
to the proxy-server.conf-sample file.
Change-Id: Id18176d3790fd187e304f0e33e3f74a94dc5305c

This follows the same pattern of configuration used in the
object-expirer. When the container-recociler has a configuration value
for processes it expects that many instances of the reconciler to be
configured with a process value from [0, processes).
Change-Id: Ie46bda37ca3f6e692ec31a4ddcd46f343fb1aeca

Each reconciler process can now reconcile more than one queue entry at a
time, up to the configured concurrency.
By default concurrency is 1. There is no expected change to existing
behavior. Entries are processed serially one a time.
Change-Id: I72e9601b58c2f20bb1294876bb39f2c78827d5f8

This patch makes the reconciler PPI aware. It does this by adding a
helper method `can_reconcile_policy` that is used to check that the
policies used for the source and destination aren't in the middle of a
PPI (their ring doesn't have next_part_power set).
In order to accomplish this the reconciler has had to include the
POLICIES singleton and grown swift_dir and ring_check_interval config options.
Closes-Bug: #1934314
Change-Id: I78a94dd1be90913a7a75d90850ec5ef4a85be4db

Change-Id: I91e9bc02d94fe7ea6e89307305705c383087845a

UpgradeImpact
=============
Operators should verify that encryption is not enabled in their
reconciler pipelines; having it enabled there may harm data durability.
For more information, see https://launchpad.net/bugs/1910804
Change-Id: I1a1d78ed91d940ef0b4eba186dcafd714b4fb808
Closes-Bug: #1910804 

The goal is to modify schedule priority and I/O scheduling class and
priority of daemon/server via configuration.
Setting is optional, default keeps current behaviour.
Use case:
Prioritize object-server to object-auditor, because all user's requests
needed to be served in peak hours and audit could wait.
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
DocImpact
Change-Id: I1018a18f4706daabdb84574ffd9a58d831e68396

Multiple files and documents showed that log_statsd_host had
a default value, usually localhost. This was incorrect, instead
setting a value for log_statsd_host enables statsd logging.
Removed any reference of log_statsd_host having a default value.
Also changed descriptions to show setting a value enables logging.
Change-Id: I3ca5c0e8b8e4981de3aa6db0c476072b5a59723d
Closes-Bug: #1542227 

This change is the result of an audit through the config parameters
provided by swift and how/if they are addressed in the swift
documentation. The documentation being the sample config files in
the /etc directory or the documentation.
This change is only concerned with the config files in etc/ next
I will look at the documentation in the doc/ folder.
This change makes the following assumptions:
 - Unless stated otherwise, the commented out parameter in the
 sample configuration is the default for swift.
 - When the default in the code differs from that of the sample
 configuration, the default in the code is correct.
Container reconciler:
 Parameter: interval
 - code: 30
 - config: 300
 Result: config = 30
Object Expirer:
 Parameter: recon_cache_path
 - code: /var/cache/swift
 - config: Parameter missing
 Result: Add parameter
swift-dispersion-populate && swift-dispersion-report
 Parameter: auth_version
 - code: 1.0
 - config: 2.0 (due to being a confusing example of how to setup
 version 2.0).
 Result: Added 'auth_version = 1.0' to the right section (showing
 default and make the sample configuration for auth version
 2.0 easier to understand.
swift-drive-audit:
 Parameter: log_file_pattern
 - code: /var/log/kern.*[!.][!g][!z]
 - config: /var/log/kern*
 Result: config = /var/log/kern.*[!.][!g][!z]
 NOTE: swift-drive-audit uses a parameter called device_dir which
 defaults to '/srv/node'. In all other swift binaries/services
 there is a similar parameter called devices which stores the
 same thing. This is an inconsistency which I haven't fixed
 as this could break existing swift clusters out in the wild.
Proxy Server:
 Parameter: object_chunk_size
 - code: 65536
 - config: 8192
 Result: config = 65536
 Parameter: client_chunk_size
 - code: 65536
 - config: 8192
 Result: config = 65536
 Parameter: strict_cors_mode
 - code: True
 - config: No parameter
 Result: config = True
Account and Container replicator configuration confusion:
 NOTES:
 The account and container replicators have parameters:
 - interval
 - run_pause
 Both of these are loaded into the same variable in code:
 self.interval = int(conf.get('interval') or
 conf.get('run_pause') or 30)
 If a user sets both to different values then interval is used.
 Result: Update the configuration to make this more clear.
DocImpact
Change-Id: Iaadbb1a6284f8b3e0801bc343b29772f70f4bf6e

This daemon will take objects that are in the wrong storage policy and
move them to the right ones, or delete requests that went to the wrong
storage policy and apply them to the right ones. It operates on a
queue similar to the object-expirer's queue.
Discovering that the object is in the wrong policy will be done in
subsequent commits by the container replicator; this is the daemon
that handles them once they happen.
Like the object expirer, you only need to run one of these per cluster
see etc/container-reconciler.conf.
DocImpact
Implements: blueprint storage-policies
Change-Id: I5ea62eb77ddcbc7cfebf903429f2ee4c098771c9