Files
388a4edba66238ea031dd0c5cba361e1e9c61640
swift /etc /swift.conf-sample

184 lines
7.4 KiB
Plaintext
Raw Normal View History

[swift-hash]
swift constraints are now settable via config Change previously hard-coded constants into config variables. This allows deployers to tune their cluster more specifically based on their needs. For example, a deployment that uses direct swift access for public content may need to set a larger header value constraint to allow for the full object name to be represented in the Content- Disposition header (which browsers check to determine the name of a downloaded object). The new settings are set in the [swift-constraints] section of /etc/swift/swift.conf. Comments were also added to this config file. Cleaned up swift/common/constraints.py to pass pep8 1.3.3 Funtional tests now require constraints to be defined in /etc/test.conf or in /etc/swift/swift.conf (in the case of running the functional tests against a local swift cluster). To have any hope of tests passing, the defined constraints must match the constraints on the tested cluster. Removed a ton of "magic numbers" in both unit and functional tests. Change-Id: Ie4588e052fd158314ddca6cd8fca9bc793311465
2012年09月05日 20:49:50 -07:00
# swift_hash_path_suffix and swift_hash_path_prefix are used as part of the
# hashing algorithm when determining data placement in the cluster.
# These values should remain secret and MUST NOT change
# once a cluster has been deployed.
# Use only printable chars (python -c "import string; print(string.printable)")
swift constraints are now settable via config Change previously hard-coded constants into config variables. This allows deployers to tune their cluster more specifically based on their needs. For example, a deployment that uses direct swift access for public content may need to set a larger header value constraint to allow for the full object name to be represented in the Content- Disposition header (which browsers check to determine the name of a downloaded object). The new settings are set in the [swift-constraints] section of /etc/swift/swift.conf. Comments were also added to this config file. Cleaned up swift/common/constraints.py to pass pep8 1.3.3 Funtional tests now require constraints to be defined in /etc/test.conf or in /etc/swift/swift.conf (in the case of running the functional tests against a local swift cluster). To have any hope of tests passing, the defined constraints must match the constraints on the tested cluster. Removed a ton of "magic numbers" in both unit and functional tests. Change-Id: Ie4588e052fd158314ddca6cd8fca9bc793311465
2012年09月05日 20:49:50 -07:00
swift_hash_path_suffix = changeme
swift_hash_path_prefix = changeme
swift constraints are now settable via config Change previously hard-coded constants into config variables. This allows deployers to tune their cluster more specifically based on their needs. For example, a deployment that uses direct swift access for public content may need to set a larger header value constraint to allow for the full object name to be represented in the Content- Disposition header (which browsers check to determine the name of a downloaded object). The new settings are set in the [swift-constraints] section of /etc/swift/swift.conf. Comments were also added to this config file. Cleaned up swift/common/constraints.py to pass pep8 1.3.3 Funtional tests now require constraints to be defined in /etc/test.conf or in /etc/swift/swift.conf (in the case of running the functional tests against a local swift cluster). To have any hope of tests passing, the defined constraints must match the constraints on the tested cluster. Removed a ton of "magic numbers" in both unit and functional tests. Change-Id: Ie4588e052fd158314ddca6cd8fca9bc793311465
2012年09月05日 20:49:50 -07:00
Add Storage Policy Support The basic idea here is to replace the use of a single object ring in the Application class with a collection of object rings. The collection includes not only the Ring object itself but the policy name associated with it, the filename for the .gz and any other metadata associated with the policy that may be needed. When containers are created, a policy (thus a specific obj ring) is selected allowing apps to specify policy at container creation time and leverage policies simply by using different containers for object operations. The policy collection is based off of info in the swift.conf file. The format of the sections in the .conf file is as follows: swift.conf format: [storage-policy:0] name = chicken [storage-policy:1] name = turkey default = yes With the above format: - Policy 0 will always be used for access to existing containers without the policy specified. The ring name for policy 0 is always 'object', assuring backwards compatiblity. The parser will always create a policy 0 even if not specified - The policy with 'default=yes' is the one used for new container creation. This allows the admin to specify which policy is used without forcing the application to add the metadata. This commit simply introduces storage policies and the loading thereof; nobody's using it yet. That will follow in subsequent commits. Expose storage policies in /info DocImpact Implements: blueprint storage-policies Change-Id: Ica05f41ecf3adb3648cc9182f11f1c8c5c678985
2014年03月17日 12:18:25 -07:00
# storage policies are defined here and determine various characteristics
# about how objects are stored and treated. Policies are specified by name on
# a per container basis. Names are case-insensitive. The policy index is
# specified in the section header and is used internally. The policy with
# index 0 is always used for legacy containers and can be given a name for use
# in metadata however the ring file name will always be 'object.ring.gz' for
# backwards compatibility. If no policies are defined a policy with index 0
# will be automatically created for backwards compatibility and given the name
# Policy-0. A default policy is used when creating new containers when no
# policy is specified in the request. If no other policies are defined the
# policy with index 0 will be declared the default. If multiple policies are
# defined you must define a policy with index 0 and you must specify a
# default. It is recommended you always define a section for
# storage-policy:0. Aliases are not required when defining a storage policy.
Add support for policy types, 'erasure_coding' policy This patch extends the StoragePolicy class for non-replication storage policies, the first one being "erasure coding". Changes: - Add 'policy_type' support to BaseStoragePolicy class - Disallow direct instantiation of BaseStoragePolicy class - Subclass BaseStoragePolicy - "StoragePolicy": . Replication policy, default . policy_type = 'replication' - "ECStoragePolicy": . Erasure Coding policy . policy_type = 'erasure_coding' . Private member variables ec_type (EC backend), ec_num_data_fragments (number of fragments original data split into after erasure coding operation), ec_num_parity_fragments (number of parity fragments generated during erasure coding) . Private methods EC specific attributes and ring validator methods. - Swift will use PyECLib, a Python Erasure Coding library, for erasure coding operations. PyECLib is already an approved OpenStack core requirement. (https://bitbucket.org/kmgreen2/pyeclib/) - Add test cases for - 'policy_type' StoragePolicy member - policy_type == 'erasure_coding' DocImpact Co-Authored-By: Alistair Coles <alistair.coles@hp.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Paul Luse <paul.e.luse@intel.com> Co-Authored-By: Samuel Merritt <sam@swiftstack.com> Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com> Change-Id: Ie0e09796e3ec45d3e656fb7540d0e5a5709b8386 Implements: blueprint ec-proxy-work
2014年06月30日 11:14:28 -07:00
#
# A 'policy_type' argument is also supported but is not mandatory. Default
# policy type 'replication' is used when 'policy_type' is unspecified.
Add Storage Policy Support The basic idea here is to replace the use of a single object ring in the Application class with a collection of object rings. The collection includes not only the Ring object itself but the policy name associated with it, the filename for the .gz and any other metadata associated with the policy that may be needed. When containers are created, a policy (thus a specific obj ring) is selected allowing apps to specify policy at container creation time and leverage policies simply by using different containers for object operations. The policy collection is based off of info in the swift.conf file. The format of the sections in the .conf file is as follows: swift.conf format: [storage-policy:0] name = chicken [storage-policy:1] name = turkey default = yes With the above format: - Policy 0 will always be used for access to existing containers without the policy specified. The ring name for policy 0 is always 'object', assuring backwards compatiblity. The parser will always create a policy 0 even if not specified - The policy with 'default=yes' is the one used for new container creation. This allows the admin to specify which policy is used without forcing the application to add the metadata. This commit simply introduces storage policies and the loading thereof; nobody's using it yet. That will follow in subsequent commits. Expose storage policies in /info DocImpact Implements: blueprint storage-policies Change-Id: Ica05f41ecf3adb3648cc9182f11f1c8c5c678985
2014年03月17日 12:18:25 -07:00
[storage-policy:0]
name = Policy-0
default = yes
Add support for policy types, 'erasure_coding' policy This patch extends the StoragePolicy class for non-replication storage policies, the first one being "erasure coding". Changes: - Add 'policy_type' support to BaseStoragePolicy class - Disallow direct instantiation of BaseStoragePolicy class - Subclass BaseStoragePolicy - "StoragePolicy": . Replication policy, default . policy_type = 'replication' - "ECStoragePolicy": . Erasure Coding policy . policy_type = 'erasure_coding' . Private member variables ec_type (EC backend), ec_num_data_fragments (number of fragments original data split into after erasure coding operation), ec_num_parity_fragments (number of parity fragments generated during erasure coding) . Private methods EC specific attributes and ring validator methods. - Swift will use PyECLib, a Python Erasure Coding library, for erasure coding operations. PyECLib is already an approved OpenStack core requirement. (https://bitbucket.org/kmgreen2/pyeclib/) - Add test cases for - 'policy_type' StoragePolicy member - policy_type == 'erasure_coding' DocImpact Co-Authored-By: Alistair Coles <alistair.coles@hp.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Paul Luse <paul.e.luse@intel.com> Co-Authored-By: Samuel Merritt <sam@swiftstack.com> Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com> Change-Id: Ie0e09796e3ec45d3e656fb7540d0e5a5709b8386 Implements: blueprint ec-proxy-work
2014年06月30日 11:14:28 -07:00
#policy_type = replication
aliases = yellow, orange
Add Storage Policy Support The basic idea here is to replace the use of a single object ring in the Application class with a collection of object rings. The collection includes not only the Ring object itself but the policy name associated with it, the filename for the .gz and any other metadata associated with the policy that may be needed. When containers are created, a policy (thus a specific obj ring) is selected allowing apps to specify policy at container creation time and leverage policies simply by using different containers for object operations. The policy collection is based off of info in the swift.conf file. The format of the sections in the .conf file is as follows: swift.conf format: [storage-policy:0] name = chicken [storage-policy:1] name = turkey default = yes With the above format: - Policy 0 will always be used for access to existing containers without the policy specified. The ring name for policy 0 is always 'object', assuring backwards compatiblity. The parser will always create a policy 0 even if not specified - The policy with 'default=yes' is the one used for new container creation. This allows the admin to specify which policy is used without forcing the application to add the metadata. This commit simply introduces storage policies and the loading thereof; nobody's using it yet. That will follow in subsequent commits. Expose storage policies in /info DocImpact Implements: blueprint storage-policies Change-Id: Ica05f41ecf3adb3648cc9182f11f1c8c5c678985
2014年03月17日 12:18:25 -07:00
# the following section would declare a policy called 'silver', the number of
# replicas will be determined by how the ring is built. In this example the
# 'silver' policy could have a lower or higher # of replicas than the
# 'Policy-0' policy above. The ring filename will be 'object-1.ring.gz'. You
# may only specify one storage policy section as the default. If you changed
# this section to specify 'silver' as the default, when a client created a new
# container w/o a policy specified, it will get the 'silver' policy because
# this config has specified it as the default. However if a legacy container
# (one created with a pre-policy version of swift) is accessed, it is known
# implicitly to be assigned to the policy with index 0 as opposed to the
# current default. Note that even without specifying any aliases, a policy
# always has at least the default name stored in aliases because this field is
# used to contain all human readable names for a storage policy.
#
Add Storage Policy Support The basic idea here is to replace the use of a single object ring in the Application class with a collection of object rings. The collection includes not only the Ring object itself but the policy name associated with it, the filename for the .gz and any other metadata associated with the policy that may be needed. When containers are created, a policy (thus a specific obj ring) is selected allowing apps to specify policy at container creation time and leverage policies simply by using different containers for object operations. The policy collection is based off of info in the swift.conf file. The format of the sections in the .conf file is as follows: swift.conf format: [storage-policy:0] name = chicken [storage-policy:1] name = turkey default = yes With the above format: - Policy 0 will always be used for access to existing containers without the policy specified. The ring name for policy 0 is always 'object', assuring backwards compatiblity. The parser will always create a policy 0 even if not specified - The policy with 'default=yes' is the one used for new container creation. This allows the admin to specify which policy is used without forcing the application to add the metadata. This commit simply introduces storage policies and the loading thereof; nobody's using it yet. That will follow in subsequent commits. Expose storage policies in /info DocImpact Implements: blueprint storage-policies Change-Id: Ica05f41ecf3adb3648cc9182f11f1c8c5c678985
2014年03月17日 12:18:25 -07:00
#[storage-policy:1]
#name = silver
Add support for policy types, 'erasure_coding' policy This patch extends the StoragePolicy class for non-replication storage policies, the first one being "erasure coding". Changes: - Add 'policy_type' support to BaseStoragePolicy class - Disallow direct instantiation of BaseStoragePolicy class - Subclass BaseStoragePolicy - "StoragePolicy": . Replication policy, default . policy_type = 'replication' - "ECStoragePolicy": . Erasure Coding policy . policy_type = 'erasure_coding' . Private member variables ec_type (EC backend), ec_num_data_fragments (number of fragments original data split into after erasure coding operation), ec_num_parity_fragments (number of parity fragments generated during erasure coding) . Private methods EC specific attributes and ring validator methods. - Swift will use PyECLib, a Python Erasure Coding library, for erasure coding operations. PyECLib is already an approved OpenStack core requirement. (https://bitbucket.org/kmgreen2/pyeclib/) - Add test cases for - 'policy_type' StoragePolicy member - policy_type == 'erasure_coding' DocImpact Co-Authored-By: Alistair Coles <alistair.coles@hp.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Paul Luse <paul.e.luse@intel.com> Co-Authored-By: Samuel Merritt <sam@swiftstack.com> Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com> Change-Id: Ie0e09796e3ec45d3e656fb7540d0e5a5709b8386 Implements: blueprint ec-proxy-work
2014年06月30日 11:14:28 -07:00
#policy_type = replication
# The following declares a storage policy of type 'erasure_coding' which uses
# Erasure Coding for data reliability. Please refer to Swift documentation for
Add support for policy types, 'erasure_coding' policy This patch extends the StoragePolicy class for non-replication storage policies, the first one being "erasure coding". Changes: - Add 'policy_type' support to BaseStoragePolicy class - Disallow direct instantiation of BaseStoragePolicy class - Subclass BaseStoragePolicy - "StoragePolicy": . Replication policy, default . policy_type = 'replication' - "ECStoragePolicy": . Erasure Coding policy . policy_type = 'erasure_coding' . Private member variables ec_type (EC backend), ec_num_data_fragments (number of fragments original data split into after erasure coding operation), ec_num_parity_fragments (number of parity fragments generated during erasure coding) . Private methods EC specific attributes and ring validator methods. - Swift will use PyECLib, a Python Erasure Coding library, for erasure coding operations. PyECLib is already an approved OpenStack core requirement. (https://bitbucket.org/kmgreen2/pyeclib/) - Add test cases for - 'policy_type' StoragePolicy member - policy_type == 'erasure_coding' DocImpact Co-Authored-By: Alistair Coles <alistair.coles@hp.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Paul Luse <paul.e.luse@intel.com> Co-Authored-By: Samuel Merritt <sam@swiftstack.com> Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com> Change-Id: Ie0e09796e3ec45d3e656fb7540d0e5a5709b8386 Implements: blueprint ec-proxy-work
2014年06月30日 11:14:28 -07:00
# details on how the 'erasure_coding' storage policy is implemented.
#
# Swift uses PyECLib, a Python Erasure coding API library, for encode/decode
# operations. Please refer to Swift documentation for details on how to
# install PyECLib.
#
# When defining an EC policy, 'policy_type' needs to be 'erasure_coding' and
# EC configuration parameters 'ec_type', 'ec_num_data_fragments' and
# 'ec_num_parity_fragments' must be specified. 'ec_type' is chosen from the
# list of EC backends supported by PyECLib. The ring configured for the
# storage policy must have it's "replica" count configured to
# 'ec_num_data_fragments' + 'ec_num_parity_fragments' - this requirement is
# validated when services start. 'ec_object_segment_size' is the amount of
# data that will be buffered up before feeding a segment into the
# encoder/decoder. More information about these configuration options and
# supported `ec_type` schemes is available in the Swift documentation. Please
# refer to Swift documentation for details on how to configure EC policies.
#
# The example 'deepfreeze10-4' policy defined below is a _sample_
# configuration with an alias of 'df10-4' as well as 10 'data' and 4 'parity'
# fragments. 'ec_type' defines the Erasure Coding scheme.
# 'liberasurecode_rs_vand' (Reed-Solomon Vandermonde) is used as an example
# below.
Add support for policy types, 'erasure_coding' policy This patch extends the StoragePolicy class for non-replication storage policies, the first one being "erasure coding". Changes: - Add 'policy_type' support to BaseStoragePolicy class - Disallow direct instantiation of BaseStoragePolicy class - Subclass BaseStoragePolicy - "StoragePolicy": . Replication policy, default . policy_type = 'replication' - "ECStoragePolicy": . Erasure Coding policy . policy_type = 'erasure_coding' . Private member variables ec_type (EC backend), ec_num_data_fragments (number of fragments original data split into after erasure coding operation), ec_num_parity_fragments (number of parity fragments generated during erasure coding) . Private methods EC specific attributes and ring validator methods. - Swift will use PyECLib, a Python Erasure Coding library, for erasure coding operations. PyECLib is already an approved OpenStack core requirement. (https://bitbucket.org/kmgreen2/pyeclib/) - Add test cases for - 'policy_type' StoragePolicy member - policy_type == 'erasure_coding' DocImpact Co-Authored-By: Alistair Coles <alistair.coles@hp.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Paul Luse <paul.e.luse@intel.com> Co-Authored-By: Samuel Merritt <sam@swiftstack.com> Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com> Change-Id: Ie0e09796e3ec45d3e656fb7540d0e5a5709b8386 Implements: blueprint ec-proxy-work
2014年06月30日 11:14:28 -07:00
#
#[storage-policy:2]
#name = deepfreeze10-4
#aliases = df10-4
Add support for policy types, 'erasure_coding' policy This patch extends the StoragePolicy class for non-replication storage policies, the first one being "erasure coding". Changes: - Add 'policy_type' support to BaseStoragePolicy class - Disallow direct instantiation of BaseStoragePolicy class - Subclass BaseStoragePolicy - "StoragePolicy": . Replication policy, default . policy_type = 'replication' - "ECStoragePolicy": . Erasure Coding policy . policy_type = 'erasure_coding' . Private member variables ec_type (EC backend), ec_num_data_fragments (number of fragments original data split into after erasure coding operation), ec_num_parity_fragments (number of parity fragments generated during erasure coding) . Private methods EC specific attributes and ring validator methods. - Swift will use PyECLib, a Python Erasure Coding library, for erasure coding operations. PyECLib is already an approved OpenStack core requirement. (https://bitbucket.org/kmgreen2/pyeclib/) - Add test cases for - 'policy_type' StoragePolicy member - policy_type == 'erasure_coding' DocImpact Co-Authored-By: Alistair Coles <alistair.coles@hp.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Paul Luse <paul.e.luse@intel.com> Co-Authored-By: Samuel Merritt <sam@swiftstack.com> Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com> Change-Id: Ie0e09796e3ec45d3e656fb7540d0e5a5709b8386 Implements: blueprint ec-proxy-work
2014年06月30日 11:14:28 -07:00
#policy_type = erasure_coding
#ec_type = liberasurecode_rs_vand
Add support for policy types, 'erasure_coding' policy This patch extends the StoragePolicy class for non-replication storage policies, the first one being "erasure coding". Changes: - Add 'policy_type' support to BaseStoragePolicy class - Disallow direct instantiation of BaseStoragePolicy class - Subclass BaseStoragePolicy - "StoragePolicy": . Replication policy, default . policy_type = 'replication' - "ECStoragePolicy": . Erasure Coding policy . policy_type = 'erasure_coding' . Private member variables ec_type (EC backend), ec_num_data_fragments (number of fragments original data split into after erasure coding operation), ec_num_parity_fragments (number of parity fragments generated during erasure coding) . Private methods EC specific attributes and ring validator methods. - Swift will use PyECLib, a Python Erasure Coding library, for erasure coding operations. PyECLib is already an approved OpenStack core requirement. (https://bitbucket.org/kmgreen2/pyeclib/) - Add test cases for - 'policy_type' StoragePolicy member - policy_type == 'erasure_coding' DocImpact Co-Authored-By: Alistair Coles <alistair.coles@hp.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Paul Luse <paul.e.luse@intel.com> Co-Authored-By: Samuel Merritt <sam@swiftstack.com> Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com> Change-Id: Ie0e09796e3ec45d3e656fb7540d0e5a5709b8386 Implements: blueprint ec-proxy-work
2014年06月30日 11:14:28 -07:00
#ec_num_data_fragments = 10
#ec_num_parity_fragments = 4
#ec_object_segment_size = 1048576
swift constraints are now settable via config Change previously hard-coded constants into config variables. This allows deployers to tune their cluster more specifically based on their needs. For example, a deployment that uses direct swift access for public content may need to set a larger header value constraint to allow for the full object name to be represented in the Content- Disposition header (which browsers check to determine the name of a downloaded object). The new settings are set in the [swift-constraints] section of /etc/swift/swift.conf. Comments were also added to this config file. Cleaned up swift/common/constraints.py to pass pep8 1.3.3 Funtional tests now require constraints to be defined in /etc/test.conf or in /etc/swift/swift.conf (in the case of running the functional tests against a local swift cluster). To have any hope of tests passing, the defined constraints must match the constraints on the tested cluster. Removed a ton of "magic numbers" in both unit and functional tests. Change-Id: Ie4588e052fd158314ddca6cd8fca9bc793311465
2012年09月05日 20:49:50 -07:00
# The swift-constraints section sets the basic constraints on data
Add support for policy types, 'erasure_coding' policy This patch extends the StoragePolicy class for non-replication storage policies, the first one being "erasure coding". Changes: - Add 'policy_type' support to BaseStoragePolicy class - Disallow direct instantiation of BaseStoragePolicy class - Subclass BaseStoragePolicy - "StoragePolicy": . Replication policy, default . policy_type = 'replication' - "ECStoragePolicy": . Erasure Coding policy . policy_type = 'erasure_coding' . Private member variables ec_type (EC backend), ec_num_data_fragments (number of fragments original data split into after erasure coding operation), ec_num_parity_fragments (number of parity fragments generated during erasure coding) . Private methods EC specific attributes and ring validator methods. - Swift will use PyECLib, a Python Erasure Coding library, for erasure coding operations. PyECLib is already an approved OpenStack core requirement. (https://bitbucket.org/kmgreen2/pyeclib/) - Add test cases for - 'policy_type' StoragePolicy member - policy_type == 'erasure_coding' DocImpact Co-Authored-By: Alistair Coles <alistair.coles@hp.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Paul Luse <paul.e.luse@intel.com> Co-Authored-By: Samuel Merritt <sam@swiftstack.com> Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com> Change-Id: Ie0e09796e3ec45d3e656fb7540d0e5a5709b8386 Implements: blueprint ec-proxy-work
2014年06月30日 11:14:28 -07:00
# saved in the swift cluster. These constraints are automatically
# published by the proxy server in responses to /info requests.
swift constraints are now settable via config Change previously hard-coded constants into config variables. This allows deployers to tune their cluster more specifically based on their needs. For example, a deployment that uses direct swift access for public content may need to set a larger header value constraint to allow for the full object name to be represented in the Content- Disposition header (which browsers check to determine the name of a downloaded object). The new settings are set in the [swift-constraints] section of /etc/swift/swift.conf. Comments were also added to this config file. Cleaned up swift/common/constraints.py to pass pep8 1.3.3 Funtional tests now require constraints to be defined in /etc/test.conf or in /etc/swift/swift.conf (in the case of running the functional tests against a local swift cluster). To have any hope of tests passing, the defined constraints must match the constraints on the tested cluster. Removed a ton of "magic numbers" in both unit and functional tests. Change-Id: Ie4588e052fd158314ddca6cd8fca9bc793311465
2012年09月05日 20:49:50 -07:00
[swift-constraints]
# max_file_size is the largest "normal" object that can be saved in
# the cluster. This is also the limit on the size of each segment of
# a "large" object when using the large object manifest support.
# This value is set in bytes. Setting it to lower than 1MiB will cause
# some tests to fail. It is STRONGLY recommended to leave this value at
# the default (5 * 2**30 + 2).
#max_file_size = 5368709122
# max_meta_name_length is the max number of bytes in the utf8 encoding
# of the name portion of a metadata header.
#max_meta_name_length = 128
# max_meta_value_length is the max number of bytes in the utf8 encoding
# of a metadata value
#max_meta_value_length = 256
# max_meta_count is the max number of metadata keys that can be stored
# on a single account, container, or object
#max_meta_count = 90
# max_meta_overall_size is the max number of bytes in the utf8 encoding
# of the metadata (keys + values)
#max_meta_overall_size = 4096
# max_header_size is the max number of bytes in the utf8 encoding of each
Fix Error 400 Header Line Too Long Fix Error 400 Header Line Too Long when using Identity v3 PKI Tokens Uses swift.conf max_header_size option to set wsgi.MAX_HEADER_LINE, allowing the operator to customize this parameter. The default value has been let to 8192 to avoid unexpected configuration change on deployed platforms. The max_header_size option has to be increased (for example to 16384), to accomodate for large Identity v3 PKI tokens, including more than 7 catalog entries. The default max header line size of 8192 is exceeded in the following scenario: - Auth tokens generated by Keystone v3 API include the catalog. - Keystone's catalog contains more than 7 services. Similar fixes have been merged in other projects. Change-Id: Ia838b18331f57dfd02b9f71d4523d4059f38e600 Closes-Bug: 1190149
2014年01月14日 15:58:38 +00:00
# header. Using 8192 as default because eventlet use 8192 as max size of
# header line. This value may need to be increased when using identity
# v3 API tokens including more than 7 catalog entries.
# See also include_service_catalog in proxy-server.conf-sample
# (documented in overview_auth.rst)
#max_header_size = 8192
Increase httplib._MAXHEADERS Python 2.7.9+ and 3.2.6+ limits the number of maximum headers in httplib to 100 [1,2,3]. This setting is too low for Swift. By default the maximum number of allowed headers depends on the number of max allowed metadata settings plus a default value of 32 for regular http headers. If for some reason this is not enough (custom middleware for example) it can be increased with the extra_header_count constraint. [1] https://bugs.python.org/issue16037 [2] https://hg.python.org/cpython/raw-file/15c95b7d81dc/Misc/NEWS [3] https://hg.python.org/cpython/raw-file/v3.2.6/Misc/NEWS Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Matthew Oliver <matt@oliver.net.au> Co-Authored-By: Thomas Herve <therve@redhat.com> Change-Id: I388fd697ec88476024b0e9f1ae75ba35ff765282
2015年06月26日 08:54:22 +00:00
# By default the maximum number of allowed headers depends on the number of max
# allowed metadata settings plus a default value of 32 for regular http
# headers. If for some reason this is not enough (custom middleware for
# example) it can be increased with the extra_header_count constraint.
#extra_header_count = 0
Increase httplib._MAXHEADERS Python 2.7.9+ and 3.2.6+ limits the number of maximum headers in httplib to 100 [1,2,3]. This setting is too low for Swift. By default the maximum number of allowed headers depends on the number of max allowed metadata settings plus a default value of 32 for regular http headers. If for some reason this is not enough (custom middleware for example) it can be increased with the extra_header_count constraint. [1] https://bugs.python.org/issue16037 [2] https://hg.python.org/cpython/raw-file/15c95b7d81dc/Misc/NEWS [3] https://hg.python.org/cpython/raw-file/v3.2.6/Misc/NEWS Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Matthew Oliver <matt@oliver.net.au> Co-Authored-By: Thomas Herve <therve@redhat.com> Change-Id: I388fd697ec88476024b0e9f1ae75ba35ff765282
2015年06月26日 08:54:22 +00:00
swift constraints are now settable via config Change previously hard-coded constants into config variables. This allows deployers to tune their cluster more specifically based on their needs. For example, a deployment that uses direct swift access for public content may need to set a larger header value constraint to allow for the full object name to be represented in the Content- Disposition header (which browsers check to determine the name of a downloaded object). The new settings are set in the [swift-constraints] section of /etc/swift/swift.conf. Comments were also added to this config file. Cleaned up swift/common/constraints.py to pass pep8 1.3.3 Funtional tests now require constraints to be defined in /etc/test.conf or in /etc/swift/swift.conf (in the case of running the functional tests against a local swift cluster). To have any hope of tests passing, the defined constraints must match the constraints on the tested cluster. Removed a ton of "magic numbers" in both unit and functional tests. Change-Id: Ie4588e052fd158314ddca6cd8fca9bc793311465
2012年09月05日 20:49:50 -07:00
# max_object_name_length is the max number of bytes in the utf8 encoding
# of an object name
#max_object_name_length = 1024
# container_listing_limit is the default (and max) number of items
# returned for a container listing request
#container_listing_limit = 10000
# account_listing_limit is the default (and max) number of items returned
# for an account listing request
#account_listing_limit = 10000
# max_account_name_length is the max number of bytes in the utf8 encoding
# of an account name
#max_account_name_length = 256
# max_container_name_length is the max number of bytes in the utf8 encoding
# of a container name
#max_container_name_length = 256
Check if REST API version is valid Swift doesn't check if the used API version is valid. Currently there is only one valid REST API version, but that might change in the future. This patch enforces "v1" or "v1.0" as the version string when accessing account, containers and objects. The list of accepted version strings can be manually overridden using a comma-separated list in swift.conf to make this backward-compatible. The constraint loader has been modified slightly to accept strings as well as integers. Any request to an account, container, and object which does not provide the correct version string will get a 400 BadRequest response. The allowed api versions are by default excluded from /info. Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: John Dickinson <me@not.mn> Closes Bug #1437442 Change-Id: I5ab6e236544378abf2eab562ab759513d09bc256
2015年03月27日 15:50:38 -07:00
# By default all REST API calls should use "v1" or "v1.0" as the version string,
# for example "/v1/account". This can be manually overridden to make this
# backward-compatible, in case a different version string has been used before.
# Use a comma-separated list in case of multiple allowed versions, for example
# valid_api_versions = v0,v1,v2
# This is only enforced for account, container and object requests. The allowed
# api versions are by default excluded from /info.
# valid_api_versions = v1,v1.0